org.apache.flink.fs.gs.storage

Class GSBlobStorageImpl

java.lang.Object

org.apache.flink.fs.gs.storage.GSBlobStorageImpl

All Implemented Interfaces:

GSBlobStorage

public class GSBlobStorageImpl
extends Object
implements GSBlobStorage

BlobStorage implementation for Google storage.

Constructor Summary

Constructors

Constructor and Description
GSBlobStorageImpl(com.google.cloud.storage.Storage storage) Constructs a GSBlobStorage instance.

Method Summary

Modifier and Type	Method and Description
void	compose(List<GSBlobIdentifier> sourceBlobIdentifiers, GSBlobIdentifier targetBlobIdentifier) Composes multiple blobs into one.
void	copy(GSBlobIdentifier sourceBlobIdentifier, GSBlobIdentifier targetBlobIdentifier) Copies from a source blob id to a target blob id.
void	createBlob(GSBlobIdentifier blobIdentifier) Create an empty blob.
List<Boolean>	delete(Iterable<GSBlobIdentifier> blobIdentifiers) Deletes blobs.
Optional<GSBlobStorage.BlobMetadata>	getMetadata(GSBlobIdentifier blobIdentifier) Gets blob metadata.
List<GSBlobIdentifier>	list(String bucketName, String objectPrefix) Lists all the blobs in a bucket matching a given prefix.
GSBlobStorage.WriteChannel	writeBlob(GSBlobIdentifier blobIdentifier) Creates a write channel with the default chunk size.
GSBlobStorage.WriteChannel	writeBlob(GSBlobIdentifier blobIdentifier, org.apache.flink.configuration.MemorySize chunkSize) Creates a write channel with the specified chunk size.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GSBlobStorageImpl

public GSBlobStorageImpl(com.google.cloud.storage.Storage storage)

Constructs a GSBlobStorage instance.

Parameters:

storage - The wrapped Google Storage instance.

Method Detail

writeBlob

public GSBlobStorage.WriteChannel writeBlob(GSBlobIdentifier blobIdentifier)

Description copied from interface: GSBlobStorage

Creates a write channel with the default chunk size.

Specified by:

writeBlob in interface GSBlobStorage

Parameters:

blobIdentifier - The blob identifier to which to write

Returns:

The WriteChannel helper

writeBlob

public GSBlobStorage.WriteChannel writeBlob(GSBlobIdentifier blobIdentifier,
                                            org.apache.flink.configuration.MemorySize chunkSize)

Description copied from interface: GSBlobStorage

Creates a write channel with the specified chunk size.

Specified by:

writeBlob in interface GSBlobStorage

Parameters:

blobIdentifier - The blob identifier to which to write

chunkSize - The chunk size, must be > 0

Returns:

The WriteChannel helper

createBlob

public void createBlob(GSBlobIdentifier blobIdentifier)

Description copied from interface: GSBlobStorage

Create an empty blob.

Specified by:

createBlob in interface GSBlobStorage

Parameters:

blobIdentifier - The blob to create

getMetadata

public Optional<GSBlobStorage.BlobMetadata> getMetadata(GSBlobIdentifier blobIdentifier)

Description copied from interface: GSBlobStorage

Gets blob metadata.

Specified by:

getMetadata in interface GSBlobStorage

Parameters:

blobIdentifier - The blob identifier

Returns:

The blob metadata, if the blob exists. Empty if the blob doesn't exist.

list

public List<GSBlobIdentifier> list(String bucketName,
                                   String objectPrefix)

Description copied from interface: GSBlobStorage

Lists all the blobs in a bucket matching a given prefix.

Specified by:

list in interface GSBlobStorage

Parameters:

bucketName - The bucket name

objectPrefix - The object prefix

Returns:

The found blobs ids

copy

public void copy(GSBlobIdentifier sourceBlobIdentifier,
                 GSBlobIdentifier targetBlobIdentifier)

Description copied from interface: GSBlobStorage

Copies from a source blob id to a target blob id. Does not delete the source blob.

Specified by:

copy in interface GSBlobStorage

Parameters:

sourceBlobIdentifier - The source blob identifier

targetBlobIdentifier - The target glob identifier

compose

public void compose(List<GSBlobIdentifier> sourceBlobIdentifiers,
                    GSBlobIdentifier targetBlobIdentifier)

Description copied from interface: GSBlobStorage

Composes multiple blobs into one. Does not delete any of the source blobs.

Specified by:

compose in interface GSBlobStorage

Parameters:

sourceBlobIdentifiers - The source blob identifiers to combine, max of 32

targetBlobIdentifier - The target blob identifier

delete

public List<Boolean> delete(Iterable<GSBlobIdentifier> blobIdentifiers)

Description copied from interface: GSBlobStorage

Deletes blobs. Note that this does not fail if blobs don't exist.

Specified by:

delete in interface GSBlobStorage

Parameters:

blobIdentifiers - The blob identifiers to delete

Returns:

The results of each delete operation.