org.apache.flink.fs.gs.storage

Interface GSBlobStorage

All Known Implementing Classes:

GSBlobStorageImpl

public interface GSBlobStorage

Abstract blob storage, used to simplify interface to Google storage and make it mockable.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static interface	GSBlobStorage.BlobMetadata Abstract blob metadata.
static interface	GSBlobStorage.WriteChannel Abstract blob write channel.

Method Summary

Modifier and Type	Method and Description
void	compose(List<GSBlobIdentifier> sourceBlobIdentifiers, GSBlobIdentifier targetBlobIdentifier) Composes multiple blobs into one.
void	copy(GSBlobIdentifier sourceBlobIdentifier, GSBlobIdentifier targetBlobIdentifier) Copies from a source blob id to a target blob id.
void	createBlob(GSBlobIdentifier blobIdentifier) Create an empty blob.
List<Boolean>	delete(Iterable<GSBlobIdentifier> blobIdentifiers) Deletes blobs.
Optional<GSBlobStorage.BlobMetadata>	getMetadata(GSBlobIdentifier blobIdentifier) Gets blob metadata.
List<GSBlobIdentifier>	list(String bucketName, String prefix) Lists all the blobs in a bucket matching a given prefix.
GSBlobStorage.WriteChannel	writeBlob(GSBlobIdentifier blobIdentifier) Creates a write channel with the default chunk size.
GSBlobStorage.WriteChannel	writeBlob(GSBlobIdentifier blobIdentifier, org.apache.flink.configuration.MemorySize chunkSize) Creates a write channel with the specified chunk size.

Method Detail

writeBlob

GSBlobStorage.WriteChannel writeBlob(GSBlobIdentifier blobIdentifier)

Creates a write channel with the default chunk size.

Parameters:

blobIdentifier - The blob identifier to which to write

Returns:

The WriteChannel helper

writeBlob

GSBlobStorage.WriteChannel writeBlob(GSBlobIdentifier blobIdentifier,
                                     org.apache.flink.configuration.MemorySize chunkSize)

Creates a write channel with the specified chunk size.

Parameters:

blobIdentifier - The blob identifier to which to write

chunkSize - The chunk size, must be > 0

Returns:

The WriteChannel helper

createBlob

void createBlob(GSBlobIdentifier blobIdentifier)

Create an empty blob.

Parameters:

blobIdentifier - The blob to create

getMetadata

Optional<GSBlobStorage.BlobMetadata> getMetadata(GSBlobIdentifier blobIdentifier)

Gets blob metadata.

Parameters:

blobIdentifier - The blob identifier

Returns:

The blob metadata, if the blob exists. Empty if the blob doesn't exist.

list

List<GSBlobIdentifier> list(String bucketName,
                            String prefix)

Lists all the blobs in a bucket matching a given prefix.

Parameters:

bucketName - The bucket name

prefix - The object prefix

Returns:

The found blobs ids

copy

void copy(GSBlobIdentifier sourceBlobIdentifier,
          GSBlobIdentifier targetBlobIdentifier)

Copies from a source blob id to a target blob id. Does not delete the source blob.

Parameters:

sourceBlobIdentifier - The source blob identifier

targetBlobIdentifier - The target glob identifier

compose

void compose(List<GSBlobIdentifier> sourceBlobIdentifiers,
             GSBlobIdentifier targetBlobIdentifier)

Composes multiple blobs into one. Does not delete any of the source blobs.

Parameters:

sourceBlobIdentifiers - The source blob identifiers to combine, max of 32

targetBlobIdentifier - The target blob identifier

delete

List<Boolean> delete(Iterable<GSBlobIdentifier> blobIdentifiers)

Deletes blobs. Note that this does not fail if blobs don't exist.

Parameters:

blobIdentifiers - The blob identifiers to delete

Returns:

The results of each delete operation.