org.apache.bookkeeper.mledger

Interface LedgerOffloader

All Known Implementing Classes:

NullLedgerOffloader

@InterfaceAudience.LimitedPrivate
 @InterfaceStability.Evolving
public interface LedgerOffloader

Interface for offloading ledgers to long-term storage.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static interface	LedgerOffloader.OffloadHandle Used to store driver info, buffer entries, mark progress, etc.
static class	LedgerOffloader.OffloadResult

Field Summary

Fields

Modifier and Type	Field and Description
static String	METADATA_SOFTWARE_GITSHA_KEY
static String	METADATA_SOFTWARE_VERSION_KEY

Method Summary

Modifier and Type	Method and Description
void	close() Close the resources if necessary
CompletableFuture<Void>	deleteOffloaded(long ledgerId, UUID uid, Map<String,String> offloadDriverMetadata) Delete a ledger from long term storage.
default CompletableFuture<Void>	deleteOffloaded(UUID uid, Map<String,String> offloadDriverMetadata)
default Map<String,String>	getOffloadDriverMetadata() Get offload driver metadata.
String	getOffloadDriverName() Get offload driver name.
OffloadPolicies	getOffloadPolicies() Get offload policies of this LedgerOffloader
CompletableFuture<Void>	offload(org.apache.bookkeeper.client.api.ReadHandle ledger, UUID uid, Map<String,String> extraMetadata) Offload the passed in ledger to longterm storage.
default CompletableFuture<org.apache.bookkeeper.client.api.ReadHandle>	readOffloaded(long ledgerId, MLDataFormats.OffloadContext ledgerContext, Map<String,String> offloadDriverMetadata)
CompletableFuture<org.apache.bookkeeper.client.api.ReadHandle>	readOffloaded(long ledgerId, UUID uid, Map<String,String> offloadDriverMetadata) Create a ReadHandle which can be used to read a ledger back from longterm storage.
default CompletableFuture<LedgerOffloader.OffloadHandle>	streamingOffload(ManagedLedger ml, UUID uid, long beginLedger, long beginEntry, Map<String,String> driverMetadata) Begin offload the passed in ledgers to longterm storage, it will finish when a segment reached it's size or time.

Field Detail

METADATA_SOFTWARE_VERSION_KEY

static final String METADATA_SOFTWARE_VERSION_KEY

See Also:

Constant Field Values

METADATA_SOFTWARE_GITSHA_KEY

static final String METADATA_SOFTWARE_GITSHA_KEY

See Also:

Constant Field Values

Method Detail

getOffloadDriverName

String getOffloadDriverName()

Get offload driver name.

Returns:

offload driver name.

getOffloadDriverMetadata

default Map<String,String> getOffloadDriverMetadata()

Get offload driver metadata.

The driver metadata will be recorded as part of the metadata of the original ledger.

Returns:

offload driver metadata.

offload

CompletableFuture<Void> offload(org.apache.bookkeeper.client.api.ReadHandle ledger,
                                UUID uid,
                                Map<String,String> extraMetadata)

Offload the passed in ledger to longterm storage. Metadata passed in is for inspection purposes only and should be stored alongside the ledger data. When the returned future completes, the ledger has been persisted to the longterm storage, so it is safe to delete the original copy in bookkeeper. The uid is used to identify an attempt to offload. The implementation should use this to deterministically generate a unique name for the offloaded object. This uid will be stored in the managed ledger metadata before attempting the call to offload(). If a subsequent or concurrent call to offload() finds a uid in the metadata, it will attempt to cleanup this attempt with a call to #deleteOffloaded(ReadHandle,UUID). Once the offload attempt completes, the managed ledger will update its metadata again, to record the completion, ensuring that subsequent calls will not attempt to offload the same ledger again.

Parameters:

ledger - the ledger to offload

uid - unique id to identity this offload attempt

extraMetadata - metadata to be stored with the offloaded ledger for informational purposes

Returns:

a future, which when completed, denotes that the offload has been successful.

streamingOffload

default CompletableFuture<LedgerOffloader.OffloadHandle> streamingOffload(ManagedLedger ml,
                                                                          UUID uid,
                                                                          long beginLedger,
                                                                          long beginEntry,
                                                                          Map<String,String> driverMetadata)

Begin offload the passed in ledgers to longterm storage, it will finish when a segment reached it's size or time. Should only be called once for a LedgerOffloader instance. Metadata passed in is for inspection purposes only and should be stored alongside the segment data. When the returned OffloaderHandle.getOffloadResultAsync completes, the corresponding ledgers has been persisted to the longterm storage, so it is safe to delete the original copy in bookkeeper. The uid is used to identify an attempt to offload. The implementation should use this to deterministically generate a unique name for the offloaded object. This uid will be stored in the managed ledger metadata before attempting the call to streamingOffload(). If a subsequent or concurrent call to streamingOffload() finds a uid in the metadata, it will attempt to cleanup this attempt with a call to #deleteOffloaded(ReadHandle,UUID). Once the offload attempt completes, the managed ledger will update its metadata again, to record the completion, ensuring that subsequent calls will not attempt to offload the same ledger again.

Returns:

an OffloaderHandle, which when `completeFuture()` completed, denotes that the offload has been successful.

readOffloaded

CompletableFuture<org.apache.bookkeeper.client.api.ReadHandle> readOffloaded(long ledgerId,
                                                                             UUID uid,
                                                                             Map<String,String> offloadDriverMetadata)

Create a ReadHandle which can be used to read a ledger back from longterm storage. The passed uid, will be match the uid of a previous successful call to #offload(ReadHandle,UUID,Map).

Parameters:

ledgerId - the ID of the ledger to load from longterm storage

uid - unique ID for previous successful offload attempt

offloadDriverMetadata - offload driver metadata

Returns:

a future, which when completed, returns a ReadHandle

deleteOffloaded

CompletableFuture<Void> deleteOffloaded(long ledgerId,
                                        UUID uid,
                                        Map<String,String> offloadDriverMetadata)

Delete a ledger from long term storage. The passed uid, will be match the uid of a previous call to #offload(ReadHandle,UUID,Map), which may or may not have been successful.

Parameters:

ledgerId - the ID of the ledger to delete from longterm storage

uid - unique ID for previous offload attempt

offloadDriverMetadata - offload driver metadata

Returns:

a future, which when completed, signifies that the ledger has been deleted

readOffloaded

default CompletableFuture<org.apache.bookkeeper.client.api.ReadHandle> readOffloaded(long ledgerId,
                                                                                     MLDataFormats.OffloadContext ledgerContext,
                                                                                     Map<String,String> offloadDriverMetadata)

deleteOffloaded

default CompletableFuture<Void> deleteOffloaded(UUID uid,
                                                Map<String,String> offloadDriverMetadata)

getOffloadPolicies

OffloadPolicies getOffloadPolicies()

Get offload policies of this LedgerOffloader

Returns:

offload policies

close

void close()

Close the resources if necessary