org.apache.flink.streaming.api.functions.sink.filesystem

接口 BucketWriter<IN,BucketID>

@Internal
public interface BucketWriter<IN,BucketID>

An interface for factories that create the different writers.

嵌套类概要

嵌套类

限定符和类型	接口和说明
static interface	BucketWriter.PendingFile This represents the file that can not write any data to.

方法概要

限定符和类型	方法和说明
boolean	cleanupInProgressFileRecoverable(InProgressFileWriter.InProgressFileRecoverable inProgressFileRecoverable) Frees up any resources that were previously occupied in order to be able to recover from a (potential) failure.
WriterProperties	getProperties()
InProgressFileWriter<IN,BucketID>	openNewInProgressFile(BucketID bucketID, org.apache.flink.core.fs.Path path, long creationTime) Used to create a new InProgressFileWriter.
BucketWriter.PendingFile	recoverPendingFile(InProgressFileWriter.PendingFileRecoverable pendingFileRecoverable) Recovers a pending file for finalizing and committing.
InProgressFileWriter<IN,BucketID>	resumeInProgressFileFrom(BucketID bucketID, InProgressFileWriter.InProgressFileRecoverable inProgressFileSnapshot, long creationTime) Used to resume a InProgressFileWriter from a InProgressFileWriter.InProgressFileRecoverable.

方法详细资料

openNewInProgressFile

InProgressFileWriter<IN,BucketID> openNewInProgressFile(BucketID bucketID,
                                                        org.apache.flink.core.fs.Path path,
                                                        long creationTime)
                                                 throws IOException

Used to create a new InProgressFileWriter.

参数:

bucketID - the id of the bucket this writer is writing to.

path - the path this writer will write to.

creationTime - the creation time of the file.

返回:

the new InProgressFileWriter

抛出:

IOException - Thrown if creating a writer fails.

resumeInProgressFileFrom

InProgressFileWriter<IN,BucketID> resumeInProgressFileFrom(BucketID bucketID,
                                                           InProgressFileWriter.InProgressFileRecoverable inProgressFileSnapshot,
                                                           long creationTime)
                                                    throws IOException

Used to resume a InProgressFileWriter from a InProgressFileWriter.InProgressFileRecoverable.

参数:

bucketID - the id of the bucket this writer is writing to.

inProgressFileSnapshot - the state of the part file.

creationTime - the creation time of the file.

返回:

the resumed InProgressFileWriter

抛出:

IOException - Thrown if resuming a writer fails.

getProperties

WriterProperties getProperties()

返回:

the property of the BucketWriter

recoverPendingFile

BucketWriter.PendingFile recoverPendingFile(InProgressFileWriter.PendingFileRecoverable pendingFileRecoverable)
                                     throws IOException

Recovers a pending file for finalizing and committing.

参数:

pendingFileRecoverable - The handle with the recovery information.

返回:

A pending file

抛出:

IOException - Thrown if recovering a pending file fails.

cleanupInProgressFileRecoverable

boolean cleanupInProgressFileRecoverable(InProgressFileWriter.InProgressFileRecoverable inProgressFileRecoverable)
                                  throws IOException

Frees up any resources that were previously occupied in order to be able to recover from a (potential) failure.

NOTE: This operation should not throw an exception, but return false if the cleanup did not happen for any reason.

参数:

inProgressFileRecoverable - the InProgressFileWriter.InProgressFileRecoverable whose state we want to clean-up.

返回:

true if the resources were successfully freed, false otherwise (e.g. the file to be deleted was not there for any reason - already deleted or never created).

抛出:

IOException - if an I/O error occurs