org.apache.flink.core.fs.local

Class LocalRecoverableWriter

java.lang.Object

org.apache.flink.core.fs.local.LocalRecoverableWriter

All Implemented Interfaces:

RecoverableWriter

@Internal
public class LocalRecoverableWriter
extends Object
implements RecoverableWriter

A RecoverableWriter for the LocalFileSystem.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.core.fs.RecoverableWriter

RecoverableWriter.CommitRecoverable, RecoverableWriter.ResumeRecoverable

Constructor Summary

Constructors

Constructor and Description
LocalRecoverableWriter(LocalFileSystem fs)

Method Summary

Modifier and Type	Method and Description
SimpleVersionedSerializer<RecoverableWriter.CommitRecoverable>	getCommitRecoverableSerializer() The serializer for the CommitRecoverable types created in this writer.
SimpleVersionedSerializer<RecoverableWriter.ResumeRecoverable>	getResumeRecoverableSerializer() The serializer for the ResumeRecoverable types created in this writer.
RecoverableFsDataOutputStream	open(Path filePath) Opens a new recoverable stream to write to the given path.
RecoverableFsDataOutputStream	recover(RecoverableWriter.ResumeRecoverable recoverable) Resumes a recoverable stream consistently at the point indicated by the given ResumeRecoverable.
RecoverableFsDataOutputStream.Committer	recoverForCommit(RecoverableWriter.CommitRecoverable recoverable) Recovers a recoverable stream consistently at the point indicated by the given CommitRecoverable for finalizing and committing.
boolean	supportsResume() Checks whether the writer and its streams support resuming (appending to) files after recovery (via the RecoverableWriter.recover(ResumeRecoverable) method).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LocalRecoverableWriter

public LocalRecoverableWriter(LocalFileSystem fs)

Method Detail

open

public RecoverableFsDataOutputStream open(Path filePath)
                                   throws IOException

Description copied from interface: RecoverableWriter

Opens a new recoverable stream to write to the given path. Whether existing files will be overwritten is implementation specific and should not be relied upon.

Specified by:

open in interface RecoverableWriter

Parameters:

filePath - The path of the file/object to write to.

Returns:

A new RecoverableFsDataOutputStream writing a new file/object.

Throws:

IOException - Thrown if the stream could not be opened/initialized.

recover

public RecoverableFsDataOutputStream recover(RecoverableWriter.ResumeRecoverable recoverable)
                                      throws IOException

Description copied from interface: RecoverableWriter

Resumes a recoverable stream consistently at the point indicated by the given ResumeRecoverable. Future writes to the stream will continue / append the file as of that point.

This method is optional and whether it is supported is indicated through the RecoverableWriter.supportsResume() method.

Specified by:

recover in interface RecoverableWriter

Parameters:

recoverable - The opaque handle with the recovery information.

Returns:

A recoverable stream writing to the file/object as it was at the point when the ResumeRecoverable was created.

Throws:

IOException - Thrown, if resuming fails.

recoverForCommit

public RecoverableFsDataOutputStream.Committer recoverForCommit(RecoverableWriter.CommitRecoverable recoverable)
                                                         throws IOException

Description copied from interface: RecoverableWriter

Recovers a recoverable stream consistently at the point indicated by the given CommitRecoverable for finalizing and committing. This will publish the target file with exactly the data that was written up to the point then the CommitRecoverable was created.

Specified by:

recoverForCommit in interface RecoverableWriter

Parameters:

recoverable - The opaque handle with the recovery information.

Returns:

A committer that publishes the target file.

Throws:

IOException - Thrown, if recovery fails.

getCommitRecoverableSerializer

public SimpleVersionedSerializer<RecoverableWriter.CommitRecoverable> getCommitRecoverableSerializer()

Description copied from interface: RecoverableWriter

The serializer for the CommitRecoverable types created in this writer. This serializer should be used to store the CommitRecoverable in checkpoint state or other forms of persistent state.

Specified by:

getCommitRecoverableSerializer in interface RecoverableWriter

getResumeRecoverableSerializer

public SimpleVersionedSerializer<RecoverableWriter.ResumeRecoverable> getResumeRecoverableSerializer()

Description copied from interface: RecoverableWriter

The serializer for the ResumeRecoverable types created in this writer. This serializer should be used to store the ResumeRecoverable in checkpoint state or other forms of persistent state.

Specified by:

getResumeRecoverableSerializer in interface RecoverableWriter

supportsResume

public boolean supportsResume()

Description copied from interface: RecoverableWriter

Checks whether the writer and its streams support resuming (appending to) files after recovery (via the RecoverableWriter.recover(ResumeRecoverable) method).

If true, then this writer supports the RecoverableWriter.recover(ResumeRecoverable) method. If false, then that method may not be supported and streams can only be recovered via RecoverableWriter.recoverForCommit(CommitRecoverable).

Specified by:

supportsResume in interface RecoverableWriter