org.apache.flink.streaming.runtime.operators

类 CheckpointCommitter

java.lang.Object

org.apache.flink.streaming.runtime.operators.CheckpointCommitter

所有已实现的接口:

Serializable

public abstract class CheckpointCommitter
extends Object
implements Serializable

This class is used to save information about which sink operator instance has committed checkpoints to a backend.

The current checkpointing mechanism is ill-suited for sinks relying on backends that do not support roll-backs. When dealing with such a system, while trying to get exactly-once semantics, one may neither commit data while creating the snapshot (since another sink instance may fail, leading to a replay on the same data) nor when receiving a checkpoint-complete notification (since a subsequent failure would leave us with no knowledge as to whether data was committed or not).

A CheckpointCommitter can be used to solve the second problem by saving whether an instance committed all data belonging to a checkpoint. This data must be stored in a backend that is persistent across retries (which rules out Flink's state mechanism) and accessible from all machines, like a database or distributed file.

There is no mandate as to how the resource is shared; there may be one resource for all Flink jobs, or one for each job/operator/-instance separately. This implies that the resource must not be cleaned up by the system itself, and as such should kept as small as possible.

另请参阅:

序列化表格

字段概要

字段

限定符和类型	字段和说明
protected String	jobId
protected static org.slf4j.Logger	LOG
protected String	operatorId

构造器概要

构造器

构造器和说明
CheckpointCommitter()

方法概要

限定符和类型	方法和说明
abstract void	close() Closes the resource/connection to it.
abstract void	commitCheckpoint(int subtaskIdx, long checkpointID) Mark the given checkpoint as completed in the resource.
abstract void	createResource() Creates/opens/connects to the resource that is used to store information.
abstract boolean	isCheckpointCommitted(int subtaskIdx, long checkpointID) Checked the resource whether the given checkpoint was committed completely.
abstract void	open() Opens/connects to the resource, and possibly creates it beforehand.
void	setJobId(String id) Internally used to set the job ID after instantiation.
void	setOperatorId(String id) Internally used to set the operator ID after instantiation.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

LOG

protected static final org.slf4j.Logger LOG

jobId

protected String jobId

operatorId

protected String operatorId

构造器详细资料

CheckpointCommitter

public CheckpointCommitter()

方法详细资料

setJobId

public void setJobId(String id)
              throws Exception

Internally used to set the job ID after instantiation.

参数:

id -

抛出:

Exception

setOperatorId

public void setOperatorId(String id)
                   throws Exception

Internally used to set the operator ID after instantiation.

参数:

id -

抛出:

Exception

open

public abstract void open()
                   throws Exception

Opens/connects to the resource, and possibly creates it beforehand.

抛出:

Exception

close

public abstract void close()
                    throws Exception

Closes the resource/connection to it. The resource should generally still exist after this call.

抛出:

Exception

createResource

public abstract void createResource()
                             throws Exception

Creates/opens/connects to the resource that is used to store information. Called once directly after instantiation.

抛出:

Exception

commitCheckpoint

public abstract void commitCheckpoint(int subtaskIdx,
                                      long checkpointID)
                               throws Exception

Mark the given checkpoint as completed in the resource.

参数:

subtaskIdx - the index of the subtask responsible for committing the checkpoint.

checkpointID - the id of the checkpoint to be committed.

抛出:

Exception

isCheckpointCommitted

public abstract boolean isCheckpointCommitted(int subtaskIdx,
                                              long checkpointID)
                                       throws Exception

Checked the resource whether the given checkpoint was committed completely.

参数:

subtaskIdx - the index of the subtask responsible for committing the checkpoint.

checkpointID - the id of the checkpoint we are interested in.

返回:

true if the checkpoint was committed completely, false otherwise

抛出:

Exception