程序包 org.apache.seatunnel.engine.server.checkpoint

类 CheckpointManager

java.lang.Object

org.apache.seatunnel.engine.server.checkpoint.CheckpointManager

public class CheckpointManager
extends Object

Used to manage all checkpoints for a job.

Maintain the life cycle of the CheckpointCoordinator through the CheckpointPlan and the status of the job.

构造器概要

构造器

构造器	说明
CheckpointManager(long jobId, boolean isStartWithSavePoint, com.hazelcast.spi.impl.NodeEngine nodeEngine, JobMaster jobMaster, Map<Integer,CheckpointPlan> checkpointPlanMap, org.apache.seatunnel.engine.common.config.server.CheckpointConfig checkpointConfig, ExecutorService executorService, com.hazelcast.map.IMap<Object,Object> runningJobStateIMap)

方法概要

修饰符和类型	方法	说明
void	acknowledgeTask(TaskAcknowledgeOperation ackOperation)	Called by the Task.
org.apache.seatunnel.engine.common.utils.PassiveCompletableFuture<CheckpointCoordinatorState>	cancelCheckpoint(int pipelineId)	Call By JobMaster If all the tasks canceled or some task failed, JobMaster will call this method to cancel checkpoint coordinator.
void	clearCheckpointIfNeed(org.apache.seatunnel.engine.core.job.JobStatus jobStatus)	Called by the JobMaster.
CheckpointCoordinator	getCheckpointCoordinator(int pipelineId)
protected void	handleCheckpointError(int pipelineId, boolean neverRestore)
boolean	isCompletedPipeline(int pipelineId)	Called by the JobMaster.
boolean	isPipelineSavePointEnd(PipelineLocation pipelineLocation)
boolean	isSavePointEnd()
CompletableFuture<Void>	listenPipeline(int pipelineId, org.apache.seatunnel.engine.core.job.PipelineStatus pipelineStatus)	Called by the JobMaster.
void	readyToClose(TaskLocation taskLocation)	Called by the SourceSplitEnumeratorTask.
void	readyToCloseIdleTask(TaskLocation taskLocation)	Called by the SourceSplitEnumeratorTask.
void	reportCheckpointErrorFromTask(TaskLocation taskLocation, String errorMsg)
void	reportedPipelineRunning(int pipelineId, boolean alreadyStarted)
void	reportedTask(TaskReportStatusOperation reportStatusOperation)	Called by the Task.
protected com.hazelcast.spi.impl.operationservice.impl.InvocationFuture<?>	sendOperationToMemberNode(TaskOperation operation)
org.apache.seatunnel.engine.common.utils.PassiveCompletableFuture<CompletedCheckpoint>[]	triggerSavePoints()	Called by the JobMaster, actually triggered by the user.
void	triggerSchemaChangeAfterCheckpoint(TriggerSchemaChangeAfterCheckpointOperation operation)
void	triggerSchemaChangeBeforeCheckpoint(TriggerSchemaChangeBeforeCheckpointOperation operation)
org.apache.seatunnel.engine.common.utils.PassiveCompletableFuture<CheckpointCoordinatorState>	waitCheckpointCoordinatorComplete(int pipelineId)	Call By JobMaster If all the tasks is finished, JobMaster will call this method to wait checkpoint coordinator complete.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

CheckpointManager

public CheckpointManager(long jobId,
                         boolean isStartWithSavePoint,
                         com.hazelcast.spi.impl.NodeEngine nodeEngine,
                         JobMaster jobMaster,
                         Map<Integer,CheckpointPlan> checkpointPlanMap,
                         org.apache.seatunnel.engine.common.config.server.CheckpointConfig checkpointConfig,
                         ExecutorService executorService,
                         com.hazelcast.map.IMap<Object,Object> runningJobStateIMap)
                  throws org.apache.seatunnel.engine.checkpoint.storage.exception.CheckpointStorageException

抛出:

org.apache.seatunnel.engine.checkpoint.storage.exception.CheckpointStorageException

方法详细资料

triggerSavePoints

public org.apache.seatunnel.engine.common.utils.PassiveCompletableFuture<CompletedCheckpoint>[] triggerSavePoints()

Called by the JobMaster, actually triggered by the user.
After the savepoint is triggered, it will cause the job to stop automatically.

reportedPipelineRunning

public void reportedPipelineRunning(int pipelineId,
                                    boolean alreadyStarted)

handleCheckpointError

protected void handleCheckpointError(int pipelineId,
                                     boolean neverRestore)

reportCheckpointErrorFromTask

public void reportCheckpointErrorFromTask(TaskLocation taskLocation,
                                          String errorMsg)

getCheckpointCoordinator

public CheckpointCoordinator getCheckpointCoordinator(int pipelineId)

reportedTask

public void reportedTask(TaskReportStatusOperation reportStatusOperation)

Called by the Task.
used by Task to report the SeaTunnelTaskState of the state machine.

readyToClose

public void readyToClose(TaskLocation taskLocation)

Called by the SourceSplitEnumeratorTask.
used by SourceSplitEnumeratorTask to tell CheckpointCoordinator pipeline will trigger close barrier by SourceSplitEnumeratorTask.

readyToCloseIdleTask

public void readyToCloseIdleTask(TaskLocation taskLocation)

Called by the SourceSplitEnumeratorTask.
used by SourceSplitEnumeratorTask to tell CheckpointCoordinator pipeline will trigger close barrier of idle task by SourceSplitEnumeratorTask.

listenPipeline

public CompletableFuture<Void> listenPipeline(int pipelineId,
                                              org.apache.seatunnel.engine.core.job.PipelineStatus pipelineStatus)

Called by the JobMaster.
Listen to the PipelineStatus of the Pipeline, which is used to shut down the running CheckpointIDCounter at the end of the pipeline.

clearCheckpointIfNeed

public void clearCheckpointIfNeed(org.apache.seatunnel.engine.core.job.JobStatus jobStatus)

Called by the JobMaster.
Listen to the JobStatus of the Job.

isCompletedPipeline

public boolean isCompletedPipeline(int pipelineId)

Called by the JobMaster.
Returns whether the pipeline has completed; No need to deploy/restore the SubPlan if the pipeline has been completed;

acknowledgeTask

public void acknowledgeTask(TaskAcknowledgeOperation ackOperation)

Called by the Task.
used for the ack of the checkpoint, including the state snapshot of all Action within the Task.

triggerSchemaChangeBeforeCheckpoint

public void triggerSchemaChangeBeforeCheckpoint(TriggerSchemaChangeBeforeCheckpointOperation operation)

triggerSchemaChangeAfterCheckpoint

public void triggerSchemaChangeAfterCheckpoint(TriggerSchemaChangeAfterCheckpointOperation operation)

isSavePointEnd

public boolean isSavePointEnd()

isPipelineSavePointEnd

public boolean isPipelineSavePointEnd(PipelineLocation pipelineLocation)

sendOperationToMemberNode

protected com.hazelcast.spi.impl.operationservice.impl.InvocationFuture<?> sendOperationToMemberNode(TaskOperation operation)

cancelCheckpoint

public org.apache.seatunnel.engine.common.utils.PassiveCompletableFuture<CheckpointCoordinatorState> cancelCheckpoint(int pipelineId)

Call By JobMaster If all the tasks canceled or some task failed, JobMaster will call this method to cancel checkpoint coordinator.

参数:

pipelineId -

返回:

waitCheckpointCoordinatorComplete

public org.apache.seatunnel.engine.common.utils.PassiveCompletableFuture<CheckpointCoordinatorState> waitCheckpointCoordinatorComplete(int pipelineId)

Call By JobMaster If all the tasks is finished, JobMaster will call this method to wait checkpoint coordinator complete.

参数:

pipelineId -

返回: