Package org.apache.iotdb.confignode.procedure.impl.pipe

Class AbstractOperatePipeProcedureV2

java.lang.Object

org.apache.iotdb.confignode.procedure.Procedure<Env>

org.apache.iotdb.confignode.procedure.impl.StateMachineProcedure<ConfigNodeProcedureEnv,TState>

org.apache.iotdb.confignode.procedure.impl.node.AbstractNodeProcedure<OperatePipeTaskState>

org.apache.iotdb.confignode.procedure.impl.pipe.AbstractOperatePipeProcedureV2

All Implemented Interfaces:

Comparable<Procedure<ConfigNodeProcedureEnv>>

Direct Known Subclasses:

AlterPipeProcedureV2, CreatePipeProcedureV2, DropPipeProcedureV2, PipeHandleLeaderChangeProcedure, PipeHandleMetaChangeProcedure, PipeMetaSyncProcedure, StartPipeProcedureV2, StopPipeProcedureV2

public abstract class AbstractOperatePipeProcedureV2
extends AbstractNodeProcedure<OperatePipeTaskState>

This procedure manages 2 kinds of PIPE operations:

• PIPE user task operations:
  • PipeTaskOperation.CREATE_PIPE
  • PipeTaskOperation.START_PIPE
  • PipeTaskOperation.STOP_PIPE
  • PipeTaskOperation.DROP_PIPE
  • PipeTaskOperation.ALTER_PIPE
• PIPE runtime task operations:
  • PipeTaskOperation.HANDLE_LEADER_CHANGE
  • PipeTaskOperation.SYNC_PIPE_META
  • PipeTaskOperation.HANDLE_PIPE_META_CHANGE

This class extends AbstractNodeProcedure to make sure that pipe task procedures can be executed in sequence and node procedures can be locked when a pipe task procedure is running.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.iotdb.confignode.procedure.impl.StateMachineProcedure

StateMachineProcedure.Flow

Field Summary

Fields

Modifier and Type	Field	Description
protected boolean	isRollbackFromOperateOnDataNodesSuccessful
protected boolean	isRollbackFromValidateTaskSuccessful
protected AtomicReference<PipeTaskInfo>	pipeTaskInfo

Fields inherited from class org.apache.iotdb.confignode.procedure.impl.StateMachineProcedure

isGeneratedByPipe

Fields inherited from class org.apache.iotdb.confignode.procedure.Procedure

NO_PROC_ID, NO_TIMEOUT

Constructor Summary

Constructors

Constructor	Description
AbstractOperatePipeProcedureV2()

Method Summary

Modifier and Type	Method	Description
protected ProcedureLockState	acquireLock(ConfigNodeProcedureEnv configNodeProcedureEnv)	Acquire a lock, user should override it if necessary.
protected AtomicReference<PipeTaskInfo>	acquireLockInternal(ConfigNodeProcedureEnv configNodeProcedureEnv)
void	deserialize(ByteBuffer byteBuffer)
protected Map<Integer,TPushPipeMetaResp>	dropSinglePipeOnDataNodes(String pipeName, ConfigNodeProcedureEnv env)	Drop a pipe on all the dataNodes.
abstract void	executeFromCalculateInfoForTask(ConfigNodeProcedureEnv env)	Execute at state OperatePipeTaskState.CALCULATE_INFO_FOR_TASK.
abstract void	executeFromOperateOnDataNodes(ConfigNodeProcedureEnv env)	Execute at state OperatePipeTaskState.OPERATE_ON_DATA_NODES.
protected StateMachineProcedure.Flow	executeFromState(ConfigNodeProcedureEnv env, OperatePipeTaskState state)	Called to perform a single step of the specified 'state' of the procedure.
abstract boolean	executeFromValidateTask(ConfigNodeProcedureEnv env)	Execute at state OperatePipeTaskState.VALIDATE_TASK.
abstract void	executeFromWriteConfigNodeConsensus(ConfigNodeProcedureEnv env)	Execute at state OperatePipeTaskState.WRITE_CONFIG_NODE_CONSENSUS.‘
protected OperatePipeTaskState	getInitialState()	Return the initial state object that will be used for the first call to executeFromState().
protected abstract PipeTaskOperation	getOperation()
protected OperatePipeTaskState	getState(int stateId)	Convert an ordinal (or state id) to an Enum (or more descriptive) state object.
protected int	getStateId(OperatePipeTaskState state)	Convert the Enum (or more descriptive) state object to an ordinal (or state id).
protected boolean	isRollbackSupported(OperatePipeTaskState state)	Used by the default implementation of abort() to know if the current state can be aborted and rollback can be triggered.
static String	parsePushPipeMetaExceptionForPipe(String pipeName, Map<Integer,TPushPipeMetaResp> respMap)	Parsing the given pipe's or all pipes' pushPipeMeta exceptions to string.
protected Map<Integer,TPushPipeMetaResp>	pushPipeMetaToDataNodes(ConfigNodeProcedureEnv env)	Pushing all the pipeMeta's to all the dataNodes, forcing an update to the pipe's runtime state.
static Map<Integer,TPushPipeMetaResp>	pushPipeMetaToDataNodes(ConfigNodeProcedureEnv env, AtomicReference<PipeTaskInfo> pipeTaskInfo)	Pushing all the pipeMeta's to all the dataNodes, forcing an update to the pipe's runtime state.
protected void	pushPipeMetaToDataNodesIgnoreException(ConfigNodeProcedureEnv env)
protected Map<Integer,TPushPipeMetaResp>	pushSinglePipeMetaToDataNodes(String pipeName, ConfigNodeProcedureEnv env)	Pushing one pipeMeta to all the dataNodes, forcing an update to the pipe's runtime state.
protected void	releaseLock(ConfigNodeProcedureEnv configNodeProcedureEnv)	Release a lock, user should override it if necessary.
abstract void	rollbackFromCalculateInfoForTask(ConfigNodeProcedureEnv env)
abstract void	rollbackFromOperateOnDataNodes(ConfigNodeProcedureEnv env)
abstract void	rollbackFromValidateTask(ConfigNodeProcedureEnv env)
abstract void	rollbackFromWriteConfigNodeConsensus(ConfigNodeProcedureEnv env)
protected void	rollbackState(ConfigNodeProcedureEnv env, OperatePipeTaskState state)	Called to perform the rollback of the specified state.
void	serialize(DataOutputStream stream)

Methods inherited from class org.apache.iotdb.confignode.procedure.impl.StateMachineProcedure

addChildProcedure, execute, getCurrentState, getCycles, isEofState, isStateDeserialized, rollback, setNextState, toStringState

Methods inherited from class org.apache.iotdb.confignode.procedure.Procedure

addStackIndex, compareTo, deserializeTypeInfoForCompatibility, doAcquireLock, doExecute, doReleaseLock, doRollback, elapsedTime, getChildrenLatch, getException, getLastUpdate, getParentProcId, getProcId, getProcType, getResult, getRootProcId, getStackIndexes, getState, getSubmittedTime, getTimeout, getTimeoutTimestamp, hasChildren, hasException, hasLock, hasParent, hasTimeout, haveSameParent, holdLock, incChildrenLatch, isFailed, isFinished, isInitializing, isLockedWhenLoading, isRunnable, isSuccess, isWaiting, isYieldAfterExecution, removeStackIndex, restoreLock, setChildrenLatch, setFailure, setFailure, setLastUpdate, setParentProcId, setProcId, setProcRunnable, setResult, setRootProcedureId, setRootProcId, setStackIndexes, setState, setSubmittedTime, setTimeout, setTimeoutFailure, toString, toStringClassDetails, toStringDetails, toStringSimpleSB, updateMetricsOnFinish, updateMetricsOnSubmit, updateTimestamp, wasExecuted

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

isRollbackFromOperateOnDataNodesSuccessful

protected boolean isRollbackFromOperateOnDataNodesSuccessful

isRollbackFromValidateTaskSuccessful

protected boolean isRollbackFromValidateTaskSuccessful

pipeTaskInfo

protected AtomicReference<PipeTaskInfo> pipeTaskInfo

Constructor Details

AbstractOperatePipeProcedureV2

public AbstractOperatePipeProcedureV2()

Method Details

acquireLockInternal

protected AtomicReference<PipeTaskInfo> acquireLockInternal(ConfigNodeProcedureEnv configNodeProcedureEnv)

acquireLock

protected ProcedureLockState acquireLock(ConfigNodeProcedureEnv configNodeProcedureEnv)

Description copied from class: Procedure

Acquire a lock, user should override it if necessary.

Overrides:

acquireLock in class AbstractNodeProcedure<OperatePipeTaskState>

Parameters:

configNodeProcedureEnv - environment

Returns:

state of lock

releaseLock

protected void releaseLock(ConfigNodeProcedureEnv configNodeProcedureEnv)

Description copied from class: Procedure

Release a lock, user should override it if necessary.

Overrides:

releaseLock in class AbstractNodeProcedure<OperatePipeTaskState>

Parameters:

configNodeProcedureEnv - env

getOperation

protected abstract PipeTaskOperation getOperation()

executeFromValidateTask

public abstract boolean executeFromValidateTask(ConfigNodeProcedureEnv env)
                                         throws PipeException

Execute at state OperatePipeTaskState.VALIDATE_TASK.

Returns:

false if this procedure can skip subsequent stages (start RUNNING pipe or stop STOPPED pipe without runtime exception)

Throws:

PipeException - if validation for pipe parameters failed

executeFromCalculateInfoForTask

public abstract void executeFromCalculateInfoForTask(ConfigNodeProcedureEnv env)

Execute at state OperatePipeTaskState.CALCULATE_INFO_FOR_TASK.

executeFromWriteConfigNodeConsensus

public abstract void executeFromWriteConfigNodeConsensus(ConfigNodeProcedureEnv env)
                                                  throws PipeException

Execute at state OperatePipeTaskState.WRITE_CONFIG_NODE_CONSENSUS.‘

Throws:

PipeException - if configNode consensus write failed

executeFromOperateOnDataNodes

public abstract void executeFromOperateOnDataNodes(ConfigNodeProcedureEnv env)
                                            throws PipeException,
IOException

Execute at state OperatePipeTaskState.OPERATE_ON_DATA_NODES.

Throws:

PipeException - if push pipe metas to dataNodes failed

IOException - Exception when Serializing to byte buffer

executeFromState

protected StateMachineProcedure.Flow executeFromState(ConfigNodeProcedureEnv env,
 OperatePipeTaskState state)
                                               throws InterruptedException

Description copied from class: StateMachineProcedure

Called to perform a single step of the specified 'state' of the procedure.

Specified by:

executeFromState in class StateMachineProcedure<ConfigNodeProcedureEnv,OperatePipeTaskState>

state - state to execute

Returns:

Flow.NO_MORE_STATE if the procedure is completed, Flow.HAS_MORE_STATE if there is another step.

Throws:

InterruptedException

isRollbackSupported

protected boolean isRollbackSupported(OperatePipeTaskState state)

Description copied from class: StateMachineProcedure

Used by the default implementation of abort() to know if the current state can be aborted and rollback can be triggered.

Overrides:

isRollbackSupported in class StateMachineProcedure<ConfigNodeProcedureEnv,OperatePipeTaskState>

rollbackState

protected void rollbackState(ConfigNodeProcedureEnv env,
 OperatePipeTaskState state)
                      throws IOException,
InterruptedException,
ProcedureException

Description copied from class: StateMachineProcedure

Called to perform the rollback of the specified state.

Specified by:

rollbackState in class StateMachineProcedure<ConfigNodeProcedureEnv,OperatePipeTaskState>

state - state to rollback

Throws:

IOException - temporary failure, the rollback will retry later

InterruptedException

ProcedureException

rollbackFromValidateTask

public abstract void rollbackFromValidateTask(ConfigNodeProcedureEnv env)

rollbackFromCalculateInfoForTask

public abstract void rollbackFromCalculateInfoForTask(ConfigNodeProcedureEnv env)

rollbackFromWriteConfigNodeConsensus

public abstract void rollbackFromWriteConfigNodeConsensus(ConfigNodeProcedureEnv env)

rollbackFromOperateOnDataNodes

public abstract void rollbackFromOperateOnDataNodes(ConfigNodeProcedureEnv env)
                                             throws IOException

Throws:

IOException

getState

protected OperatePipeTaskState getState(int stateId)

Description copied from class: StateMachineProcedure

Convert an ordinal (or state id) to an Enum (or more descriptive) state object.

Specified by:

getState in class StateMachineProcedure<ConfigNodeProcedureEnv,OperatePipeTaskState>

Parameters:

stateId - the ordinal() of the state enum (or state id)

Returns:

the state enum object

getStateId

protected int getStateId(OperatePipeTaskState state)

Description copied from class: StateMachineProcedure

Convert the Enum (or more descriptive) state object to an ordinal (or state id).

Specified by:

getStateId in class StateMachineProcedure<ConfigNodeProcedureEnv,OperatePipeTaskState>

Parameters:

state - the state enum object

Returns:

stateId the ordinal() of the state enum (or state id)

getInitialState

protected OperatePipeTaskState getInitialState()

Description copied from class: StateMachineProcedure

Return the initial state object that will be used for the first call to executeFromState().

Specified by:

getInitialState in class StateMachineProcedure<ConfigNodeProcedureEnv,OperatePipeTaskState>

Returns:

the initial state enum object

pushPipeMetaToDataNodes

protected Map<Integer,TPushPipeMetaResp> pushPipeMetaToDataNodes(ConfigNodeProcedureEnv env)
                                                          throws IOException

Pushing all the pipeMeta's to all the dataNodes, forcing an update to the pipe's runtime state.

Parameters:

env - ConfigNodeProcedureEnv

Returns:

The responseMap after pushing pipe meta

Throws:

IOException - Exception when Serializing to byte buffer

pushPipeMetaToDataNodes

public static Map<Integer,TPushPipeMetaResp> pushPipeMetaToDataNodes(ConfigNodeProcedureEnv env,
 AtomicReference<PipeTaskInfo> pipeTaskInfo)
                                                              throws IOException

Pushing all the pipeMeta's to all the dataNodes, forcing an update to the pipe's runtime state.

Parameters:

env - ConfigNodeProcedureEnv

pipeTaskInfo - PipeTaskInfo managed outside this procedure

Returns:

The responseMap after pushing pipe meta

Throws:

IOException - Exception when Serializing to byte buffer

parsePushPipeMetaExceptionForPipe

public static String parsePushPipeMetaExceptionForPipe(String pipeName,
 Map<Integer,TPushPipeMetaResp> respMap)

Parsing the given pipe's or all pipes' pushPipeMeta exceptions to string.

Parameters:

pipeName - The given pipe's pipe name, null if report all pipes' exceptions.

respMap - The responseMap after pushing pipe meta

Returns:

Error messages for the given pipe after pushing pipe meta

pushPipeMetaToDataNodesIgnoreException

protected void pushPipeMetaToDataNodesIgnoreException(ConfigNodeProcedureEnv env)

pushSinglePipeMetaToDataNodes

protected Map<Integer,TPushPipeMetaResp> pushSinglePipeMetaToDataNodes(String pipeName,
 ConfigNodeProcedureEnv env)
                                                                throws IOException

Pushing one pipeMeta to all the dataNodes, forcing an update to the pipe's runtime state.

Parameters:

pipeName - pipe name of the pipe to push

env - ConfigNodeProcedureEnv

Returns:

The responseMap after pushing pipe meta

Throws:

IOException - Exception when Serializing to byte buffer

dropSinglePipeOnDataNodes

protected Map<Integer,TPushPipeMetaResp> dropSinglePipeOnDataNodes(String pipeName,
 ConfigNodeProcedureEnv env)

Drop a pipe on all the dataNodes.

Parameters:

pipeName - pipe name of the pipe to drop

env - ConfigNodeProcedureEnv

Returns:

The responseMap after pushing pipe meta

serialize

public void serialize(DataOutputStream stream)
               throws IOException

Overrides:

serialize in class StateMachineProcedure<ConfigNodeProcedureEnv,OperatePipeTaskState>

Throws:

IOException

deserialize

public void deserialize(ByteBuffer byteBuffer)

Overrides:

deserialize in class StateMachineProcedure<ConfigNodeProcedureEnv,OperatePipeTaskState>