org.apache.flink.compiler.dag

Class DataSourceNode

java.lang.Object

org.apache.flink.compiler.dag.OptimizerNode

org.apache.flink.compiler.dag.DataSourceNode

All Implemented Interfaces:

EstimateProvider, DumpableNode<OptimizerNode>, Visitable<OptimizerNode>

public class DataSourceNode
extends OptimizerNode

The optimizer's internal representation of a data source.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.compiler.dag.OptimizerNode

OptimizerNode.UnclosedBranchDescriptor

Field Summary

Fields inherited from class org.apache.flink.compiler.dag.OptimizerNode

cachedPlans, closedBranchingNodes, costWeight, estimatedNumRecords, estimatedOutputSize, hereJoinedBranches, id, MAX_DYNAMIC_PATH_COST_WEIGHT, onDynamicPath, openBranches, remappedKeys, uniqueFields

Constructor Summary

Constructors

Constructor and Description
DataSourceNode(GenericDataSourceBase<?,?> pactContract) Creates a new DataSourceNode for the given contract.

Method Summary

Modifier and Type	Method and Description
void	accept(Visitor<OptimizerNode> visitor) This method implements the visit of a depth-first graph traversing visitor.
void	computeInterestingPropertiesForInputs(CostEstimator estimator) Tells the node to compute the interesting properties for its inputs.
protected void	computeOperatorSpecificDefaultEstimates(DataStatistics statistics)
void	computeUnclosedBranchStack() This method causes the node to compute the description of open branches in its sub-plan.
List<PlanNode>	getAlternativePlans(CostEstimator estimator) Computes the plan alternatives for this node, an implicitly for all nodes that are children of this node.
List<PactConnection>	getIncomingConnections() This method needs to be overridden by subclasses to return the children.
String	getName() Gets the name of this node.
GenericDataSourceBase<?,?>	getPactContract() Gets the contract object for this data source node.
boolean	isFieldConstant(int input, int fieldNumber) Checks whether a field is modified by the user code or whether it is kept unchanged.
void	setDegreeOfParallelism(int degreeOfParallelism) Sets the degree of parallelism for the contract represented by this optimizer node.
void	setInput(Map<Operator<?>,OptimizerNode> contractToNode) This function is for plan translation purposes.

Methods inherited from class org.apache.flink.compiler.dag.OptimizerNode

addBroadcastConnection, addClosedBranch, addClosedBranches, addOutgoingConnection, areBranchCompatible, clearInterestingProperties, computeOutputEstimates, computeUnclosedBranchStackForBroadcastInputs, computeUnionOfInterestingPropertiesFromSuccessors, createUniqueFieldsForNode, getBranchesForParent, getBroadcastConnectionNames, getBroadcastConnections, getClosedBranchingNodes, getConstantKeySet, getCostWeight, getDegreeOfParallelism, getDumpableInputs, getEstimatedAvgWidthPerOutputRecord, getEstimatedNumRecords, getEstimatedOutputSize, getId, getInterestingProperties, getMaxDepth, getMinimalMemoryAcrossAllSubTasks, getOpenBranches, getOptimizerNode, getOutgoingConnections, getPlanNode, getPredecessors, getRemappedKeys, getUniqueFields, hasUnclosedBranches, haveAllOutputConnectionInterestingProperties, identifyDynamicPath, initId, isBranching, isOnDynamicPath, mergeLists, prunePlanAlternatives, prunePlanAlternativesWithCommonBranching, readStubAnnotations, readUniqueFieldsAnnotation, removeClosedBranches, setBroadcastInputs, setEstimatedNumRecords, setEstimatedOutputSize, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

DataSourceNode

public DataSourceNode(GenericDataSourceBase<?,?> pactContract)

Creates a new DataSourceNode for the given contract.

Parameters:

pactContract - The data source contract object.

Method Detail

getPactContract

public GenericDataSourceBase<?,?> getPactContract()

Gets the contract object for this data source node.

Overrides:

getPactContract in class OptimizerNode

Returns:

The contract.

getName

public String getName()

Description copied from class: OptimizerNode

Gets the name of this node. This returns either the name of the PACT, or a string marking the node as a data source or a data sink.

Specified by:

getName in class OptimizerNode

Returns:

The node name.

setDegreeOfParallelism

public void setDegreeOfParallelism(int degreeOfParallelism)

Description copied from class: OptimizerNode

Sets the degree of parallelism for the contract represented by this optimizer node. The degree of parallelism denotes how many parallel instances of the user function will be spawned during the execution. If this value is set to -1, then the system will take the default number of parallel instances.

Overrides:

setDegreeOfParallelism in class OptimizerNode

Parameters:

degreeOfParallelism - The degree of parallelism to set.

getIncomingConnections

public List<PactConnection> getIncomingConnections()

Description copied from class: OptimizerNode

This method needs to be overridden by subclasses to return the children.

Specified by:

getIncomingConnections in class OptimizerNode

Returns:

The list of incoming links.

setInput

public void setInput(Map<Operator<?>,OptimizerNode> contractToNode)

Description copied from class: OptimizerNode

This function is for plan translation purposes. Upon invocation, the implementing subclasses should examine its contained contract and look at the contracts that feed their data into that contract. The method should then create a PactConnection for each of those inputs.

In addition, the nodes must set the shipping strategy of the connection, if a suitable optimizer hint is found.

Specified by:

setInput in class OptimizerNode

Parameters:

contractToNode - The map to translate the contracts to their corresponding optimizer nodes.

computeOperatorSpecificDefaultEstimates

protected void computeOperatorSpecificDefaultEstimates(DataStatistics statistics)

Specified by:

computeOperatorSpecificDefaultEstimates in class OptimizerNode

computeInterestingPropertiesForInputs

public void computeInterestingPropertiesForInputs(CostEstimator estimator)

Description copied from class: OptimizerNode

Tells the node to compute the interesting properties for its inputs. The interesting properties for the node itself must have been computed before. The node must then see how many of interesting properties it preserves and add its own.

Specified by:

computeInterestingPropertiesForInputs in class OptimizerNode

Parameters:

estimator - The CostEstimator instance to use for plan cost estimation.

computeUnclosedBranchStack

public void computeUnclosedBranchStack()

Description copied from class: OptimizerNode

This method causes the node to compute the description of open branches in its sub-plan. An open branch describes, that a (transitive) child node had multiple outputs, which have not all been re-joined in the sub-plan. This method needs to set the openBranches field to a stack of unclosed branches, the latest one top. A branch is considered closed, if some later node sees all of the branching node's outputs, no matter if there have been more branches to different paths in the meantime.

Specified by:

computeUnclosedBranchStack in class OptimizerNode

getAlternativePlans

public List<PlanNode> getAlternativePlans(CostEstimator estimator)

Description copied from class: OptimizerNode

Computes the plan alternatives for this node, an implicitly for all nodes that are children of this node. This method must determine for each alternative the global and local properties and the costs. This method may recursively call getAlternatives() on its children to get their plan alternatives, and build its own alternatives on top of those.

Specified by:

getAlternativePlans in class OptimizerNode

Parameters:

estimator - The cost estimator used to estimate the costs of each plan alternative.

Returns:

A list containing all plan alternatives.

isFieldConstant

public boolean isFieldConstant(int input,
                               int fieldNumber)

Description copied from class: OptimizerNode

Checks whether a field is modified by the user code or whether it is kept unchanged.

Specified by:

isFieldConstant in class OptimizerNode

Parameters:

input - The input number.

fieldNumber - The position of the field.

Returns:

True if the field is not changed by the user function, false otherwise.

accept

public void accept(Visitor<OptimizerNode> visitor)

Description copied from class: OptimizerNode

This method implements the visit of a depth-first graph traversing visitor. Implementors must first call the preVisit() method, then hand the visitor to their children, and finally call the postVisit() method.

Specified by:

accept in interface Visitable<OptimizerNode>

Specified by:

accept in class OptimizerNode

Parameters:

visitor - The graph traversing visitor.

See Also:

Visitable.accept(org.apache.flink.util.Visitor)