OptimizerNode (flink-compiler 0.7.0-incubating API)

java.lang.Object
- org.apache.flink.compiler.dag.OptimizerNode

All Implemented Interfaces:

EstimateProvider, DumpableNode<OptimizerNode>, Visitable<OptimizerNode>

Direct Known Subclasses:

AbstractPartialSolutionNode, DataSinkNode, DataSourceNode, SingleInputNode, TwoInputNode
```
public abstract class OptimizerNode
extends Object
implements Visitable<OptimizerNode>, EstimateProvider, DumpableNode<OptimizerNode>
```
This class represents a node in the optimizer's internal representation of the PACT plan. It contains extra information about estimates, hints and data properties.

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

static class OptimizerNode.UnclosedBranchDescriptor

Nested Classes
Modifier and Type	Class and Description
`static class`	`OptimizerNode.UnclosedBranchDescriptor`

Field Summary

Fields
Modifier and Type	Field and Description
`protected List<PlanNode>`	`cachedPlans`
`protected Set<OptimizerNode>`	`closedBranchingNodes`
`protected int`	`costWeight`
`protected long`	`estimatedNumRecords`
`protected long`	`estimatedOutputSize`
`protected List<OptimizerNode>`	`hereJoinedBranches`
`protected int`	`id`
`static int`	`MAX_DYNAMIC_PATH_COST_WEIGHT`
`protected boolean`	`onDynamicPath`
`protected List<OptimizerNode.UnclosedBranchDescriptor>`	`openBranches`
`protected int[][]`	`remappedKeys`
`protected Set<FieldSet>`	`uniqueFields`

Constructor Summary

Constructors
Modifier Constructor and Description

OptimizerNode(Operator<?> op)
Creates a new node for the optimizer plan.

protected OptimizerNode(OptimizerNode toCopy)

Constructors
Modifier	Constructor and Description
	`OptimizerNode(Operator<?> op)` Creates a new node for the optimizer plan.
`protected`	`OptimizerNode(OptimizerNode toCopy)`

Method Summary

All Methods Instance Methods Abstract Methods Concrete Methods
Modifier and Type	Method and Description
`abstract void`	`accept(Visitor<OptimizerNode> visitor)` This method implements the visit of a depth-first graph traversing visitor.
`void`	`addBroadcastConnection(String name, PactConnection broadcastConnection)` Adds the broadcast connection identified by the given `name` to this node.
`protected void`	`addClosedBranch(OptimizerNode alreadyClosed)`
`protected void`	`addClosedBranches(Set<OptimizerNode> alreadyClosed)`
`void`	`addOutgoingConnection(PactConnection pactConnection)` Adds a new outgoing connection to this node.
`protected boolean`	`areBranchCompatible(PlanNode plan1, PlanNode plan2)` Checks whether to candidate plans for the sub-plan of this node are comparable.
`void`	`clearInterestingProperties()`
`abstract void`	`computeInterestingPropertiesForInputs(CostEstimator estimator)` Tells the node to compute the interesting properties for its inputs.
`protected abstract void`	`computeOperatorSpecificDefaultEstimates(DataStatistics statistics)`
`void`	`computeOutputEstimates(DataStatistics statistics)` Causes this node to compute its output estimates (such as number of rows, size in bytes) based on the inputs and the compiler hints.
`abstract void`	`computeUnclosedBranchStack()` This method causes the node to compute the description of open branches in its sub-plan.
`protected List<OptimizerNode.UnclosedBranchDescriptor>`	`computeUnclosedBranchStackForBroadcastInputs(List<OptimizerNode.UnclosedBranchDescriptor> branchesSoFar)`
`void`	`computeUnionOfInterestingPropertiesFromSuccessors()` Computes all the interesting properties that are relevant to this node.
`List<FieldSet>`	`createUniqueFieldsForNode()` An optional method where nodes can describe which fields will be unique in their output.
`abstract List<PlanNode>`	`getAlternativePlans(CostEstimator estimator)` Computes the plan alternatives for this node, an implicitly for all nodes that are children of this node.
`protected List<OptimizerNode.UnclosedBranchDescriptor>`	`getBranchesForParent(PactConnection toParent)`
`List<String>`	`getBroadcastConnectionNames()` Return the list of names associated with broadcast inputs for this node.
`List<PactConnection>`	`getBroadcastConnections()` Return the list of inputs associated with broadcast variables for this node.
`Set<OptimizerNode>`	`getClosedBranchingNodes()`
`protected int[]`	`getConstantKeySet(int input)` Returns the key columns for the specific input, if all keys are preserved by this node.
`int`	`getCostWeight()`
`int`	`getDegreeOfParallelism()` Gets the degree of parallelism for the contract represented by this optimizer node.
`Iterable<DumpableConnection<OptimizerNode>>`	`getDumpableInputs()`
`float`	`getEstimatedAvgWidthPerOutputRecord()` Gets the estimated number of bytes per record.
`long`	`getEstimatedNumRecords()` Gets the estimated number of records in the output of this node.
`long`	`getEstimatedOutputSize()` Gets the estimated output size from this node.
`int`	`getId()` Gets the ID of this node.
`abstract List<PactConnection>`	`getIncomingConnections()` This method needs to be overridden by subclasses to return the children.
`InterestingProperties`	`getInterestingProperties()` Gets the properties that are interesting for this node to produce.
`int`	`getMaxDepth()`
`long`	`getMinimalMemoryAcrossAllSubTasks()` Gets the amount of memory that all subtasks of this task have jointly available.
`abstract String`	`getName()` Gets the name of this node.
`List<OptimizerNode.UnclosedBranchDescriptor>`	`getOpenBranches()`
`OptimizerNode`	`getOptimizerNode()`
`List<PactConnection>`	`getOutgoingConnections()` The list of outgoing connections from this node to succeeding tasks.
`Operator<?>`	`getPactContract()` Gets the object that specifically describes the contract of this node.
`PlanNode`	`getPlanNode()`
`Iterable<OptimizerNode>`	`getPredecessors()` Gets an iterator over the predecessors.
`int[]`	`getRemappedKeys(int input)`
`Set<FieldSet>`	`getUniqueFields()` Gets the FieldSets which are unique in the output of the node.
`boolean`	`hasUnclosedBranches()`
`boolean`	`haveAllOutputConnectionInterestingProperties()` Checks, if all outgoing connections have their interesting properties set from their target nodes.
`void`	`identifyDynamicPath(int costWeight)`
`void`	`initId(int id)` Sets the ID of this node.
`boolean`	`isBranching()` Checks whether this node has branching output.
`abstract boolean`	`isFieldConstant(int input, int fieldNumber)` Checks whether a field is modified by the user code or whether it is kept unchanged.
`boolean`	`isOnDynamicPath()`
`protected boolean`	`mergeLists(List<OptimizerNode.UnclosedBranchDescriptor> child1open, List<OptimizerNode.UnclosedBranchDescriptor> child2open, List<OptimizerNode.UnclosedBranchDescriptor> result)` The node IDs are assigned in graph-traversal order (pre-order), hence, each list is sorted by ID in ascending order and all consecutive lists start with IDs in ascending order.
`protected void`	`prunePlanAlternatives(List<PlanNode> plans)`
`protected void`	`prunePlanAlternativesWithCommonBranching(List<PlanNode> plans)`
`protected void`	`readStubAnnotations()` Reads all stub annotations, i.e.
`protected void`	`readUniqueFieldsAnnotation()`
`protected void`	`removeClosedBranches(List<OptimizerNode.UnclosedBranchDescriptor> openList)`
`void`	`setBroadcastInputs(Map<Operator<?>,OptimizerNode> operatorToNode)` This function is for plan translation purposes.
`void`	`setDegreeOfParallelism(int degreeOfParallelism)` Sets the degree of parallelism for the contract represented by this optimizer node.
`void`	`setEstimatedNumRecords(long estimatedNumRecords)`
`void`	`setEstimatedOutputSize(long estimatedOutputSize)`
`abstract void`	`setInput(Map<Operator<?>,OptimizerNode> contractToNode)` This function is for plan translation purposes.
`String`	`toString()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - MAX_DYNAMIC_PATH_COST_WEIGHT
```
public static final int MAX_DYNAMIC_PATH_COST_WEIGHT
```
    See Also:
    
    Constant Field Values
  - openBranches
```
protected List<OptimizerNode.UnclosedBranchDescriptor> openBranches
```
  - closedBranchingNodes
```
protected Set<OptimizerNode> closedBranchingNodes
```
  - hereJoinedBranches
```
protected List<OptimizerNode> hereJoinedBranches
```
  - estimatedOutputSize
```
protected long estimatedOutputSize
```
  - estimatedNumRecords
```
protected long estimatedNumRecords
```
  - uniqueFields
```
protected Set<FieldSet> uniqueFields
```
  - id
```
protected int id
```
  - costWeight
```
protected int costWeight
```
  - onDynamicPath
```
protected boolean onDynamicPath
```
  - cachedPlans
```
protected List<PlanNode> cachedPlans
```
  - remappedKeys
```
protected int[][] remappedKeys
```
- Constructor Detail
  - OptimizerNode
```
public OptimizerNode(Operator<?> op)
```
    Creates a new node for the optimizer plan.
    
    Parameters:
    
    op - The operator that the node represents.
  - OptimizerNode
```
protected OptimizerNode(OptimizerNode toCopy)
```
- Method Detail
  - getName
```
public abstract String getName()
```
    Gets the name of this node. This returns either the name of the PACT, or a string marking the node as a data source or a data sink.
    
    Returns:
    
    The node name.
  - setInput
```
public abstract void setInput(Map<Operator<?>,OptimizerNode> contractToNode)
```
    This function is for plan translation purposes. Upon invocation, the implementing subclasses should examine its contained contract and look at the contracts that feed their data into that contract. The method should then create a PactConnection for each of those inputs.
    In addition, the nodes must set the shipping strategy of the connection, if a suitable optimizer hint is found.
    
    Parameters:
    
    contractToNode - The map to translate the contracts to their corresponding optimizer nodes.
  - setBroadcastInputs
```
public void setBroadcastInputs(Map<Operator<?>,OptimizerNode> operatorToNode)
                        throws CompilerException
```
    This function is for plan translation purposes. Upon invocation, this method creates a PactConnection for each one of the broadcast inputs associated with the Operator referenced by this node.
    The PactConnections must set its shipping strategy type to BROADCAST.
    
    Parameters:
    
    operatorToNode - The map associating operators with their corresponding optimizer nodes.
    
    Throws:
    
    CompilerException
  - getIncomingConnections
```
public abstract List<PactConnection> getIncomingConnections()
```
    This method needs to be overridden by subclasses to return the children.
    
    Returns:
    
    The list of incoming links.
  - computeInterestingPropertiesForInputs
```
public abstract void computeInterestingPropertiesForInputs(CostEstimator estimator)
```
    Tells the node to compute the interesting properties for its inputs. The interesting properties for the node itself must have been computed before. The node must then see how many of interesting properties it preserves and add its own.
    
    Parameters:
    
    estimator - The CostEstimator instance to use for plan cost estimation.
  - computeUnclosedBranchStack
```
public abstract void computeUnclosedBranchStack()
```
    This method causes the node to compute the description of open branches in its sub-plan. An open branch describes, that a (transitive) child node had multiple outputs, which have not all been re-joined in the sub-plan. This method needs to set the openBranches field to a stack of unclosed branches, the latest one top. A branch is considered closed, if some later node sees all of the branching node's outputs, no matter if there have been more branches to different paths in the meantime.
  - computeUnclosedBranchStackForBroadcastInputs
```
protected List<OptimizerNode.UnclosedBranchDescriptor> computeUnclosedBranchStackForBroadcastInputs(List<OptimizerNode.UnclosedBranchDescriptor> branchesSoFar)
```
  - getAlternativePlans
```
public abstract List<PlanNode> getAlternativePlans(CostEstimator estimator)
```
    Computes the plan alternatives for this node, an implicitly for all nodes that are children of this node. This method must determine for each alternative the global and local properties and the costs. This method may recursively call getAlternatives() on its children to get their plan alternatives, and build its own alternatives on top of those.
    
    Parameters:
    
    estimator - The cost estimator used to estimate the costs of each plan alternative.
    
    Returns:
    
    A list containing all plan alternatives.
  - accept
```
public abstract void accept(Visitor<OptimizerNode> visitor)
```
    This method implements the visit of a depth-first graph traversing visitor. Implementors must first call the preVisit() method, then hand the visitor to their children, and finally call the postVisit() method.
    
    Specified by:
    
    accept in interface Visitable<OptimizerNode>
    
    Parameters:
    
    visitor - The graph traversing visitor.
    
    See Also:
    
    Visitable.accept(org.apache.flink.util.Visitor)
  - isFieldConstant
```
public abstract boolean isFieldConstant(int input,
                                        int fieldNumber)
```
    Checks whether a field is modified by the user code or whether it is kept unchanged.
    
    Parameters:
    
    input - The input number.
    
    fieldNumber - The position of the field.
    
    Returns:
    
    True if the field is not changed by the user function, false otherwise.
  - getPredecessors
```
public Iterable<OptimizerNode> getPredecessors()
```
    Description copied from interface: DumpableNode
    
    Gets an iterator over the predecessors.
    
    Specified by:
    
    getPredecessors in interface DumpableNode<OptimizerNode>
    
    Returns:
    
    An iterator over the predecessors.
  - getId
```
public int getId()
```
    Gets the ID of this node. If the id has not yet been set, this method returns -1;
    
    Returns:
    
    This node's id, or -1, if not yet set.
  - initId
```
public void initId(int id)
```
    Sets the ID of this node.
    
    Parameters:
    
    id - The id for this node.
  - addBroadcastConnection
```
public void addBroadcastConnection(String name,
                                   PactConnection broadcastConnection)
```
    Adds the broadcast connection identified by the given name to this node.
    
    Parameters:
    
    broadcastConnection - The connection to add.
  - getBroadcastConnectionNames
```
public List<String> getBroadcastConnectionNames()
```
    Return the list of names associated with broadcast inputs for this node.
  - getBroadcastConnections
```
public List<PactConnection> getBroadcastConnections()
```
    Return the list of inputs associated with broadcast variables for this node.
  - addOutgoingConnection
```
public void addOutgoingConnection(PactConnection pactConnection)
```
    Adds a new outgoing connection to this node.
    
    Parameters:
    
    pactConnection - The connection to add.
  - getOutgoingConnections
```
public List<PactConnection> getOutgoingConnections()
```
    The list of outgoing connections from this node to succeeding tasks.
    
    Returns:
    
    The list of outgoing connections.
  - getPactContract
```
public Operator<?> getPactContract()
```
    Gets the object that specifically describes the contract of this node.
    
    Returns:
    
    This node's contract.
  - getDegreeOfParallelism
```
public int getDegreeOfParallelism()
```
    Gets the degree of parallelism for the contract represented by this optimizer node. The degree of parallelism denotes how many parallel instances of the user function will be spawned during the execution. If this value is -1, then the system will take the default number of parallel instances.
    
    Returns:
    
    The degree of parallelism.
  - setDegreeOfParallelism
```
public void setDegreeOfParallelism(int degreeOfParallelism)
```
    Sets the degree of parallelism for the contract represented by this optimizer node. The degree of parallelism denotes how many parallel instances of the user function will be spawned during the execution. If this value is set to -1, then the system will take the default number of parallel instances.
    
    Parameters:
    
    degreeOfParallelism - The degree of parallelism to set.
    
    Throws:
    
    IllegalArgumentException - If the degree of parallelism is smaller than one and not -1.
  - getMinimalMemoryAcrossAllSubTasks
```
public long getMinimalMemoryAcrossAllSubTasks()
```
    Gets the amount of memory that all subtasks of this task have jointly available.
    
    Returns:
    
    The total amount of memory across all subtasks.
  - isOnDynamicPath
```
public boolean isOnDynamicPath()
```
  - identifyDynamicPath
```
public void identifyDynamicPath(int costWeight)
```
  - getCostWeight
```
public int getCostWeight()
```
  - getMaxDepth
```
public int getMaxDepth()
```
  - getInterestingProperties
```
public InterestingProperties getInterestingProperties()
```
    Gets the properties that are interesting for this node to produce.
    
    Returns:
    
    The interesting properties for this node, or null, if not yet computed.
  - getEstimatedOutputSize
```
public long getEstimatedOutputSize()
```
    Description copied from interface: EstimateProvider
    
    Gets the estimated output size from this node.
    
    Specified by:
    
    getEstimatedOutputSize in interface EstimateProvider
    
    Returns:
    
    The estimated output size.
  - getEstimatedNumRecords
```
public long getEstimatedNumRecords()
```
    Description copied from interface: EstimateProvider
    
    Gets the estimated number of records in the output of this node.
    
    Specified by:
    
    getEstimatedNumRecords in interface EstimateProvider
    
    Returns:
    
    The estimated number of records.
  - setEstimatedOutputSize
```
public void setEstimatedOutputSize(long estimatedOutputSize)
```
  - setEstimatedNumRecords
```
public void setEstimatedNumRecords(long estimatedNumRecords)
```
  - getEstimatedAvgWidthPerOutputRecord
```
public float getEstimatedAvgWidthPerOutputRecord()
```
    Description copied from interface: EstimateProvider
    
    Gets the estimated number of bytes per record.
    
    Specified by:
    
    getEstimatedAvgWidthPerOutputRecord in interface EstimateProvider
    
    Returns:
    
    The estimated number of bytes per record.
  - isBranching
```
public boolean isBranching()
```
    Checks whether this node has branching output. A node's output is branched, if it has more than one output connection.
    
    Returns:
    
    True, if the node's output branches. False otherwise.
  - haveAllOutputConnectionInterestingProperties
```
public boolean haveAllOutputConnectionInterestingProperties()
```
    Checks, if all outgoing connections have their interesting properties set from their target nodes.
    
    Returns:
    
    True, if on all outgoing connections, the interesting properties are set. False otherwise.
  - computeUnionOfInterestingPropertiesFromSuccessors
```
public void computeUnionOfInterestingPropertiesFromSuccessors()
```
    Computes all the interesting properties that are relevant to this node. The interesting properties are a union of the interesting properties on each outgoing connection. However, if two interesting properties on the outgoing connections overlap, the interesting properties will occur only once in this set. For that, this method deduplicates and merges the interesting properties. This method returns copies of the original interesting properties objects and leaves the original objects, contained by the connections, unchanged.
  - clearInterestingProperties
```
public void clearInterestingProperties()
```
  - computeOutputEstimates
```
public void computeOutputEstimates(DataStatistics statistics)
```
    Causes this node to compute its output estimates (such as number of rows, size in bytes) based on the inputs and the compiler hints. The compiler hints are instantiated with conservative default values which are used if no other values are provided. Nodes may access the statistics to determine relevant information.
    
    Parameters:
    
    statistics - The statistics object which may be accessed to get statistical information. The parameter may be null, if no statistics are available.
  - computeOperatorSpecificDefaultEstimates
```
protected abstract void computeOperatorSpecificDefaultEstimates(DataStatistics statistics)
```
  - readStubAnnotations
```
protected void readStubAnnotations()
```
    Reads all stub annotations, i.e. which fields remain constant, what cardinality bounds the functions have, which fields remain unique.
  - readUniqueFieldsAnnotation
```
protected void readUniqueFieldsAnnotation()
```
  - getConstantKeySet
```
protected int[] getConstantKeySet(int input)
```
    Returns the key columns for the specific input, if all keys are preserved by this node. Null, otherwise.
  - createUniqueFieldsForNode
```
public List<FieldSet> createUniqueFieldsForNode()
```
    An optional method where nodes can describe which fields will be unique in their output.
  - getUniqueFields
```
public Set<FieldSet> getUniqueFields()
```
    Gets the FieldSets which are unique in the output of the node.
  - prunePlanAlternatives
```
protected void prunePlanAlternatives(List<PlanNode> plans)
```
  - prunePlanAlternativesWithCommonBranching
```
protected void prunePlanAlternativesWithCommonBranching(List<PlanNode> plans)
```
  - hasUnclosedBranches
```
public boolean hasUnclosedBranches()
```
  - getClosedBranchingNodes
```
public Set<OptimizerNode> getClosedBranchingNodes()
```
  - getOpenBranches
```
public List<OptimizerNode.UnclosedBranchDescriptor> getOpenBranches()
```
  - getBranchesForParent
```
protected List<OptimizerNode.UnclosedBranchDescriptor> getBranchesForParent(PactConnection toParent)
```
  - removeClosedBranches
```
protected void removeClosedBranches(List<OptimizerNode.UnclosedBranchDescriptor> openList)
```
  - addClosedBranches
```
protected void addClosedBranches(Set<OptimizerNode> alreadyClosed)
```
  - addClosedBranch
```
protected void addClosedBranch(OptimizerNode alreadyClosed)
```
  - areBranchCompatible
```
protected boolean areBranchCompatible(PlanNode plan1,
                                      PlanNode plan2)
```
    Checks whether to candidate plans for the sub-plan of this node are comparable. The two alternative plans are comparable, if a) There is no branch in the sub-plan of this node b) Both candidates have the same candidate as the child at the last open branch.
    
    Parameters:
    
    plan1 -
    
    plan2 -
    
    Returns:
    
    True if the nodes are branch compatible in the inputs.
  - mergeLists
```
protected final boolean mergeLists(List<OptimizerNode.UnclosedBranchDescriptor> child1open,
                                   List<OptimizerNode.UnclosedBranchDescriptor> child2open,
                                   List<OptimizerNode.UnclosedBranchDescriptor> result)
```
    The node IDs are assigned in graph-traversal order (pre-order), hence, each list is sorted by ID in ascending order and all consecutive lists start with IDs in ascending order.
  - getOptimizerNode
```
public OptimizerNode getOptimizerNode()
```
    Specified by:
    
    getOptimizerNode in interface DumpableNode<OptimizerNode>
  - getPlanNode
```
public PlanNode getPlanNode()
```
    Specified by:
    
    getPlanNode in interface DumpableNode<OptimizerNode>
  - getDumpableInputs
```
public Iterable<DumpableConnection<OptimizerNode>> getDumpableInputs()
```
    Specified by:
    
    getDumpableInputs in interface DumpableNode<OptimizerNode>
  - toString
```
public String toString()
```
    Overrides:
    
    toString in class Object
  - getRemappedKeys
```
public int[] getRemappedKeys(int input)
```

Class OptimizerNode

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

MAX_DYNAMIC_PATH_COST_WEIGHT

openBranches

closedBranchingNodes

hereJoinedBranches

estimatedOutputSize

estimatedNumRecords

uniqueFields

id

costWeight

onDynamicPath

cachedPlans

remappedKeys

Constructor Detail

OptimizerNode

OptimizerNode

Method Detail

getName

setInput

setBroadcastInputs

getIncomingConnections

computeInterestingPropertiesForInputs

computeUnclosedBranchStack

computeUnclosedBranchStackForBroadcastInputs

getAlternativePlans

accept

isFieldConstant

getPredecessors

getId

initId

addBroadcastConnection

getBroadcastConnectionNames

getBroadcastConnections

addOutgoingConnection

getOutgoingConnections

getPactContract

getDegreeOfParallelism

setDegreeOfParallelism

getMinimalMemoryAcrossAllSubTasks

isOnDynamicPath

identifyDynamicPath

getCostWeight

getMaxDepth

getInterestingProperties

getEstimatedOutputSize

getEstimatedNumRecords

setEstimatedOutputSize

setEstimatedNumRecords

getEstimatedAvgWidthPerOutputRecord

isBranching

haveAllOutputConnectionInterestingProperties

computeUnionOfInterestingPropertiesFromSuccessors

clearInterestingProperties

computeOutputEstimates

computeOperatorSpecificDefaultEstimates

readStubAnnotations

readUniqueFieldsAnnotation

getConstantKeySet

createUniqueFieldsForNode

getUniqueFields

prunePlanAlternatives

prunePlanAlternativesWithCommonBranching

hasUnclosedBranches

getClosedBranchingNodes

getOpenBranches

getBranchesForParent

removeClosedBranches

addClosedBranches

addClosedBranch

areBranchCompatible

mergeLists

getOptimizerNode

getPlanNode

getDumpableInputs