org.apache.flink.optimizer.dataproperties

Class RequestedGlobalProperties

java.lang.Object

org.apache.flink.optimizer.dataproperties.RequestedGlobalProperties

All Implemented Interfaces:

Cloneable

public final class RequestedGlobalProperties
extends Object
implements Cloneable

This class represents the global properties of the data that are requested by an operator. Operators request the global properties they need for correct execution. This list is an example of global properties requested by certain operators:

• "groupBy/reduce" will request the data to be partitioned in some way after the key fields.
• "map" will request the data to be in an arbitrary distribution - it has no prerequisites
• "join" will request certain properties for each input. This class represents the properties on an input alone. The properties may be partitioning on the key fields, or a combination of replication on one input and anything-but-replication on the other input.

Constructor Summary

Constructors

Constructor and Description
RequestedGlobalProperties() Initializes the global properties with no partitioning.

Method Summary

Modifier and Type	Method and Description
RequestedGlobalProperties	clone()
boolean	equals(Object obj)
RequestedGlobalProperties	filterBySemanticProperties(org.apache.flink.api.common.operators.SemanticProperties props, int input) Filters these properties by what can be preserved by the given SemanticProperties when propagated down to the given input.
org.apache.flink.api.common.functions.Partitioner<?>	getCustomPartitioner() Gets the custom partitioner associated with these properties.
org.apache.flink.api.common.distributions.DataDistribution	getDataDistribution() Gets the data distribution.
org.apache.flink.api.common.operators.Ordering	getOrdering() Gets the key order.
org.apache.flink.api.common.operators.util.FieldSet	getPartitionedFields() Gets the fields on which the data is partitioned.
PartitioningProperty	getPartitioning() Gets the partitioning property.
int	hashCode()
boolean	isMetBy(GlobalProperties props) Checks, if this set of interesting properties, is met by the given produced properties.
boolean	isTrivial() Checks, if the properties in this object are trivial, i.e.
void	parameterizeChannel(Channel channel, boolean globalDopChange, org.apache.flink.api.common.ExecutionMode exchangeMode, boolean breakPipeline) Parametrizes the ship strategy fields of a channel such that the channel produces the desired global properties.
void	reset() This method resets the properties to a state where no properties are given.
void	setAnyDistribution()
void	setAnyPartitioning(org.apache.flink.api.common.operators.util.FieldSet partitionedFields) Sets these properties to request some partitioning on the given fields.
void	setCustomPartitioned(org.apache.flink.api.common.operators.util.FieldSet partitionedFields, org.apache.flink.api.common.functions.Partitioner<?> partitioner) Sets these properties to request a custom partitioning with the given Partitioner instance.
void	setForceRebalancing()
void	setFullyReplicated()
void	setHashPartitioned(org.apache.flink.api.common.operators.util.FieldSet partitionedFields) Sets these properties to request a hash partitioning on the given fields.
void	setRandomPartitioning()
void	setRangePartitioned(org.apache.flink.api.common.operators.Ordering ordering)
void	setRangePartitioned(org.apache.flink.api.common.operators.Ordering ordering, org.apache.flink.api.common.distributions.DataDistribution dataDistribution)
String	toString()

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

RequestedGlobalProperties

public RequestedGlobalProperties()

Initializes the global properties with no partitioning.

Method Detail

setHashPartitioned

public void setHashPartitioned(org.apache.flink.api.common.operators.util.FieldSet partitionedFields)

Sets these properties to request a hash partitioning on the given fields. If the fields are provided as FieldSet, then any permutation of the fields is a valid partitioning, including subsets. If the fields are given as a FieldList, then only an exact partitioning on the fields matches this requested partitioning.

Parameters:

partitionedFields - The key fields for the partitioning.

setRangePartitioned

public void setRangePartitioned(org.apache.flink.api.common.operators.Ordering ordering)

setRangePartitioned

public void setRangePartitioned(org.apache.flink.api.common.operators.Ordering ordering,
                                org.apache.flink.api.common.distributions.DataDistribution dataDistribution)

setAnyPartitioning

public void setAnyPartitioning(org.apache.flink.api.common.operators.util.FieldSet partitionedFields)

Sets these properties to request some partitioning on the given fields. This will allow both hash partitioning and range partitioning to match. If the fields are provided as FieldSet, then any permutation of the fields is a valid partitioning, including subsets. If the fields are given as a FieldList, then only an exact partitioning on the fields matches this requested partitioning.

Parameters:

partitionedFields - The key fields for the partitioning.

setRandomPartitioning

public void setRandomPartitioning()

setAnyDistribution

public void setAnyDistribution()

setFullyReplicated

public void setFullyReplicated()

setForceRebalancing

public void setForceRebalancing()

setCustomPartitioned

public void setCustomPartitioned(org.apache.flink.api.common.operators.util.FieldSet partitionedFields,
                                 org.apache.flink.api.common.functions.Partitioner<?> partitioner)

Sets these properties to request a custom partitioning with the given Partitioner instance. If the fields are provided as FieldSet, then any permutation of the fields is a valid partitioning, including subsets. If the fields are given as a FieldList, then only an exact partitioning on the fields matches this requested partitioning.

Parameters:

partitionedFields - The key fields for the partitioning.

getPartitioning

public PartitioningProperty getPartitioning()

Gets the partitioning property.

Returns:

The partitioning property.

getPartitionedFields

public org.apache.flink.api.common.operators.util.FieldSet getPartitionedFields()

Gets the fields on which the data is partitioned.

Returns:

The partitioning fields.

getOrdering

public org.apache.flink.api.common.operators.Ordering getOrdering()

Gets the key order.

Returns:

The key order.

getDataDistribution

public org.apache.flink.api.common.distributions.DataDistribution getDataDistribution()

Gets the data distribution.

Returns:

The data distribution.

getCustomPartitioner

public org.apache.flink.api.common.functions.Partitioner<?> getCustomPartitioner()

Gets the custom partitioner associated with these properties.

Returns:

The custom partitioner associated with these properties.

isTrivial

public boolean isTrivial()

Checks, if the properties in this object are trivial, i.e. only standard values.

reset

public void reset()

This method resets the properties to a state where no properties are given.

filterBySemanticProperties

public RequestedGlobalProperties filterBySemanticProperties(org.apache.flink.api.common.operators.SemanticProperties props,
                                                            int input)

Filters these properties by what can be preserved by the given SemanticProperties when propagated down to the given input.

Parameters:

props - The SemanticProperties which define which fields are preserved.

input - The index of the operator's input.

Returns:

The filtered RequestedGlobalProperties

isMetBy

public boolean isMetBy(GlobalProperties props)

Checks, if this set of interesting properties, is met by the given produced properties.

Parameters:

props - The properties for which to check whether they meet these properties.

Returns:

True, if the properties are met, false otherwise.

parameterizeChannel

public void parameterizeChannel(Channel channel,
                                boolean globalDopChange,
                                org.apache.flink.api.common.ExecutionMode exchangeMode,
                                boolean breakPipeline)

Parametrizes the ship strategy fields of a channel such that the channel produces the desired global properties.

Parameters:

channel - The channel to parametrize.

globalDopChange - Flag indicating whether the parallelism changes between sender and receiver.

exchangeMode - The mode of data exchange (pipelined, always batch, batch only on shuffle, ...)

breakPipeline - Indicates whether this data exchange should break pipelines (unless pipelines are forced).

hashCode

public int hashCode()

Overrides:

hashCode in class Object

equals

public boolean equals(Object obj)

Overrides:

equals in class Object

toString

public String toString()

Overrides:

toString in class Object

clone

public RequestedGlobalProperties clone()

Overrides:

clone in class Object