org.apache.flink.api.java.operators

Class CrossOperator.ProjectCross<I1,I2,OUT extends Tuple>

java.lang.Object

org.apache.flink.api.java.DataSet<OUT>

org.apache.flink.api.java.operators.Operator<OUT,O>

org.apache.flink.api.java.operators.TwoInputOperator<IN1,IN2,OUT,O>

org.apache.flink.api.java.operators.TwoInputUdfOperator<I1,I2,OUT,CrossOperator<I1,I2,OUT>>

org.apache.flink.api.java.operators.CrossOperator<I1,I2,OUT>

org.apache.flink.api.java.operators.CrossOperator.ProjectCross<I1,I2,OUT>

Type Parameters:

I1 - The type of the first input DataSet of the Cross transformation.

I2 - The type of the second input DataSet of the Cross transformation.

OUT - The type of the result of the Cross transformation.

All Implemented Interfaces:

UdfOperator<CrossOperator<I1,I2,OUT>>

Enclosing class:

CrossOperator<I1,I2,OUT>

public static final class CrossOperator.ProjectCross<I1,I2,OUT extends Tuple>
extends CrossOperator<I1,I2,OUT>

A Cross transformation that projects crossing elements or fields of crossing Tuples into result Tuples.
It also represents the DataSet that is the result of a Cross transformation.

See Also:

Tuple, DataSet

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.api.java.operators.CrossOperator

CrossOperator.CrossProjection<I1,I2>, CrossOperator.DefaultCross<I1,I2>, CrossOperator.DefaultCrossFunction<T1,T2>, CrossOperator.ProjectCross<I1,I2,OUT extends Tuple>, CrossOperator.ProjectCrossFunction<T1,T2,R extends Tuple>

Field Summary

Fields inherited from class org.apache.flink.api.java.operators.Operator

dop, name

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	CrossOperator.ProjectCross(DataSet<I1> input1, DataSet<I2> input2, int[] fields, boolean[] isFromFirst, TupleTypeInfo<OUT> returnType)
protected	CrossOperator.ProjectCross(DataSet<I1> input1, DataSet<I2> input2, int[] fields, boolean[] isFromFirst, TupleTypeInfo<OUT> returnType, CrossOperator.CrossProjection<I1,I2> crossProjection)

Method Summary

Methods

Modifier and Type	Method and Description
<OUT extends Tuple> CrossOperator.ProjectCross<I1,I2,OUT>	projectFirst(int... firstFieldIndexes) Continues a ProjectCross transformation and adds fields of the first cross input to the projection. If the first cross input is a Tuple DataSet, fields can be selected by their index.
<OUT extends Tuple> CrossOperator.ProjectCross<I1,I2,OUT>	projectSecond(int... secondFieldIndexes) Continues a ProjectCross transformation and adds fields of the second cross input to the projection. If the second cross input is a Tuple DataSet, fields can be selected by their index.
<OUT extends Tuple> CrossOperator<I1,I2,OUT>	types(Class<?>... types) Deprecated.
CrossOperator<I1,I2,OUT>	withConstantSetFirst(String... constantSetFirst) Adds a constant-set annotation for the first input of the UDF.
CrossOperator<I1,I2,OUT>	withConstantSetSecond(String... constantSetSecond) Adds a constant-set annotation for the second input of the UDF.

Methods inherited from class org.apache.flink.api.java.operators.CrossOperator

generateProjectionProperties, translateToDataFlow

Methods inherited from class org.apache.flink.api.java.operators.TwoInputUdfOperator

extractSemanticAnnotationsFromUdf, getBroadcastSets, getParameters, getSemanticProperties, setSemanticProperties, withBroadcastSet, withParameters

Methods inherited from class org.apache.flink.api.java.operators.TwoInputOperator

getInput1, getInput1Type, getInput2, getInput2Type

Methods inherited from class org.apache.flink.api.java.operators.Operator

getName, getParallelism, getResultType, name, setParallelism

Methods inherited from class org.apache.flink.api.java.DataSet

aggregate, checkSameExecutionContext, clean, coGroup, cross, crossWithHuge, crossWithTiny, distinct, distinct, distinct, distinct, filter, first, flatMap, getExecutionEnvironment, getType, groupBy, groupBy, groupBy, iterate, iterateDelta, join, join, joinWithHuge, joinWithTiny, map, mapPartition, max, maxBy, min, minBy, output, partitionByHash, partitionByHash, partitionByHash, partitionCustom, partitionCustom, partitionCustom, print, printToErr, project, rebalance, reduce, reduceGroup, runOperation, sum, union, write, write, writeAsCsv, writeAsCsv, writeAsCsv, writeAsCsv, writeAsFormattedText, writeAsFormattedText, writeAsText, writeAsText

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CrossOperator.ProjectCross

protected CrossOperator.ProjectCross(DataSet<I1> input1,
                          DataSet<I2> input2,
                          int[] fields,
                          boolean[] isFromFirst,
                          TupleTypeInfo<OUT> returnType)

CrossOperator.ProjectCross

protected CrossOperator.ProjectCross(DataSet<I1> input1,
                          DataSet<I2> input2,
                          int[] fields,
                          boolean[] isFromFirst,
                          TupleTypeInfo<OUT> returnType,
                          CrossOperator.CrossProjection<I1,I2> crossProjection)

Method Detail

projectFirst

public <OUT extends Tuple> CrossOperator.ProjectCross<I1,I2,OUT> projectFirst(int... firstFieldIndexes)

Continues a ProjectCross transformation and adds fields of the first cross input to the projection.
If the first cross input is a Tuple DataSet, fields can be selected by their index. If the first cross input is not a Tuple DataSet, no parameters should be passed.
Additional fields of the first and second input can be added by chaining the method calls of projectFirst(int...) and projectSecond(int...). Note: With the current implementation, the Project transformation looses type information.

Parameters:

firstFieldIndexes - If the first input is a Tuple DataSet, the indexes of the selected fields. For a non-Tuple DataSet, do not provide parameters. The order of fields in the output tuple is defined by to the order of field indexes.

Returns:

A ProjectCross which represents the projected cross result.

See Also:

Tuple, DataSet, CrossOperator.ProjectCross

projectSecond

public <OUT extends Tuple> CrossOperator.ProjectCross<I1,I2,OUT> projectSecond(int... secondFieldIndexes)

Continues a ProjectCross transformation and adds fields of the second cross input to the projection.
If the second cross input is a Tuple DataSet, fields can be selected by their index. If the second cross input is not a Tuple DataSet, no parameters should be passed.
Additional fields of the first and second input can be added by chaining the method calls of projectFirst(int...) and projectSecond(int...). Note: With the current implementation, the Project transformation looses type information.

Parameters:

secondFieldIndexes - If the second input is a Tuple DataSet, the indexes of the selected fields. For a non-Tuple DataSet, do not provide parameters. The order of fields in the output tuple is defined by to the order of field indexes.

Returns:

A ProjectCross which represents the projected cross result.

See Also:

Tuple, DataSet, CrossOperator.ProjectCross

types

@Deprecated
public <OUT extends Tuple> CrossOperator<I1,I2,OUT> types(Class<?>... types)

Deprecated.

Deprecated method only kept for compatibility.

Parameters:

types -

Returns:

withConstantSetFirst

public CrossOperator<I1,I2,OUT> withConstantSetFirst(String... constantSetFirst)

Description copied from class: TwoInputUdfOperator

Adds a constant-set annotation for the first input of the UDF.

Constant set annotations are used by the optimizer to infer the existence of data properties (sorted, partitioned, grouped). In certain cases, these annotations allow the optimizer to generate a more efficient execution plan which can lead to improved performance. Constant set annotations can only be specified if the first input and the output type of the UDF are of Tuple data types.

A constant-set annotation is a set of constant field specifications. The constant field specification String "4->3" specifies, that this UDF copies the fourth field of an input tuple to the third field of the output tuple. Field references are zero-indexed.

NOTICE: Constant set annotations are optional, but if given need to be correct. Otherwise, the program might produce wrong results!

Overrides:

withConstantSetFirst in class TwoInputUdfOperator<I1,I2,OUT extends Tuple,CrossOperator<I1,I2,OUT extends Tuple>>

Parameters:

constantSetFirst - A list of constant field specification Strings for the first input.

Returns:

This operator with an annotated constant field set for the first input.

withConstantSetSecond

public CrossOperator<I1,I2,OUT> withConstantSetSecond(String... constantSetSecond)

Description copied from class: TwoInputUdfOperator

Adds a constant-set annotation for the second input of the UDF.

Constant set annotations are used by the optimizer to infer the existence of data properties (sorted, partitioned, grouped). In certain cases, these annotations allow the optimizer to generate a more efficient execution plan which can lead to improved performance. Constant set annotations can only be specified if the second input and the output type of the UDF are of Tuple data types.

A constant-set annotation is a set of constant field specifications. The constant field specification String "4->3" specifies, that this UDF copies the fourth field of an input tuple to the third field of the output tuple. Field references are zero-indexed.

NOTICE: Constant set annotations are optional, but if given need to be correct. Otherwise, the program might produce wrong results!

Overrides:

withConstantSetSecond in class TwoInputUdfOperator<I1,I2,OUT extends Tuple,CrossOperator<I1,I2,OUT extends Tuple>>

Parameters:

constantSetSecond - A list of constant field specification Strings for the second input.

Returns:

This operator with an annotated constant field set for the second input.