org.apache.flink.api.java.operators

Class CoGroupOperator<I1,I2,OUT>

java.lang.Object

org.apache.flink.api.java.DataSet<OUT>

org.apache.flink.api.java.operators.Operator<OUT,O>

org.apache.flink.api.java.operators.TwoInputOperator<IN1,IN2,OUT,O>

org.apache.flink.api.java.operators.TwoInputUdfOperator<I1,I2,OUT,CoGroupOperator<I1,I2,OUT>>

org.apache.flink.api.java.operators.CoGroupOperator<I1,I2,OUT>

Type Parameters:

I1 - The type of the first input DataSet of the CoGroup transformation.

I2 - The type of the second input DataSet of the CoGroup transformation.

OUT - The type of the result of the CoGroup transformation.

All Implemented Interfaces:

UdfOperator<CoGroupOperator<I1,I2,OUT>>

public class CoGroupOperator<I1,I2,OUT>
extends TwoInputUdfOperator<I1,I2,OUT,CoGroupOperator<I1,I2,OUT>>

A DataSet that is the result of a CoGroup transformation.

See Also:

DataSet

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CoGroupOperator.CoGroupOperatorSets<I1,I2> Intermediate step of a CoGroup transformation.

Field Summary

Fields inherited from class org.apache.flink.api.java.operators.Operator

dop, name

Constructor Summary

Constructors

Constructor and Description
CoGroupOperator(DataSet<I1> input1, DataSet<I2> input2, Keys<I1> keys1, Keys<I2> keys2, CoGroupFunction<I1,I2,OUT> function, TypeInformation<OUT> returnType, List<org.apache.commons.lang3.tuple.Pair<Integer,Order>> groupSortKeyOrderFirst, List<org.apache.commons.lang3.tuple.Pair<Integer,Order>> groupSortKeyOrderSecond, Partitioner<?> customPartitioner, String defaultName)
CoGroupOperator(DataSet<I1> input1, DataSet<I2> input2, Keys<I1> keys1, Keys<I2> keys2, CoGroupFunction<I1,I2,OUT> function, TypeInformation<OUT> returnType, Partitioner<?> customPartitioner, String defaultName)

Method Summary

Methods

Modifier and Type	Method and Description
protected Keys<I1>	getKeys1()
protected Keys<I2>	getKeys2()
Partitioner<?>	getPartitioner() Gets the custom partitioner used by this join, or null, if none is set.
protected CoGroupOperatorBase<?,?,OUT,?>	translateToDataFlow(Operator<I1> input1, Operator<I2> input2) Translates this java API operator into a common API operator with two inputs.
CoGroupOperator<I1,I2,OUT>	withPartitioner(Partitioner<?> partitioner) Sets a custom partitioner for the CoGroup operation.

Methods inherited from class org.apache.flink.api.java.operators.TwoInputUdfOperator

extractSemanticAnnotationsFromUdf, getBroadcastSets, getParameters, getSemanticProperties, setSemanticProperties, withBroadcastSet, withConstantSetFirst, withConstantSetSecond, withParameters

Methods inherited from class org.apache.flink.api.java.operators.TwoInputOperator

getInput1, getInput1Type, getInput2, getInput2Type

Methods inherited from class org.apache.flink.api.java.operators.Operator

getName, getParallelism, getResultType, name, setParallelism

Methods inherited from class org.apache.flink.api.java.DataSet

aggregate, checkSameExecutionContext, clean, coGroup, cross, crossWithHuge, crossWithTiny, distinct, distinct, distinct, distinct, filter, first, flatMap, getExecutionEnvironment, getType, groupBy, groupBy, groupBy, iterate, iterateDelta, join, join, joinWithHuge, joinWithTiny, map, mapPartition, max, maxBy, min, minBy, output, partitionByHash, partitionByHash, partitionByHash, partitionCustom, partitionCustom, partitionCustom, print, printToErr, project, rebalance, reduce, reduceGroup, runOperation, sum, union, write, write, writeAsCsv, writeAsCsv, writeAsCsv, writeAsCsv, writeAsFormattedText, writeAsFormattedText, writeAsText, writeAsText

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CoGroupOperator

public CoGroupOperator(DataSet<I1> input1,
               DataSet<I2> input2,
               Keys<I1> keys1,
               Keys<I2> keys2,
               CoGroupFunction<I1,I2,OUT> function,
               TypeInformation<OUT> returnType,
               Partitioner<?> customPartitioner,
               String defaultName)

CoGroupOperator

public CoGroupOperator(DataSet<I1> input1,
               DataSet<I2> input2,
               Keys<I1> keys1,
               Keys<I2> keys2,
               CoGroupFunction<I1,I2,OUT> function,
               TypeInformation<OUT> returnType,
               List<org.apache.commons.lang3.tuple.Pair<Integer,Order>> groupSortKeyOrderFirst,
               List<org.apache.commons.lang3.tuple.Pair<Integer,Order>> groupSortKeyOrderSecond,
               Partitioner<?> customPartitioner,
               String defaultName)

Method Detail

getKeys1

protected Keys<I1> getKeys1()

getKeys2

protected Keys<I2> getKeys2()

withPartitioner

public CoGroupOperator<I1,I2,OUT> withPartitioner(Partitioner<?> partitioner)

Sets a custom partitioner for the CoGroup operation. The partitioner will be called on the join keys to determine the partition a key should be assigned to. The partitioner is evaluated on both inputs in the same way.

NOTE: A custom partitioner can only be used with single-field CoGroup keys, not with composite CoGroup keys.

Parameters:

partitioner - The custom partitioner to be used.

Returns:

This CoGroup operator, to allow for function chaining.

getPartitioner

public Partitioner<?> getPartitioner()

Gets the custom partitioner used by this join, or null, if none is set.

Returns:

The custom partitioner used by this join;

translateToDataFlow

protected CoGroupOperatorBase<?,?,OUT,?> translateToDataFlow(Operator<I1> input1,
                                                 Operator<I2> input2)

Description copied from class: TwoInputOperator

Translates this java API operator into a common API operator with two inputs.

Specified by:

translateToDataFlow in class TwoInputOperator<I1,I2,OUT,CoGroupOperator<I1,I2,OUT>>

Parameters:

input1 - The first input of the operation, as a common API operator.

input2 - The second input of the operation, as a common API operator.

Returns:

The created common API operator.