org.apache.flink.streaming.api.datastream

Class GroupedDataStream<OUT>

java.lang.Object

org.apache.flink.streaming.api.datastream.DataStream<OUT>

org.apache.flink.streaming.api.datastream.GroupedDataStream<OUT>

Type Parameters:

OUT - The output type of the GroupedDataStream.

public class GroupedDataStream<OUT>
extends DataStream<OUT>

A GroupedDataStream represents a DataStream which has been partitioned by the given KeySelector. Operators like reduce(org.apache.flink.api.common.functions.ReduceFunction<OUT>), fold(R, org.apache.flink.api.common.functions.FoldFunction<OUT, R>) etc. can be applied on the GroupedDataStream to get additional functionality by the grouping.

Field Summary

Fields inherited from class org.apache.flink.streaming.api.datastream.DataStream

counter, environment, id, iterationID, iterationWaitTime, parallelism, partitioner, streamGraph, typeInfo, unionizedStreams, userDefinedNames

Constructor Summary

Constructors

Modifier	Constructor and Description
	GroupedDataStream(DataStream<OUT> dataStream, org.apache.flink.api.java.functions.KeySelector<OUT,?> keySelector) Creates a new GroupedDataStream, group inclusion is determined using a KeySelector on the elements of the DataStream.
protected	GroupedDataStream(GroupedDataStream<OUT> dataStream)

Method Summary

Modifier and Type	Method and Description
protected SingleOutputStreamOperator<OUT,?>	aggregate(AggregationFunction<OUT> aggregate)
GroupedDataStream<OUT>	copy() Creates a copy of the DataStream
<R> SingleOutputStreamOperator<R,?>	fold(R initialValue, org.apache.flink.api.common.functions.FoldFunction<OUT,R> folder) Applies a fold transformation on the grouped data stream grouped on by the given key position.
org.apache.flink.api.java.functions.KeySelector<OUT,?>	getKeySelector()
SingleOutputStreamOperator<OUT,?>	max(int positionToMax) Applies an aggregation that gives the maximum of the grouped data stream at the given position, grouped by the given key position.
SingleOutputStreamOperator<OUT,?>	maxBy(int positionToMaxBy) Applies an aggregation that that gives the current element with the maximum value at the given position for each group on a grouped data stream.
SingleOutputStreamOperator<OUT,?>	maxBy(int positionToMaxBy, boolean first) Applies an aggregation that that gives the current element with the maximum value at the given position for each group on a grouped data stream.
SingleOutputStreamOperator<OUT,?>	min(int positionToMin) Applies an aggregation that gives the minimum of the grouped data stream at the given position, grouped by the given key position.
SingleOutputStreamOperator<OUT,?>	minBy(int positionToMinBy) Applies an aggregation that that gives the current element with the minimum value at the given position for each group on a grouped data stream.
SingleOutputStreamOperator<OUT,?>	minBy(int positionToMinBy, boolean first) Applies an aggregation that that gives the current element with the minimum value at the given position for each group on a grouped data stream.
SingleOutputStreamOperator<OUT,?>	reduce(org.apache.flink.api.common.functions.ReduceFunction<OUT> reducer) Applies a reduce transformation on the grouped data stream grouped on by the given key position.
protected DataStream<OUT>	setConnectionType(StreamPartitioner<OUT> partitioner) Internal function for setting the partitioner for the DataStream
SingleOutputStreamOperator<OUT,?>	sum(int positionToSum) Applies an aggregation that sums the grouped data stream at the given position, grouped by the given key position.

Methods inherited from class org.apache.flink.streaming.api.datastream.DataStream

addSink, broadcast, checkFieldRange, clean, connect, connectGraph, count, cross, every, fillInType, filter, flatMap, forward, getClassAtPos, getExecutionConfig, getExecutionEnvironment, getId, getParallelism, getType, global, groupBy, groupBy, groupBy, iterate, iterate, join, map, max, maxBy, maxBy, min, minBy, minBy, partitionByHash, partitionByHash, partitionByHash, print, printToErr, project, rebalance, shuffle, split, sum, transform, union, window, window, write, writeAsCsv, writeAsCsv, writeAsCsv, writeAsCsv, writeAsText, writeAsText, writeAsText, writeAsText, writeToSocket

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GroupedDataStream

public GroupedDataStream(DataStream<OUT> dataStream,
                         org.apache.flink.api.java.functions.KeySelector<OUT,?> keySelector)

Creates a new GroupedDataStream, group inclusion is determined using a KeySelector on the elements of the DataStream.

Parameters:

dataStream - Base stream of data

keySelector - Function for determining group inclusion

GroupedDataStream

protected GroupedDataStream(GroupedDataStream<OUT> dataStream)

Method Detail

getKeySelector

public org.apache.flink.api.java.functions.KeySelector<OUT,?> getKeySelector()

reduce

public SingleOutputStreamOperator<OUT,?> reduce(org.apache.flink.api.common.functions.ReduceFunction<OUT> reducer)

Applies a reduce transformation on the grouped data stream grouped on by the given key position. The ReduceFunction will receive input values based on the key value. Only input values with the same key will go to the same reducer.

Overrides:

reduce in class DataStream<OUT>

Parameters:

reducer - The ReduceFunction that will be called for every element of the input values with the same key.

Returns:

The transformed DataStream.

fold

public <R> SingleOutputStreamOperator<R,?> fold(R initialValue,
                                                org.apache.flink.api.common.functions.FoldFunction<OUT,R> folder)

Applies a fold transformation on the grouped data stream grouped on by the given key position. The FoldFunction will receive input values based on the key value. Only input values with the same key will go to the same folder.

Overrides:

fold in class DataStream<OUT>

Parameters:

folder - The FoldFunction that will be called for every element of the input values with the same key.

initialValue - The initialValue passed to the folders for each key.

Returns:

The transformed DataStream.

sum

public SingleOutputStreamOperator<OUT,?> sum(int positionToSum)

Applies an aggregation that sums the grouped data stream at the given position, grouped by the given key position. Input values with the same key will be summed.

Overrides:

sum in class DataStream<OUT>

Parameters:

positionToSum - The position in the data point to sum

Returns:

The transformed DataStream.

min

public SingleOutputStreamOperator<OUT,?> min(int positionToMin)

Applies an aggregation that gives the minimum of the grouped data stream at the given position, grouped by the given key position. Input values with the same key will be minimized.

Overrides:

min in class DataStream<OUT>

Parameters:

positionToMin - The position in the data point to minimize

Returns:

The transformed DataStream.

minBy

public SingleOutputStreamOperator<OUT,?> minBy(int positionToMinBy)

Applies an aggregation that that gives the current element with the minimum value at the given position for each group on a grouped data stream. If more elements have the minimum value at the given position, the operator returns the first one by default.

Overrides:

minBy in class DataStream<OUT>

Parameters:

positionToMinBy - The position in the data point to minimize

Returns:

The transformed DataStream.

minBy

public SingleOutputStreamOperator<OUT,?> minBy(int positionToMinBy,
                                               boolean first)

Applies an aggregation that that gives the current element with the minimum value at the given position for each group on a grouped data stream. If more elements have the minimum value at the given position, the operator returns either the first or last one depending on the parameters.

Overrides:

minBy in class DataStream<OUT>

Parameters:

positionToMinBy - The position in the data point to minimize

first - If true, then the operator return the first element with the maximum value, otherwise returns the last

Returns:

The transformed DataStream.

max

public SingleOutputStreamOperator<OUT,?> max(int positionToMax)

Applies an aggregation that gives the maximum of the grouped data stream at the given position, grouped by the given key position. Input values with the same key will be maximized.

Overrides:

max in class DataStream<OUT>

Parameters:

positionToMax - The position in the data point to maximize

Returns:

The transformed DataStream.

maxBy

public SingleOutputStreamOperator<OUT,?> maxBy(int positionToMaxBy)

Applies an aggregation that that gives the current element with the maximum value at the given position for each group on a grouped data stream. If more elements have the maximum value at the given position, the operator returns the first one by default.

Overrides:

maxBy in class DataStream<OUT>

Parameters:

positionToMaxBy - The position in the data point to maximize

Returns:

The transformed DataStream.

maxBy

public SingleOutputStreamOperator<OUT,?> maxBy(int positionToMaxBy,
                                               boolean first)

Applies an aggregation that that gives the current element with the maximum value at the given position for each group on a grouped data stream. If more elements have the maximum value at the given position, the operator returns either the first or last one depending on the parameters.

Overrides:

maxBy in class DataStream<OUT>

Parameters:

positionToMaxBy - The position in the data point to maximize

first - If true, then the operator return the first element with the maximum value, otherwise returns the last

Returns:

The transformed DataStream.

aggregate

protected SingleOutputStreamOperator<OUT,?> aggregate(AggregationFunction<OUT> aggregate)

Overrides:

aggregate in class DataStream<OUT>

setConnectionType

protected DataStream<OUT> setConnectionType(StreamPartitioner<OUT> partitioner)

Description copied from class: DataStream

Internal function for setting the partitioner for the DataStream

Overrides:

setConnectionType in class DataStream<OUT>

Parameters:

partitioner - Partitioner to set.

Returns:

The modified DataStream.

copy

public GroupedDataStream<OUT> copy()

Description copied from class: DataStream

Creates a copy of the DataStream

Overrides:

copy in class DataStream<OUT>

Returns:

The copy