| Package | Description |
|---|---|
| org.apache.flink.streaming.api.datastream | |
| org.apache.flink.streaming.api.graph | |
| org.apache.flink.streaming.api.transformations | |
| org.apache.flink.streaming.runtime.io | |
| org.apache.flink.streaming.runtime.io.recovery | |
| org.apache.flink.streaming.runtime.partitioner | |
| org.apache.flink.streaming.runtime.tasks |
This package contains classes that realize streaming tasks.
|
| Modifier and Type | Method and Description |
|---|---|
protected DataStream<T> |
DataStream.setConnectionType(StreamPartitioner<T> partitioner)
Internal function for setting the partitioner for the DataStream.
|
protected DataStream<T> |
KeyedStream.setConnectionType(StreamPartitioner<T> partitioner) |
| Modifier and Type | Method and Description |
|---|---|
StreamPartitioner<?> |
StreamEdge.getPartitioner() |
| Modifier and Type | Method and Description |
|---|---|
void |
StreamGraph.addVirtualPartitionNode(Integer originalId,
Integer virtualId,
StreamPartitioner<?> partitioner,
ShuffleMode shuffleMode)
Adds a new virtual node that is used to connect a downstream vertex to an input with a
certain partitioning.
|
void |
StreamEdge.setPartitioner(StreamPartitioner<?> partitioner) |
| Constructor and Description |
|---|
StreamEdge(StreamNode sourceVertex,
StreamNode targetVertex,
int typeNumber,
long bufferTimeout,
StreamPartitioner<?> outputPartitioner,
org.apache.flink.util.OutputTag outputTag,
ShuffleMode shuffleMode,
int uniqueId) |
StreamEdge(StreamNode sourceVertex,
StreamNode targetVertex,
int typeNumber,
StreamPartitioner<?> outputPartitioner,
org.apache.flink.util.OutputTag outputTag) |
StreamEdge(StreamNode sourceVertex,
StreamNode targetVertex,
int typeNumber,
StreamPartitioner<?> outputPartitioner,
org.apache.flink.util.OutputTag outputTag,
ShuffleMode shuffleMode,
int uniqueId) |
| Modifier and Type | Method and Description |
|---|---|
StreamPartitioner<T> |
PartitionTransformation.getPartitioner()
Returns the
StreamPartitioner that must be used for partitioning the elements of the
input Transformation. |
| Constructor and Description |
|---|
PartitionTransformation(org.apache.flink.api.dag.Transformation<T> input,
StreamPartitioner<T> partitioner)
Creates a new
PartitionTransformation from the given input and StreamPartitioner. |
PartitionTransformation(org.apache.flink.api.dag.Transformation<T> input,
StreamPartitioner<T> partitioner,
ShuffleMode shuffleMode)
Creates a new
PartitionTransformation from the given input and StreamPartitioner. |
| Modifier and Type | Method and Description |
|---|---|
static <IN1,IN2> StreamTwoInputProcessor<IN1,IN2> |
StreamTwoInputProcessorFactory.create(org.apache.flink.runtime.jobgraph.tasks.AbstractInvokable ownerTask,
CheckpointedInputGate[] checkpointedInputGates,
org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager,
org.apache.flink.runtime.memory.MemoryManager memoryManager,
org.apache.flink.runtime.metrics.groups.TaskIOMetricGroup taskIOMetricGroup,
StreamStatusMaintainer streamStatusMaintainer,
TwoInputStreamOperator<IN1,IN2,?> streamOperator,
WatermarkGauge input1WatermarkGauge,
WatermarkGauge input2WatermarkGauge,
BoundedMultiInput endOfInputAware,
StreamConfig streamConfig,
org.apache.flink.configuration.Configuration taskManagerConfig,
org.apache.flink.configuration.Configuration jobConfig,
org.apache.flink.api.common.ExecutionConfig executionConfig,
ClassLoader userClassloader,
org.apache.flink.metrics.Counter numRecordsIn,
org.apache.flink.runtime.checkpoint.InflightDataRescalingDescriptor inflightDataRescalingDescriptor,
java.util.function.Function<Integer,StreamPartitioner<?>> gatePartitioners,
org.apache.flink.api.common.TaskInfo taskInfo) |
static StreamMultipleInputProcessor |
StreamMultipleInputProcessorFactory.create(org.apache.flink.runtime.jobgraph.tasks.AbstractInvokable ownerTask,
CheckpointedInputGate[] checkpointedInputGates,
StreamConfig.InputConfig[] configuredInputs,
org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager,
org.apache.flink.runtime.memory.MemoryManager memoryManager,
org.apache.flink.runtime.metrics.groups.TaskIOMetricGroup ioMetricGroup,
org.apache.flink.metrics.Counter mainOperatorRecordsIn,
StreamStatusMaintainer streamStatusMaintainer,
MultipleInputStreamOperator<?> mainOperator,
WatermarkGauge[] inputWatermarkGauges,
StreamConfig streamConfig,
org.apache.flink.configuration.Configuration taskManagerConfig,
org.apache.flink.configuration.Configuration jobConfig,
org.apache.flink.api.common.ExecutionConfig executionConfig,
ClassLoader userClassloader,
OperatorChain<?,?> operatorChain,
org.apache.flink.runtime.checkpoint.InflightDataRescalingDescriptor inflightDataRescalingDescriptor,
java.util.function.Function<Integer,StreamPartitioner<?>> gatePartitioners,
org.apache.flink.api.common.TaskInfo taskInfo) |
static <T> StreamTaskInput<T> |
StreamTaskNetworkInputFactory.create(CheckpointedInputGate checkpointedInputGate,
org.apache.flink.api.common.typeutils.TypeSerializer<T> inputSerializer,
org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager,
StatusWatermarkValve statusWatermarkValve,
int inputIndex,
org.apache.flink.runtime.checkpoint.InflightDataRescalingDescriptor rescalingDescriptorinflightDataRescalingDescriptor,
java.util.function.Function<Integer,StreamPartitioner<?>> gatePartitioners,
org.apache.flink.api.common.TaskInfo taskInfo)
Factory method for
StreamTaskNetworkInput or RescalingStreamTaskNetworkInput
depending on InflightDataRescalingDescriptor. |
| Constructor and Description |
|---|
RescalingStreamTaskNetworkInput(CheckpointedInputGate checkpointedInputGate,
org.apache.flink.api.common.typeutils.TypeSerializer<T> inputSerializer,
org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager,
StatusWatermarkValve statusWatermarkValve,
int inputIndex,
org.apache.flink.runtime.checkpoint.InflightDataRescalingDescriptor inflightDataRescalingDescriptor,
java.util.function.Function<Integer,StreamPartitioner<?>> gatePartitioners,
org.apache.flink.api.common.TaskInfo taskInfo) |
| Modifier and Type | Class and Description |
|---|---|
class |
BroadcastPartitioner<T>
Partitioner that selects all the output channels.
|
class |
CustomPartitionerWrapper<K,T>
Partitioner that selects the channel with a user defined partitioner function on a key.
|
class |
ForwardPartitioner<T>
Partitioner that forwards elements only to the locally running downstream operation.
|
class |
GlobalPartitioner<T>
Partitioner that sends all elements to the downstream operator with subtask ID=0.
|
class |
KeyGroupStreamPartitioner<T,K>
Partitioner selects the target channel based on the key group index.
|
class |
RebalancePartitioner<T>
Partitioner that distributes the data equally by cycling through the output channels.
|
class |
RescalePartitioner<T>
Partitioner that distributes the data equally by cycling through the output channels.
|
class |
ShufflePartitioner<T>
Partitioner that distributes the data equally by selecting one output channel randomly.
|
| Modifier and Type | Method and Description |
|---|---|
StreamPartitioner<T> |
CustomPartitionerWrapper.copy() |
StreamPartitioner<T> |
RebalancePartitioner.copy() |
StreamPartitioner<T> |
GlobalPartitioner.copy() |
StreamPartitioner<T> |
ShufflePartitioner.copy() |
abstract StreamPartitioner<T> |
StreamPartitioner.copy() |
StreamPartitioner<T> |
BroadcastPartitioner.copy() |
StreamPartitioner<T> |
RescalePartitioner.copy() |
StreamPartitioner<T> |
KeyGroupStreamPartitioner.copy() |
StreamPartitioner<T> |
ForwardPartitioner.copy() |
| Modifier and Type | Method and Description |
|---|---|
protected void |
MultipleInputStreamTask.createInputProcessor(List<org.apache.flink.runtime.io.network.partition.consumer.IndexedInputGate>[] inputGates,
StreamConfig.InputConfig[] inputs,
WatermarkGauge[] inputWatermarkGauges,
java.util.function.Function<Integer,StreamPartitioner<?>> gatePartitioners) |
protected abstract void |
AbstractTwoInputStreamTask.createInputProcessor(List<org.apache.flink.runtime.io.network.partition.consumer.IndexedInputGate> inputGates1,
List<org.apache.flink.runtime.io.network.partition.consumer.IndexedInputGate> inputGates2,
java.util.function.Function<Integer,StreamPartitioner<?>> gatePartitioners) |
protected void |
TwoInputStreamTask.createInputProcessor(List<org.apache.flink.runtime.io.network.partition.consumer.IndexedInputGate> inputGates1,
List<org.apache.flink.runtime.io.network.partition.consumer.IndexedInputGate> inputGates2,
java.util.function.Function<Integer,StreamPartitioner<?>> gatePartitioners) |
Copyright © 2014–2022 The Apache Software Foundation. All rights reserved.