org.apache.flink.streaming.connectors.kafka.partitioner

Class FixedPartitioner<T>

java.lang.Object

org.apache.flink.streaming.connectors.kafka.partitioner.KafkaPartitioner<T>

org.apache.flink.streaming.connectors.kafka.partitioner.FixedPartitioner<T>

All Implemented Interfaces:

Serializable

public class FixedPartitioner<T>
extends KafkaPartitioner<T>
implements Serializable

A partitioner ensuring that each internal Flink partition ends up in one Kafka partition. Note, one Kafka partition can contain multiple Flink partitions. Cases: # More Flink partitions than kafka partitions

                Flink Sinks:            Kafka Partitions
                        1       ---------------->    1
                        2   --------------/
                        3   -------------/
                        4       ------------/
 

Some (or all) kafka partitions contain the output of more than one flink partition # Fewer Flink partitions than Kafka

                Flink Sinks:            Kafka Partitions
                        1       ---------------->    1
                        2       ---------------->    2
                                                                                3
                                                                                4
                                                                                5
 

Not all Kafka partitions contain data To avoid such an unbalanced partitioning, use a round-robin kafka partitioner. (note that this will cause a lot of network connections between all the Flink instances and all the Kafka brokers

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
FixedPartitioner()

Method Summary

Methods

Modifier and Type	Method and Description
void	open(int parallelInstanceId, int parallelInstances, int[] partitions) Initializer for the Partitioner.
int	partition(T next, byte[] serializedKey, byte[] serializedValue, int numPartitions)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FixedPartitioner

public FixedPartitioner()

Method Detail

open

public void open(int parallelInstanceId,
        int parallelInstances,
        int[] partitions)

Description copied from class: KafkaPartitioner

Initializer for the Partitioner.

Overrides:

open in class KafkaPartitioner<T>

Parameters:

parallelInstanceId - 0-indexed id of the parallel instance in Flink

parallelInstances - the total number of parallel instances

partitions - an array describing the partition IDs of the available Kafka partitions.

partition

public int partition(T next,
            byte[] serializedKey,
            byte[] serializedValue,
            int numPartitions)

Specified by:

partition in class KafkaPartitioner<T>