org.apache.flink.streaming.connectors.kafka

Class FlinkKafkaProducer<IN>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.streaming.api.functions.sink.RichSinkFunction<IN>

org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer<IN>

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction, org.apache.flink.runtime.state.CheckpointListener, org.apache.flink.streaming.api.checkpoint.CheckpointedFunction, org.apache.flink.streaming.api.functions.sink.SinkFunction<IN>

@PublicEvolving
public class FlinkKafkaProducer<IN>
extends org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

Flink Sink to produce data into a Kafka topic. By default producer will use FlinkKafkaProducer.Semantic.AT_LEAST_ONCE semantic. Before using FlinkKafkaProducer.Semantic.EXACTLY_ONCE please refer to Flink's Kafka connector documentation.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	FlinkKafkaProducer.ContextStateSerializer TypeSerializer for FlinkKafkaProducer.KafkaTransactionContext.
static class	FlinkKafkaProducer.KafkaTransactionContext Context associated to this instance of the FlinkKafkaProducer.
static class	FlinkKafkaProducer.NextTransactionalIdHint Keep information required to deduce next safe to use transactional id.
static class	FlinkKafkaProducer.Semantic Semantics that can be chosen.
static class	FlinkKafkaProducer.TransactionStateSerializer TypeSerializer for FlinkKafkaProducer.KafkaTransactionState.

Nested classes/interfaces inherited from class org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction

org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction.State<TXN,CONTEXT>, org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction.StateSerializer<TXN,CONTEXT>, org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction.StateSerializerConfigSnapshot<TXN,CONTEXT>, org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction.TransactionHolder<TXN>

Nested classes/interfaces inherited from interface org.apache.flink.streaming.api.functions.sink.SinkFunction

org.apache.flink.streaming.api.functions.sink.SinkFunction.Context<T>

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_KAFKA_PRODUCERS_POOL_SIZE Default number of KafkaProducers in the pool.
static org.apache.flink.api.common.time.Time	DEFAULT_KAFKA_TRANSACTION_TIMEOUT Default value for kafka transaction timeout.
static String	KEY_DISABLE_METRICS Configuration key for disabling the metrics reporting.
protected Properties	producerConfig User defined properties for the Producer.
static int	SAFE_SCALE_DOWN_FACTOR This coefficient determines what is the safe scale down factor.
protected FlinkKafkaProducer.Semantic	semantic Semantic chosen for this instance.

Fields inherited from class org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction

pendingCommitTransactions, state, userContext

Constructor Summary

Constructors

Constructor and Description
FlinkKafkaProducer(String topicId, org.apache.flink.streaming.util.serialization.KeyedSerializationSchema<IN> serializationSchema, Properties producerConfig) Creates a FlinkKafkaProducer for a given topic.
FlinkKafkaProducer(String topicId, org.apache.flink.streaming.util.serialization.KeyedSerializationSchema<IN> serializationSchema, Properties producerConfig, FlinkKafkaProducer.Semantic semantic) Creates a FlinkKafkaProducer for a given topic.
FlinkKafkaProducer(String defaultTopicId, org.apache.flink.streaming.util.serialization.KeyedSerializationSchema<IN> serializationSchema, Properties producerConfig, Optional<org.apache.flink.streaming.connectors.kafka.partitioner.FlinkKafkaPartitioner<IN>> customPartitioner) Creates a FlinkKafkaProducer for a given topic.
FlinkKafkaProducer(String defaultTopicId, org.apache.flink.streaming.util.serialization.KeyedSerializationSchema<IN> serializationSchema, Properties producerConfig, Optional<org.apache.flink.streaming.connectors.kafka.partitioner.FlinkKafkaPartitioner<IN>> customPartitioner, FlinkKafkaProducer.Semantic semantic, int kafkaProducersPoolSize) Creates a FlinkKafkaProducer for a given topic.
FlinkKafkaProducer(String topicId, org.apache.flink.api.common.serialization.SerializationSchema<IN> serializationSchema, Properties producerConfig) Creates a FlinkKafkaProducer for a given topic.
FlinkKafkaProducer(String topicId, org.apache.flink.api.common.serialization.SerializationSchema<IN> serializationSchema, Properties producerConfig, Optional<org.apache.flink.streaming.connectors.kafka.partitioner.FlinkKafkaPartitioner<IN>> customPartitioner) Creates a FlinkKafkaProducer for a given topic.
FlinkKafkaProducer(String brokerList, String topicId, org.apache.flink.streaming.util.serialization.KeyedSerializationSchema<IN> serializationSchema) Creates a FlinkKafkaProducer for a given topic.
FlinkKafkaProducer(String brokerList, String topicId, org.apache.flink.api.common.serialization.SerializationSchema<IN> serializationSchema) Creates a FlinkKafkaProducer for a given topic.

Method Summary

Modifier and Type	Method and Description
protected void	abort(org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState transaction)
protected org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState	beginTransaction()
void	close()
protected void	commit(org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState transaction)
protected FlinkKafkaInternalProducer<byte[],byte[]>	createProducer()
protected void	finishRecoveringContext()
FlinkKafkaProducer<IN>	ignoreFailuresAfterTransactionTimeout() Disables the propagation of exceptions thrown when committing presumably timed out Kafka transactions during recovery of the job.
void	initializeState(org.apache.flink.runtime.state.FunctionInitializationContext context)
protected Optional<FlinkKafkaProducer.KafkaTransactionContext>	initializeUserContext()
void	invoke(org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState transaction, IN next, org.apache.flink.streaming.api.functions.sink.SinkFunction.Context context)
void	open(org.apache.flink.configuration.Configuration configuration) Initializes the connection to Kafka.
protected void	preCommit(org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState transaction)
protected void	recoverAndAbort(org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState transaction)
protected void	recoverAndCommit(org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState transaction)
void	setLogFailuresOnly(boolean logFailuresOnly) Defines whether the producer should fail on errors, or only log them.
void	setWriteTimestampToKafka(boolean writeTimestampToKafka) If set to true, Flink will write the (event time) timestamp attached to each record into Kafka.
void	snapshotState(org.apache.flink.runtime.state.FunctionSnapshotContext context)

Methods inherited from class org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction

currentTransaction, enableTransactionTimeoutWarnings, getUserContext, invoke, invoke, notifyCheckpointComplete, pendingTransactions, setTransactionTimeout

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SAFE_SCALE_DOWN_FACTOR

public static final int SAFE_SCALE_DOWN_FACTOR

This coefficient determines what is the safe scale down factor.

If the Flink application previously failed before first checkpoint completed or we are starting new batch of FlinkKafkaProducer from scratch without clean shutdown of the previous one, FlinkKafkaProducer doesn't know what was the set of previously used Kafka's transactionalId's. In that case, it will try to play safe and abort all of the possible transactionalIds from the range of: [0, getNumberOfParallelSubtasks() * kafkaProducersPoolSize * SAFE_SCALE_DOWN_FACTOR)

The range of available to use transactional ids is: [0, getNumberOfParallelSubtasks() * kafkaProducersPoolSize)

This means that if we decrease getNumberOfParallelSubtasks() by a factor larger then SAFE_SCALE_DOWN_FACTOR we can have a left some lingering transaction.

See Also:

Constant Field Values

DEFAULT_KAFKA_PRODUCERS_POOL_SIZE

public static final int DEFAULT_KAFKA_PRODUCERS_POOL_SIZE

Default number of KafkaProducers in the pool. See FlinkKafkaProducer.Semantic.EXACTLY_ONCE.

See Also:

Constant Field Values

DEFAULT_KAFKA_TRANSACTION_TIMEOUT

public static final org.apache.flink.api.common.time.Time DEFAULT_KAFKA_TRANSACTION_TIMEOUT

Default value for kafka transaction timeout.

KEY_DISABLE_METRICS

public static final String KEY_DISABLE_METRICS

Configuration key for disabling the metrics reporting.

See Also:

Constant Field Values

producerConfig

protected final Properties producerConfig

User defined properties for the Producer.

semantic

protected FlinkKafkaProducer.Semantic semantic

Semantic chosen for this instance.

Constructor Detail

FlinkKafkaProducer

public FlinkKafkaProducer(String brokerList,
                          String topicId,
                          org.apache.flink.api.common.serialization.SerializationSchema<IN> serializationSchema)

Creates a FlinkKafkaProducer for a given topic. The sink produces a DataStream to the topic.

Parameters:

brokerList - Comma separated addresses of the brokers

topicId - ID of the Kafka topic.

serializationSchema - User defined (keyless) serialization schema.

FlinkKafkaProducer

public FlinkKafkaProducer(String topicId,
                          org.apache.flink.api.common.serialization.SerializationSchema<IN> serializationSchema,
                          Properties producerConfig)

Creates a FlinkKafkaProducer for a given topic. The sink produces a DataStream to the topic.

Using this constructor, the default FlinkFixedPartitioner will be used as the partitioner. This default partitioner maps each sink subtask to a single Kafka partition (i.e. all records received by a sink subtask will end up in the same Kafka partition).

To use a custom partitioner, please use FlinkKafkaProducer(String, SerializationSchema, Properties, Optional) instead.

Parameters:

topicId - ID of the Kafka topic.

serializationSchema - User defined key-less serialization schema.

producerConfig - Properties with the producer configuration.

FlinkKafkaProducer

public FlinkKafkaProducer(String topicId,
                          org.apache.flink.api.common.serialization.SerializationSchema<IN> serializationSchema,
                          Properties producerConfig,
                          Optional<org.apache.flink.streaming.connectors.kafka.partitioner.FlinkKafkaPartitioner<IN>> customPartitioner)

Creates a FlinkKafkaProducer for a given topic. The sink produces its input to the topic. It accepts a key-less SerializationSchema and possibly a custom FlinkKafkaPartitioner.

Since a key-less SerializationSchema is used, all records sent to Kafka will not have an attached key. Therefore, if a partitioner is also not provided, records will be distributed to Kafka partitions in a round-robin fashion.

Parameters:

topicId - The topic to write data to

serializationSchema - A key-less serializable serialization schema for turning user objects into a kafka-consumable byte[]

producerConfig - Configuration properties for the KafkaProducer. 'bootstrap.servers.' is the only required argument.

customPartitioner - A serializable partitioner for assigning messages to Kafka partitions. If a partitioner is not provided, records will be distributed to Kafka partitions in a round-robin fashion.

FlinkKafkaProducer

public FlinkKafkaProducer(String brokerList,
                          String topicId,
                          org.apache.flink.streaming.util.serialization.KeyedSerializationSchema<IN> serializationSchema)

Creates a FlinkKafkaProducer for a given topic. The sink produces a DataStream to the topic.

Using this constructor, the default FlinkFixedPartitioner will be used as the partitioner. This default partitioner maps each sink subtask to a single Kafka partition (i.e. all records received by a sink subtask will end up in the same Kafka partition).

To use a custom partitioner, please use FlinkKafkaProducer(String, KeyedSerializationSchema, Properties, Optional) instead.

Parameters:

brokerList - Comma separated addresses of the brokers

topicId - ID of the Kafka topic.

serializationSchema - User defined serialization schema supporting key/value messages

FlinkKafkaProducer

public FlinkKafkaProducer(String topicId,
                          org.apache.flink.streaming.util.serialization.KeyedSerializationSchema<IN> serializationSchema,
                          Properties producerConfig)

Creates a FlinkKafkaProducer for a given topic. The sink produces a DataStream to the topic.

Using this constructor, the default FlinkFixedPartitioner will be used as the partitioner. This default partitioner maps each sink subtask to a single Kafka partition (i.e. all records received by a sink subtask will end up in the same Kafka partition).

To use a custom partitioner, please use FlinkKafkaProducer(String, KeyedSerializationSchema, Properties, Optional) instead.

Parameters:

topicId - ID of the Kafka topic.

serializationSchema - User defined serialization schema supporting key/value messages

producerConfig - Properties with the producer configuration.

FlinkKafkaProducer

public FlinkKafkaProducer(String topicId,
                          org.apache.flink.streaming.util.serialization.KeyedSerializationSchema<IN> serializationSchema,
                          Properties producerConfig,
                          FlinkKafkaProducer.Semantic semantic)

Creates a FlinkKafkaProducer for a given topic. The sink produces a DataStream to the topic.

Using this constructor, the default FlinkFixedPartitioner will be used as the partitioner. This default partitioner maps each sink subtask to a single Kafka partition (i.e. all records received by a sink subtask will end up in the same Kafka partition).

To use a custom partitioner, please use FlinkKafkaProducer(String, KeyedSerializationSchema, Properties, Optional, FlinkKafkaProducer.Semantic, int) instead.

Parameters:

topicId - ID of the Kafka topic.

serializationSchema - User defined serialization schema supporting key/value messages

producerConfig - Properties with the producer configuration.

semantic - Defines semantic that will be used by this producer (see FlinkKafkaProducer.Semantic).

FlinkKafkaProducer

public FlinkKafkaProducer(String defaultTopicId,
                          org.apache.flink.streaming.util.serialization.KeyedSerializationSchema<IN> serializationSchema,
                          Properties producerConfig,
                          Optional<org.apache.flink.streaming.connectors.kafka.partitioner.FlinkKafkaPartitioner<IN>> customPartitioner)

Creates a FlinkKafkaProducer for a given topic. The sink produces its input to the topic. It accepts a keyed KeyedSerializationSchema and possibly a custom FlinkKafkaPartitioner.

If a partitioner is not provided, written records will be partitioned by the attached key of each record (as determined by KeyedSerializationSchema.serializeKey(Object)). If written records do not have a key (i.e., KeyedSerializationSchema.serializeKey(Object) returns null), they will be distributed to Kafka partitions in a round-robin fashion.

Parameters:

defaultTopicId - The default topic to write data to

serializationSchema - A serializable serialization schema for turning user objects into a kafka-consumable byte[] supporting key/value messages

producerConfig - Configuration properties for the KafkaProducer. 'bootstrap.servers.' is the only required argument.

customPartitioner - A serializable partitioner for assigning messages to Kafka partitions. If a partitioner is not provided, records will be partitioned by the key of each record (determined by KeyedSerializationSchema.serializeKey(Object)). If the keys are null, then records will be distributed to Kafka partitions in a round-robin fashion.

FlinkKafkaProducer

public FlinkKafkaProducer(String defaultTopicId,
                          org.apache.flink.streaming.util.serialization.KeyedSerializationSchema<IN> serializationSchema,
                          Properties producerConfig,
                          Optional<org.apache.flink.streaming.connectors.kafka.partitioner.FlinkKafkaPartitioner<IN>> customPartitioner,
                          FlinkKafkaProducer.Semantic semantic,
                          int kafkaProducersPoolSize)

Creates a FlinkKafkaProducer for a given topic. The sink produces its input to the topic. It accepts a keyed KeyedSerializationSchema and possibly a custom FlinkKafkaPartitioner.

If a partitioner is not provided, written records will be partitioned by the attached key of each record (as determined by KeyedSerializationSchema.serializeKey(Object)). If written records do not have a key (i.e., KeyedSerializationSchema.serializeKey(Object) returns null), they will be distributed to Kafka partitions in a round-robin fashion.

Parameters:

defaultTopicId - The default topic to write data to

serializationSchema - A serializable serialization schema for turning user objects into a kafka-consumable byte[] supporting key/value messages

producerConfig - Configuration properties for the KafkaProducer. 'bootstrap.servers.' is the only required argument.

customPartitioner - A serializable partitioner for assigning messages to Kafka partitions. If a partitioner is not provided, records will be partitioned by the key of each record (determined by KeyedSerializationSchema.serializeKey(Object)). If the keys are null, then records will be distributed to Kafka partitions in a round-robin fashion.

semantic - Defines semantic that will be used by this producer (see FlinkKafkaProducer.Semantic).

kafkaProducersPoolSize - Overwrite default KafkaProducers pool size (see FlinkKafkaProducer.Semantic.EXACTLY_ONCE).

Method Detail

setWriteTimestampToKafka

public void setWriteTimestampToKafka(boolean writeTimestampToKafka)

If set to true, Flink will write the (event time) timestamp attached to each record into Kafka. Timestamps must be positive for Kafka to accept them.

Parameters:

writeTimestampToKafka - Flag indicating if Flink's internal timestamps are written to Kafka.

setLogFailuresOnly

public void setLogFailuresOnly(boolean logFailuresOnly)

Defines whether the producer should fail on errors, or only log them. If this is set to true, then exceptions will be only logged, if set to false, exceptions will be eventually thrown and cause the streaming program to fail (and enter recovery).

Parameters:

logFailuresOnly - The flag to indicate logging-only on exceptions.

ignoreFailuresAfterTransactionTimeout

public FlinkKafkaProducer<IN> ignoreFailuresAfterTransactionTimeout()

Disables the propagation of exceptions thrown when committing presumably timed out Kafka transactions during recovery of the job. If a Kafka transaction is timed out, a commit will never be successful. Hence, use this feature to avoid recovery loops of the Job. Exceptions will still be logged to inform the user that data loss might have occurred.

Note that we use System.currentTimeMillis() to track the age of a transaction. Moreover, only exceptions thrown during the recovery are caught, i.e., the producer will attempt at least one commit of the transaction before giving up.

Overrides:

ignoreFailuresAfterTransactionTimeout in class org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

open

public void open(org.apache.flink.configuration.Configuration configuration)
          throws Exception

Initializes the connection to Kafka.

Specified by:

open in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

open in class org.apache.flink.api.common.functions.AbstractRichFunction

Throws:

Exception

invoke

public void invoke(org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState transaction,
                   IN next,
                   org.apache.flink.streaming.api.functions.sink.SinkFunction.Context context)
            throws FlinkKafkaException

Specified by:

invoke in class org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

Throws:

FlinkKafkaException

close

public void close()
           throws FlinkKafkaException

Specified by:

close in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

close in class org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

Throws:

FlinkKafkaException

beginTransaction

protected org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState beginTransaction()
                                                                                                         throws FlinkKafkaException

Specified by:

beginTransaction in class org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

Throws:

FlinkKafkaException

preCommit

protected void preCommit(org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState transaction)
                  throws FlinkKafkaException

Specified by:

preCommit in class org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

Throws:

FlinkKafkaException

commit

protected void commit(org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState transaction)

Specified by:

commit in class org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

recoverAndCommit

protected void recoverAndCommit(org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState transaction)

Overrides:

recoverAndCommit in class org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

abort

protected void abort(org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState transaction)

Specified by:

abort in class org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

recoverAndAbort

protected void recoverAndAbort(org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState transaction)

Overrides:

recoverAndAbort in class org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

snapshotState

public void snapshotState(org.apache.flink.runtime.state.FunctionSnapshotContext context)
                   throws Exception

Specified by:

snapshotState in interface org.apache.flink.streaming.api.checkpoint.CheckpointedFunction

Overrides:

snapshotState in class org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

Throws:

Exception

initializeState

public void initializeState(org.apache.flink.runtime.state.FunctionInitializationContext context)
                     throws Exception

Specified by:

initializeState in interface org.apache.flink.streaming.api.checkpoint.CheckpointedFunction

Overrides:

initializeState in class org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

Throws:

Exception

initializeUserContext

protected Optional<FlinkKafkaProducer.KafkaTransactionContext> initializeUserContext()

Overrides:

initializeUserContext in class org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

finishRecoveringContext

protected void finishRecoveringContext()

Overrides:

finishRecoveringContext in class org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

createProducer

protected FlinkKafkaInternalProducer<byte[],byte[]> createProducer()