KafkaSink (Flink : Connectors : Kafka 1.14.1 API)

java.lang.Object
- org.apache.flink.connector.kafka.sink.KafkaSink<IN>

Type Parameters:

IN - type of the records written to Kafka

All Implemented Interfaces:

Serializable, org.apache.flink.api.connector.sink.Sink<IN,org.apache.flink.connector.kafka.sink.KafkaCommittable,org.apache.flink.connector.kafka.sink.KafkaWriterState,Void>
```
@PublicEvolving
public class KafkaSink<IN>
extends Object
implements org.apache.flink.api.connector.sink.Sink<IN,org.apache.flink.connector.kafka.sink.KafkaCommittable,org.apache.flink.connector.kafka.sink.KafkaWriterState,Void>
```
Flink Sink to produce data into a Kafka topic. The sink supports all delivery guarantees described by DeliveryGuarantee.
DeliveryGuarantee.NONE does not provide any guarantees: messages may be lost in case of issues on the Kafka broker and messages may be duplicated in case of a Flink failure.
DeliveryGuarantee.AT_LEAST_ONCE the sink will wait for all outstanding records in the Kafka buffers to be acknowledged by the Kafka producer on a checkpoint. No messages will be lost in case of any issue with the Kafka brokers but messages may be duplicated when Flink restarts.
DeliveryGuarantee.EXACTLY_ONCE: In this mode the KafkaSink will write all messages in a Kafka transaction that will be committed to Kafka on a checkpoint. Thus, if the consumer reads only committed data (see Kafka consumer config isolation.level), no duplicates will be seen in case of a Flink restart. However, this delays record writing effectively until a checkpoint is written, so adjust the checkpoint duration accordingly. Please ensure that you use unique transactionalIdPrefixs across your applications running on the same Kafka cluster such that multiple running jobs do not interfere in their transactions! Additionally, it is highly recommended to tweak Kafka transaction timeout (link) >> maximum checkpoint duration + maximum restart duration or data loss may happen when Kafka expires an uncommitted transaction.

See Also:: on how to construct a KafkaSink, Serialized Form

Nested Class Summary
- Nested classes/interfaces inherited from interface org.apache.flink.api.connector.sink.Sink
  org.apache.flink.api.connector.sink.Sink.InitContext, org.apache.flink.api.connector.sink.Sink.ProcessingTimeService

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`static <IN> KafkaSinkBuilder<IN>`	`builder()` Create a `KafkaSinkBuilder` to construct a new `KafkaSink`.
`Optional<org.apache.flink.api.connector.sink.Committer<org.apache.flink.connector.kafka.sink.KafkaCommittable>>`	`createCommitter()`
`Optional<org.apache.flink.api.connector.sink.GlobalCommitter<org.apache.flink.connector.kafka.sink.KafkaCommittable,Void>>`	`createGlobalCommitter()`
`org.apache.flink.api.connector.sink.SinkWriter<IN,org.apache.flink.connector.kafka.sink.KafkaCommittable,org.apache.flink.connector.kafka.sink.KafkaWriterState>`	`createWriter(org.apache.flink.api.connector.sink.Sink.InitContext context, List<org.apache.flink.connector.kafka.sink.KafkaWriterState> states)`
`Optional<org.apache.flink.core.io.SimpleVersionedSerializer<org.apache.flink.connector.kafka.sink.KafkaCommittable>>`	`getCommittableSerializer()`
`Optional<org.apache.flink.core.io.SimpleVersionedSerializer<Void>>`	`getGlobalCommittableSerializer()`
`Optional<org.apache.flink.core.io.SimpleVersionedSerializer<org.apache.flink.connector.kafka.sink.KafkaWriterState>>`	`getWriterStateSerializer()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.connector.sink.Sink
getCompatibleStateNames

Method Detail

builder
```
public static <IN> KafkaSinkBuilder<IN> builder()
```
Create a KafkaSinkBuilder to construct a new KafkaSink.

Type Parameters:

IN - type of incoming records

Returns:

KafkaSinkBuilder

createWriter

public org.apache.flink.api.connector.sink.SinkWriter<IN,org.apache.flink.connector.kafka.sink.KafkaCommittable,org.apache.flink.connector.kafka.sink.KafkaWriterState> createWriter(org.apache.flink.api.connector.sink.Sink.InitContext context,
                                                                                                                                                                                     List<org.apache.flink.connector.kafka.sink.KafkaWriterState> states)
                                                                                                                                                                              throws IOException

Specified by:: createWriter in interface org.apache.flink.api.connector.sink.Sink<IN,org.apache.flink.connector.kafka.sink.KafkaCommittable,org.apache.flink.connector.kafka.sink.KafkaWriterState,Void>
Throws:: IOException

createCommitter

public Optional<org.apache.flink.api.connector.sink.Committer<org.apache.flink.connector.kafka.sink.KafkaCommittable>> createCommitter()
                                                                                                                                throws IOException

Specified by:: createCommitter in interface org.apache.flink.api.connector.sink.Sink<IN,org.apache.flink.connector.kafka.sink.KafkaCommittable,org.apache.flink.connector.kafka.sink.KafkaWriterState,Void>
Throws:: IOException

createGlobalCommitter

public Optional<org.apache.flink.api.connector.sink.GlobalCommitter<org.apache.flink.connector.kafka.sink.KafkaCommittable,Void>> createGlobalCommitter()
                                                                                                                                                 throws IOException

Specified by:: createGlobalCommitter in interface org.apache.flink.api.connector.sink.Sink<IN,org.apache.flink.connector.kafka.sink.KafkaCommittable,org.apache.flink.connector.kafka.sink.KafkaWriterState,Void>
Throws:: IOException

getCommittableSerializer
```
public Optional<org.apache.flink.core.io.SimpleVersionedSerializer<org.apache.flink.connector.kafka.sink.KafkaCommittable>> getCommittableSerializer()
```
Specified by:

getCommittableSerializer in interface org.apache.flink.api.connector.sink.Sink<IN,org.apache.flink.connector.kafka.sink.KafkaCommittable,org.apache.flink.connector.kafka.sink.KafkaWriterState,Void>

getGlobalCommittableSerializer
```
public Optional<org.apache.flink.core.io.SimpleVersionedSerializer<Void>> getGlobalCommittableSerializer()
```
Specified by:

getGlobalCommittableSerializer in interface org.apache.flink.api.connector.sink.Sink<IN,org.apache.flink.connector.kafka.sink.KafkaCommittable,org.apache.flink.connector.kafka.sink.KafkaWriterState,Void>

getWriterStateSerializer
```
public Optional<org.apache.flink.core.io.SimpleVersionedSerializer<org.apache.flink.connector.kafka.sink.KafkaWriterState>> getWriterStateSerializer()
```
Specified by:

getWriterStateSerializer in interface org.apache.flink.api.connector.sink.Sink<IN,org.apache.flink.connector.kafka.sink.KafkaCommittable,org.apache.flink.connector.kafka.sink.KafkaWriterState,Void>

Class KafkaSink<IN>

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.api.connector.sink.Sink

Method Summary

Methods inherited from class java.lang.Object

Methods inherited from interface org.apache.flink.api.connector.sink.Sink

Method Detail

builder

createWriter

createCommitter

createGlobalCommitter

getCommittableSerializer

getGlobalCommittableSerializer

getWriterStateSerializer