org.apache.flink.connector.kafka.sink

Class KafkaSinkBuilder<IN>

java.lang.Object

org.apache.flink.connector.kafka.sink.KafkaSinkBuilder<IN>

Type Parameters:

IN - type of the records written to Kafka

@PublicEvolving
public class KafkaSinkBuilder<IN>
extends Object

Builder to construct KafkaSink.

The following example shows the minimum setup to create a KafkaSink that writes String values to a Kafka topic.

 KafkaSink<String> sink = KafkaSink
     .<String>builder
     .setBootstrapServers(MY_BOOTSTRAP_SERVERS)
     .setRecordSerializer(MY_RECORD_SERIALIZER)
     .build();
 

One can also configure different DeliveryGuarantee by using setDeliverGuarantee(DeliveryGuarantee) but keep in mind when using DeliveryGuarantee.EXACTLY_ONCE one must set the transactionalIdPrefix setTransactionalIdPrefix(String).

See Also:

for a more detailed explanation of the different guarantees.

Method Summary

Modifier and Type	Method and Description
KafkaSink<IN>	build() Constructs the KafkaSink with the configured properties.
KafkaSinkBuilder<IN>	setBootstrapServers(String bootstrapServers) Sets the Kafka bootstrap servers.
KafkaSinkBuilder<IN>	setDeliverGuarantee(org.apache.flink.connector.base.DeliveryGuarantee deliveryGuarantee) Sets the wanted the DeliveryGuarantee.
KafkaSinkBuilder<IN>	setKafkaProducerConfig(Properties kafkaProducerConfig) Sets the configuration which used to instantiate all used KafkaProducer.
KafkaSinkBuilder<IN>	setRecordSerializer(KafkaRecordSerializationSchema<IN> recordSerializer) Sets the KafkaRecordSerializationSchema that transforms incoming records to ProducerRecords.
KafkaSinkBuilder<IN>	setTransactionalIdPrefix(String transactionalIdPrefix) Sets the prefix for all created transactionalIds if DeliveryGuarantee.EXACTLY_ONCE is configured.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

setDeliverGuarantee

public KafkaSinkBuilder<IN> setDeliverGuarantee(org.apache.flink.connector.base.DeliveryGuarantee deliveryGuarantee)

Sets the wanted the DeliveryGuarantee. The default delivery guarantee is deliveryGuarantee.

Parameters:

deliveryGuarantee -

Returns:

KafkaSinkBuilder

setKafkaProducerConfig

public KafkaSinkBuilder<IN> setKafkaProducerConfig(Properties kafkaProducerConfig)

Sets the configuration which used to instantiate all used KafkaProducer.

Parameters:

kafkaProducerConfig -

Returns:

KafkaSinkBuilder

setRecordSerializer

public KafkaSinkBuilder<IN> setRecordSerializer(KafkaRecordSerializationSchema<IN> recordSerializer)

Sets the KafkaRecordSerializationSchema that transforms incoming records to ProducerRecords.

Parameters:

recordSerializer -

Returns:

KafkaSinkBuilder

setTransactionalIdPrefix

public KafkaSinkBuilder<IN> setTransactionalIdPrefix(String transactionalIdPrefix)

Sets the prefix for all created transactionalIds if DeliveryGuarantee.EXACTLY_ONCE is configured.

It is mandatory to always set this value with DeliveryGuarantee.EXACTLY_ONCE to prevent corrupted transactions if multiple jobs using the KafkaSink run against the same Kafka Cluster. The default prefix is transactionalIdPrefix.

The size of the prefix is capped by MAXIMUM_PREFIX_BYTES formatted with UTF-8.

It is important to keep the prefix stable across application restarts. If the prefix changes it might happen that lingering transactions are not correctly aborted and newly written messages are not immediately consumable until the transactions timeout.

Parameters:

transactionalIdPrefix -

Returns:

KafkaSinkBuilder

setBootstrapServers

public KafkaSinkBuilder<IN> setBootstrapServers(String bootstrapServers)

Sets the Kafka bootstrap servers.

Parameters:

bootstrapServers - a comma separated list of valid URIs to reach the Kafka broker

Returns:

KafkaSinkBuilder

build

public KafkaSink<IN> build()

Constructs the KafkaSink with the configured properties.

Returns:

KafkaSink