org.apache.flink.statefun.sdk.kafka

Class KafkaEgressBuilder<OutT>

java.lang.Object

org.apache.flink.statefun.sdk.kafka.KafkaEgressBuilder<OutT>

Type Parameters:

OutT - The type written out to the cluster by the Egress.

public final class KafkaEgressBuilder<OutT>
extends java.lang.Object

A builder class for creating an EgressSpec that writes data out to a Kafka cluster. By default the egress will use withAtLeastOnceProducerSemantics().

Method Summary

Modifier and Type	Method and Description
KafkaEgressSpec<OutT>	build()
static <OutT> KafkaEgressBuilder<OutT>	forIdentifier(EgressIdentifier<OutT> egressIdentifier)
KafkaEgressBuilder<OutT>	withAtLeastOnceProducerSemantics() KafkaProducerSemantic.AT_LEAST_ONCE the egress will wait for all outstanding messages in the Kafka buffers to be acknowledged by the Kafka producer on a checkpoint.
KafkaEgressBuilder<OutT>	withExactlyOnceProducerSemantics(java.time.Duration transactionTimeoutDuration) KafkaProducerSemantic.EXACTLY_ONCE the egress will write all messages in a Kafka transaction that will be committed to Kafka on a checkpoint.
KafkaEgressBuilder<OutT>	withKafkaAddress(java.lang.String kafkaAddress)
KafkaEgressBuilder<OutT>	withKafkaProducerPoolSize(int poolSize)
KafkaEgressBuilder<OutT>	withNoProducerSemantics() KafkaProducerSemantic.NONE means that nothing will be guaranteed.
KafkaEgressBuilder<OutT>	withProducerSemantic(KafkaProducerSemantic producerSemantic)
KafkaEgressBuilder<OutT>	withProperties(java.util.Properties properties) Configuration properties for the KafkaProducer.
KafkaEgressBuilder<OutT>	withProperty(java.lang.String key, java.lang.String value) A configuration property for the KafkaProducer.
KafkaEgressBuilder<OutT>	withSerializer(java.lang.Class<? extends KafkaEgressSerializer<OutT>> serializer)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

forIdentifier

public static <OutT> KafkaEgressBuilder<OutT> forIdentifier(EgressIdentifier<OutT> egressIdentifier)

Type Parameters:

OutT - The type the egress will output.

Parameters:

egressIdentifier - A unique egress identifier.

Returns:

A KafkaIngressBuilder.

withKafkaAddress

public KafkaEgressBuilder<OutT> withKafkaAddress(java.lang.String kafkaAddress)

Parameters:

kafkaAddress - Comma separated addresses of the brokers.

withProperty

public KafkaEgressBuilder<OutT> withProperty(java.lang.String key,
                                             java.lang.String value)

A configuration property for the KafkaProducer.

withProperties

public KafkaEgressBuilder<OutT> withProperties(java.util.Properties properties)

Configuration properties for the KafkaProducer.

withSerializer

public KafkaEgressBuilder<OutT> withSerializer(java.lang.Class<? extends KafkaEgressSerializer<OutT>> serializer)

Parameters:

serializer - A serializer schema for turning user objects into a kafka-consumable byte[] supporting key/value messages.

withKafkaProducerPoolSize

public KafkaEgressBuilder<OutT> withKafkaProducerPoolSize(int poolSize)

Parameters:

poolSize - Overwrite default KafkaProducers pool size. The default is 5.

withExactlyOnceProducerSemantics

public KafkaEgressBuilder<OutT> withExactlyOnceProducerSemantics(java.time.Duration transactionTimeoutDuration)

KafkaProducerSemantic.EXACTLY_ONCE the egress will write all messages in a Kafka transaction that will be committed to Kafka on a checkpoint.

With exactly-once producer semantics, users must also specify the transaction timeout. Note that this value must not be larger than the transaction.max.timeout.ms value configured on Kafka brokers (by default, this is 15 minutes).

Parameters:

transactionTimeoutDuration - the transaction timeout.

withAtLeastOnceProducerSemantics

public KafkaEgressBuilder<OutT> withAtLeastOnceProducerSemantics()

KafkaProducerSemantic.AT_LEAST_ONCE the egress will wait for all outstanding messages in the Kafka buffers to be acknowledged by the Kafka producer on a checkpoint.

withNoProducerSemantics

public KafkaEgressBuilder<OutT> withNoProducerSemantics()

KafkaProducerSemantic.NONE means that nothing will be guaranteed. Messages can be lost and/or duplicated in case of failure.

withProducerSemantic

public KafkaEgressBuilder<OutT> withProducerSemantic(KafkaProducerSemantic producerSemantic)

build

public KafkaEgressSpec<OutT> build()

Returns:

An EgressSpec that can be used in a StatefulFunctionModule.