org.apache.beam.sdk.io.kafka

Class KafkaIO.TypedWrite<K,V>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PCollection<KV<K,V>>,PDone>

org.apache.beam.sdk.io.kafka.KafkaIO.TypedWrite<K,V>

All Implemented Interfaces:

Serializable, HasDisplayData

Direct Known Subclasses:

KafkaIO.Write

Enclosing class:

KafkaIO

public static class KafkaIO.TypedWrite<K,V>
extends PTransform<PCollection<KV<K,V>>,PDone>

A PTransform to write to a Kafka topic. See KafkaIO for more information on usage and configuration.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected Coder<K>	keyCoder
protected Map<String,Object>	producerConfig
protected com.google.common.base.Optional<SerializableFunction<Map<String,Object>,org.apache.kafka.clients.producer.Producer<K,V>>>	producerFactoryFnOpt
protected String	topic
protected Coder<V>	valueCoder

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	TypedWrite(String topic, Coder<K> keyCoder, Coder<V> valueCoder, Map<String,Object> producerConfig, com.google.common.base.Optional<SerializableFunction<Map<String,Object>,org.apache.kafka.clients.producer.Producer<K,V>>> producerFactoryFnOpt)

Method Summary

Modifier and Type	Method and Description
PDone	apply(PCollection<KV<K,V>> input)
void	validate(PCollection<KV<K,V>> input)
PTransform<PCollection<V>,PDone>	values() Returns a new transform that writes just the values to Kafka.
KafkaIO.TypedWrite<K,V>	withProducerFactoryFn(SerializableFunction<Map<String,Object>,org.apache.kafka.clients.producer.Producer<K,V>> producerFactoryFn) Returns a new KafkaIO.Write with a custom function to create Kafka producer.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, populateDisplayData, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

topic

protected final String topic

keyCoder

protected final Coder<K> keyCoder

valueCoder

protected final Coder<V> valueCoder

producerFactoryFnOpt

protected final com.google.common.base.Optional<SerializableFunction<Map<String,Object>,org.apache.kafka.clients.producer.Producer<K,V>>> producerFactoryFnOpt

producerConfig

protected final Map<String,Object> producerConfig

Constructor Detail

TypedWrite

protected TypedWrite(String topic,
                     Coder<K> keyCoder,
                     Coder<V> valueCoder,
                     Map<String,Object> producerConfig,
                     com.google.common.base.Optional<SerializableFunction<Map<String,Object>,org.apache.kafka.clients.producer.Producer<K,V>>> producerFactoryFnOpt)

Method Detail

withProducerFactoryFn

public KafkaIO.TypedWrite<K,V> withProducerFactoryFn(SerializableFunction<Map<String,Object>,org.apache.kafka.clients.producer.Producer<K,V>> producerFactoryFn)

Returns a new KafkaIO.Write with a custom function to create Kafka producer. Primarily used for tests. Default is KafkaProducer

values

public PTransform<PCollection<V>,PDone> values()

Returns a new transform that writes just the values to Kafka. This is useful for writing collections of values rather thank KVs.

apply

public PDone apply(PCollection<KV<K,V>> input)

Overrides:

apply in class PTransform<PCollection<KV<K,V>>,PDone>

validate

public void validate(PCollection<KV<K,V>> input)

Overrides:

validate in class PTransform<PCollection<KV<K,V>>,PDone>