org.apache.flink.streaming.connectors.kafka

类 FlinkKafkaProducer010<T>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.streaming.api.functions.sink.RichSinkFunction<IN>

org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerBase<T>

org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer010<T>

所有已实现的接口:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction, org.apache.flink.streaming.api.checkpoint.CheckpointedFunction, org.apache.flink.streaming.api.functions.sink.SinkFunction<T>

@PublicEvolving
public class FlinkKafkaProducer010<T>
extends org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerBase<T>

Flink Sink to produce data into a Kafka topic. This producer is compatible with Kafka 0.10.x

另请参阅:

序列化表格

嵌套类概要

嵌套类

限定符和类型	类和说明
static class	FlinkKafkaProducer010.FlinkKafkaProducer010Configuration<T> 已过时。 This class is deprecated since the factory methods writeToKafkaWithTimestamps for the producer are also deprecated.

从接口继承的嵌套类/接口 org.apache.flink.streaming.api.functions.sink.SinkFunction

org.apache.flink.streaming.api.functions.sink.SinkFunction.Context<T>

字段概要

从类继承的字段 org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerBase

asyncException, callback, defaultTopicId, flinkKafkaPartitioner, flushOnCheckpoint, KEY_DISABLE_METRICS, logFailuresOnly, pendingRecords, pendingRecordsLock, producer, producerConfig, schema, topicPartitionsMap

构造器概要

构造器

构造器和说明
FlinkKafkaProducer010(String topicId, org.apache.flink.streaming.util.serialization.KeyedSerializationSchema<T> serializationSchema, Properties producerConfig) Creates a FlinkKafkaProducer for a given topic.
FlinkKafkaProducer010(String topicId, org.apache.flink.streaming.util.serialization.KeyedSerializationSchema<T> serializationSchema, Properties producerConfig, org.apache.flink.streaming.connectors.kafka.partitioner.FlinkKafkaPartitioner<T> customPartitioner) Creates a FlinkKafkaProducer for a given topic.
FlinkKafkaProducer010(String topicId, org.apache.flink.api.common.serialization.SerializationSchema<T> serializationSchema, Properties producerConfig) Creates a FlinkKafkaProducer for a given topic. the sink produces a DataStream to the topic.
FlinkKafkaProducer010(String topicId, org.apache.flink.api.common.serialization.SerializationSchema<T> serializationSchema, Properties producerConfig, org.apache.flink.streaming.connectors.kafka.partitioner.FlinkKafkaPartitioner<T> customPartitioner) Creates a FlinkKafkaProducer for a given topic.
FlinkKafkaProducer010(String brokerList, String topicId, org.apache.flink.streaming.util.serialization.KeyedSerializationSchema<T> serializationSchema) Creates a FlinkKafkaProducer for a given topic.
FlinkKafkaProducer010(String brokerList, String topicId, org.apache.flink.api.common.serialization.SerializationSchema<T> serializationSchema) Creates a FlinkKafkaProducer for a given topic.

方法概要

限定符和类型	方法和说明
protected void	flush()
void	invoke(T value, org.apache.flink.streaming.api.functions.sink.SinkFunction.Context context)
void	setWriteTimestampToKafka(boolean writeTimestampToKafka) If set to true, Flink will write the (event time) timestamp attached to each record into Kafka.
static <T> FlinkKafkaProducer010.FlinkKafkaProducer010Configuration<T>	writeToKafkaWithTimestamps(org.apache.flink.streaming.api.datastream.DataStream<T> inStream, String topicId, org.apache.flink.streaming.util.serialization.KeyedSerializationSchema<T> serializationSchema, Properties producerConfig) 已过时。 Use FlinkKafkaProducer010(String, KeyedSerializationSchema, Properties) and call setWriteTimestampToKafka(boolean).
static <T> FlinkKafkaProducer010.FlinkKafkaProducer010Configuration<T>	writeToKafkaWithTimestamps(org.apache.flink.streaming.api.datastream.DataStream<T> inStream, String topicId, org.apache.flink.streaming.util.serialization.KeyedSerializationSchema<T> serializationSchema, Properties producerConfig, org.apache.flink.streaming.connectors.kafka.partitioner.FlinkKafkaPartitioner<T> customPartitioner) 已过时。 Use FlinkKafkaProducer010(String, KeyedSerializationSchema, Properties, FlinkKafkaPartitioner) and call setWriteTimestampToKafka(boolean).
static <T> FlinkKafkaProducer010.FlinkKafkaProducer010Configuration<T>	writeToKafkaWithTimestamps(org.apache.flink.streaming.api.datastream.DataStream<T> inStream, String topicId, org.apache.flink.api.common.serialization.SerializationSchema<T> serializationSchema, Properties producerConfig) 已过时。 Use FlinkKafkaProducer010(String, KeyedSerializationSchema, Properties) and call setWriteTimestampToKafka(boolean).

从类继承的方法 org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerBase

checkErroneous, close, getKafkaProducer, getPartitionsByTopic, getPropertiesFromBrokerList, initializeState, numPendingRecords, open, setFlushOnCheckpoint, setLogFailuresOnly, snapshotState

从类继承的方法 org.apache.flink.api.common.functions.AbstractRichFunction

getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.flink.streaming.api.functions.sink.SinkFunction

invoke

构造器详细资料

FlinkKafkaProducer010

public FlinkKafkaProducer010(String brokerList,
                             String topicId,
                             org.apache.flink.api.common.serialization.SerializationSchema<T> serializationSchema)

Creates a FlinkKafkaProducer for a given topic. The sink produces a DataStream to the topic.

Using this constructor, the default FlinkFixedPartitioner will be used as the partitioner. This default partitioner maps each sink subtask to a single Kafka partition (i.e. all records received by a sink subtask will end up in the same Kafka partition).

To use a custom partitioner, please use FlinkKafkaProducer010(String, SerializationSchema, Properties, FlinkKafkaPartitioner) instead.

参数:

brokerList - Comma separated addresses of the brokers

topicId - ID of the Kafka topic.

serializationSchema - User defined key-less serialization schema.

FlinkKafkaProducer010

public FlinkKafkaProducer010(String topicId,
                             org.apache.flink.api.common.serialization.SerializationSchema<T> serializationSchema,
                             Properties producerConfig)

Creates a FlinkKafkaProducer for a given topic. the sink produces a DataStream to the topic.

Using this constructor, the default FlinkFixedPartitioner will be used as the partitioner. This default partitioner maps each sink subtask to a single Kafka partition (i.e. all records received by a sink subtask will end up in the same Kafka partition).

To use a custom partitioner, please use FlinkKafkaProducer010(String, SerializationSchema, Properties, FlinkKafkaPartitioner) instead.

参数:

topicId - ID of the Kafka topic.

serializationSchema - User defined key-less serialization schema.

producerConfig - Properties with the producer configuration.

FlinkKafkaProducer010

public FlinkKafkaProducer010(String topicId,
                             org.apache.flink.api.common.serialization.SerializationSchema<T> serializationSchema,
                             Properties producerConfig,
                             @Nullable
                             org.apache.flink.streaming.connectors.kafka.partitioner.FlinkKafkaPartitioner<T> customPartitioner)

Creates a FlinkKafkaProducer for a given topic. The sink produces its input to the topic. It accepts a key-less SerializationSchema and possibly a custom FlinkKafkaPartitioner.

Since a key-less SerializationSchema is used, all records sent to Kafka will not have an attached key. Therefore, if a partitioner is also not provided, records will be distributed to Kafka partitions in a round-robin fashion.

参数:

topicId - The topic to write data to

serializationSchema - A key-less serializable serialization schema for turning user objects into a kafka-consumable byte[]

producerConfig - Configuration properties for the KafkaProducer. 'bootstrap.servers.' is the only required argument.

customPartitioner - A serializable partitioner for assigning messages to Kafka partitions. If set to null, records will be distributed to Kafka partitions in a round-robin fashion.

FlinkKafkaProducer010

public FlinkKafkaProducer010(String brokerList,
                             String topicId,
                             org.apache.flink.streaming.util.serialization.KeyedSerializationSchema<T> serializationSchema)

Creates a FlinkKafkaProducer for a given topic. The sink produces a DataStream to the topic.

Using this constructor, the default FlinkFixedPartitioner will be used as the partitioner. This default partitioner maps each sink subtask to a single Kafka partition (i.e. all records received by a sink subtask will end up in the same Kafka partition).

To use a custom partitioner, please use FlinkKafkaProducer010(String, KeyedSerializationSchema, Properties, FlinkKafkaPartitioner) instead.

参数:

brokerList - Comma separated addresses of the brokers

topicId - ID of the Kafka topic.

serializationSchema - User defined serialization schema supporting key/value messages

FlinkKafkaProducer010

public FlinkKafkaProducer010(String topicId,
                             org.apache.flink.streaming.util.serialization.KeyedSerializationSchema<T> serializationSchema,
                             Properties producerConfig)

Creates a FlinkKafkaProducer for a given topic. The sink produces a DataStream to the topic.

Using this constructor, the default FlinkFixedPartitioner will be used as the partitioner. This default partitioner maps each sink subtask to a single Kafka partition (i.e. all records received by a sink subtask will end up in the same Kafka partition).

To use a custom partitioner, please use FlinkKafkaProducer010(String, KeyedSerializationSchema, Properties, FlinkKafkaPartitioner) instead.

参数:

topicId - ID of the Kafka topic.

serializationSchema - User defined serialization schema supporting key/value messages

producerConfig - Properties with the producer configuration.

FlinkKafkaProducer010

public FlinkKafkaProducer010(String topicId,
                             org.apache.flink.streaming.util.serialization.KeyedSerializationSchema<T> serializationSchema,
                             Properties producerConfig,
                             @Nullable
                             org.apache.flink.streaming.connectors.kafka.partitioner.FlinkKafkaPartitioner<T> customPartitioner)

Creates a FlinkKafkaProducer for a given topic. The sink produces its input to the topic. It accepts a keyed KeyedSerializationSchema and possibly a custom FlinkKafkaPartitioner.

If a partitioner is not provided, written records will be partitioned by the attached key of each record (as determined by KeyedSerializationSchema.serializeKey(Object)). If written records do not have a key (i.e., KeyedSerializationSchema.serializeKey(Object) returns null), they will be distributed to Kafka partitions in a round-robin fashion.

参数:

topicId - The topic to write data to

serializationSchema - A serializable serialization schema for turning user objects into a kafka-consumable byte[] supporting key/value messages

producerConfig - Configuration properties for the KafkaProducer. 'bootstrap.servers.' is the only required argument.

customPartitioner - A serializable partitioner for assigning messages to Kafka partitions. If set to null, records will be partitioned by the key of each record (determined by KeyedSerializationSchema.serializeKey(Object)). If the keys are null, then records will be distributed to Kafka partitions in a round-robin fashion.

方法详细资料

setWriteTimestampToKafka

public void setWriteTimestampToKafka(boolean writeTimestampToKafka)

If set to true, Flink will write the (event time) timestamp attached to each record into Kafka. Timestamps must be positive for Kafka to accept them.

参数:

writeTimestampToKafka - Flag indicating if Flink's internal timestamps are written to Kafka.

writeToKafkaWithTimestamps

@Deprecated
public static <T> FlinkKafkaProducer010.FlinkKafkaProducer010Configuration<T> writeToKafkaWithTimestamps(org.apache.flink.streaming.api.datastream.DataStream<T> inStream,
                                                                                                                     String topicId,
                                                                                                                     org.apache.flink.streaming.util.serialization.KeyedSerializationSchema<T> serializationSchema,
                                                                                                                     Properties producerConfig)

已过时。 Use FlinkKafkaProducer010(String, KeyedSerializationSchema, Properties) and call setWriteTimestampToKafka(boolean).

Creates a FlinkKafkaProducer for a given topic. The sink produces a DataStream to the topic.

This constructor allows writing timestamps to Kafka, it follow approach (b) (see above)

参数:

inStream - The stream to write to Kafka

topicId - ID of the Kafka topic.

serializationSchema - User defined serialization schema supporting key/value messages

producerConfig - Properties with the producer configuration.

writeToKafkaWithTimestamps

@Deprecated
public static <T> FlinkKafkaProducer010.FlinkKafkaProducer010Configuration<T> writeToKafkaWithTimestamps(org.apache.flink.streaming.api.datastream.DataStream<T> inStream,
                                                                                                                     String topicId,
                                                                                                                     org.apache.flink.api.common.serialization.SerializationSchema<T> serializationSchema,
                                                                                                                     Properties producerConfig)

已过时。 Use FlinkKafkaProducer010(String, KeyedSerializationSchema, Properties) and call setWriteTimestampToKafka(boolean).

Creates a FlinkKafkaProducer for a given topic. the sink produces a DataStream to the topic.

This constructor allows writing timestamps to Kafka, it follow approach (b) (see above)

参数:

inStream - The stream to write to Kafka

topicId - ID of the Kafka topic.

serializationSchema - User defined (keyless) serialization schema.

producerConfig - Properties with the producer configuration.

writeToKafkaWithTimestamps

@Deprecated
public static <T> FlinkKafkaProducer010.FlinkKafkaProducer010Configuration<T> writeToKafkaWithTimestamps(org.apache.flink.streaming.api.datastream.DataStream<T> inStream,
                                                                                                                     String topicId,
                                                                                                                     org.apache.flink.streaming.util.serialization.KeyedSerializationSchema<T> serializationSchema,
                                                                                                                     Properties producerConfig,
                                                                                                                     org.apache.flink.streaming.connectors.kafka.partitioner.FlinkKafkaPartitioner<T> customPartitioner)

已过时。 Use FlinkKafkaProducer010(String, KeyedSerializationSchema, Properties, FlinkKafkaPartitioner) and call setWriteTimestampToKafka(boolean).

Creates a FlinkKafkaProducer for a given topic. The sink produces a DataStream to the topic.

This constructor allows writing timestamps to Kafka, it follow approach (b) (see above)

参数:

inStream - The stream to write to Kafka

topicId - The name of the target topic

serializationSchema - A serializable serialization schema for turning user objects into a kafka-consumable byte[] supporting key/value messages

producerConfig - Configuration properties for the KafkaProducer. 'bootstrap.servers.' is the only required argument.

customPartitioner - A serializable partitioner for assigning messages to Kafka partitions.

invoke

public void invoke(T value,
                   org.apache.flink.streaming.api.functions.sink.SinkFunction.Context context)
            throws Exception

指定者:

invoke 在接口中 org.apache.flink.streaming.api.functions.sink.SinkFunction<T>

覆盖:

invoke 在类中 org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerBase<T>

抛出:

Exception

flush

protected void flush()

指定者:

flush 在类中 org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerBase<T>