org.apache.flink.streaming.connectors.elasticsearch6

Class ElasticsearchSink<T>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.streaming.api.functions.sink.RichSinkFunction<T>

org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase<T,org.elasticsearch.client.RestHighLevelClient>

org.apache.flink.streaming.connectors.elasticsearch6.ElasticsearchSink<T>

Type Parameters:

T - Type of the elements handled by this sink

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction, org.apache.flink.streaming.api.checkpoint.CheckpointedFunction, org.apache.flink.streaming.api.functions.sink.SinkFunction<T>

@PublicEvolving
public class ElasticsearchSink<T>
extends org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase<T,org.elasticsearch.client.RestHighLevelClient>

Elasticsearch 6.x sink that requests multiple ActionRequests against a cluster for each incoming element.

The sink internally uses a RestHighLevelClient to communicate with an Elasticsearch cluster. The sink will fail if no cluster can be connected to using the provided transport addresses passed to the constructor.

Internally, the sink will use a BulkProcessor to send ActionRequests. This will buffer elements before sending a request to the cluster. The behaviour of the BulkProcessor can be configured using these config keys:

• bulk.flush.max.actions: Maximum amount of elements to buffer
• bulk.flush.max.size.mb: Maximum amount of data (in megabytes) to buffer
• bulk.flush.interval.ms: Interval at which to flush data regardless of the other two settings in milliseconds

You also have to provide an ElasticsearchSinkFunction. This is used to create multiple ActionRequests for each incoming element. See the class level documentation of ElasticsearchSinkFunction for an example.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ElasticsearchSink.Builder<T> A builder for creating an ElasticsearchSink.

Nested classes/interfaces inherited from class org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase

org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.BulkFlushBackoffPolicy, org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.FlushBackoffType

Nested classes/interfaces inherited from interface org.apache.flink.streaming.api.functions.sink.SinkFunction

org.apache.flink.streaming.api.functions.sink.SinkFunction.Context<T>

Field Summary

Fields inherited from class org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase

CONFIG_KEY_BULK_FLUSH_BACKOFF_DELAY, CONFIG_KEY_BULK_FLUSH_BACKOFF_ENABLE, CONFIG_KEY_BULK_FLUSH_BACKOFF_RETRIES, CONFIG_KEY_BULK_FLUSH_BACKOFF_TYPE, CONFIG_KEY_BULK_FLUSH_INTERVAL_MS, CONFIG_KEY_BULK_FLUSH_MAX_ACTIONS, CONFIG_KEY_BULK_FLUSH_MAX_SIZE_MB

Method Summary

Methods inherited from class org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase

buildBulkProcessor, close, disableFlushOnCheckpoint, initializeState, invoke, open, snapshotState

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.streaming.api.functions.sink.SinkFunction

invoke