org.apache.flink.streaming.connectors.elasticsearch

类 ElasticsearchSinkBase<T,C extends AutoCloseable>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.streaming.api.functions.sink.RichSinkFunction<T>

org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase<T,C>

类型参数:

T - Type of the elements handled by this sink

C - Type of the Elasticsearch client, which implements AutoCloseable

所有已实现的接口:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction, org.apache.flink.streaming.api.checkpoint.CheckpointedFunction, org.apache.flink.streaming.api.functions.sink.SinkFunction<T>

@Internal
public abstract class ElasticsearchSinkBase<T,C extends AutoCloseable>
extends org.apache.flink.streaming.api.functions.sink.RichSinkFunction<T>
implements org.apache.flink.streaming.api.checkpoint.CheckpointedFunction

Base class for all Flink Elasticsearch Sinks.

This class implements the common behaviour across Elasticsearch versions, such as the use of an internal BulkProcessor to buffer multiple ActionRequests before sending the requests to the cluster, as well as passing input records to the user provided ElasticsearchSinkFunction for processing.

The version specific API calls for different Elasticsearch versions should be defined by a concrete implementation of a ElasticsearchApiCallBridge, which is provided to the constructor of this class. This call bridge is used, for example, to create a Elasticsearch Client, handle failed item responses, etc.

另请参阅:

序列化表格

嵌套类概要

嵌套类

限定符和类型	类和说明
static class	ElasticsearchSinkBase.BulkFlushBackoffPolicy Provides a backoff policy for bulk requests.
static class	ElasticsearchSinkBase.FlushBackoffType Used to control whether the retry delay should increase exponentially or remain constant.

从接口继承的嵌套类/接口 org.apache.flink.streaming.api.functions.sink.SinkFunction

org.apache.flink.streaming.api.functions.sink.SinkFunction.Context<T>

字段概要

字段

限定符和类型	字段和说明
static String	CONFIG_KEY_BULK_FLUSH_BACKOFF_DELAY
static String	CONFIG_KEY_BULK_FLUSH_BACKOFF_ENABLE
static String	CONFIG_KEY_BULK_FLUSH_BACKOFF_RETRIES
static String	CONFIG_KEY_BULK_FLUSH_BACKOFF_TYPE
static String	CONFIG_KEY_BULK_FLUSH_INTERVAL_MS
static String	CONFIG_KEY_BULK_FLUSH_MAX_ACTIONS
static String	CONFIG_KEY_BULK_FLUSH_MAX_SIZE_MB

构造器概要

构造器

构造器和说明
ElasticsearchSinkBase(ElasticsearchApiCallBridge<C> callBridge, Map<String,String> userConfig, ElasticsearchSinkFunction<T> elasticsearchSinkFunction, ActionRequestFailureHandler failureHandler)

方法概要

限定符和类型	方法和说明
protected org.elasticsearch.action.bulk.BulkProcessor	buildBulkProcessor(org.elasticsearch.action.bulk.BulkProcessor.Listener listener) Build the BulkProcessor.
void	close()
void	disableFlushOnCheckpoint() Disable flushing on checkpoint.
void	initializeState(org.apache.flink.runtime.state.FunctionInitializationContext context)
void	invoke(T value, org.apache.flink.streaming.api.functions.sink.SinkFunction.Context context)
void	open(org.apache.flink.configuration.Configuration parameters)
void	snapshotState(org.apache.flink.runtime.state.FunctionSnapshotContext context)

从类继承的方法 org.apache.flink.api.common.functions.AbstractRichFunction

getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.flink.streaming.api.functions.sink.SinkFunction

invoke

字段详细资料

CONFIG_KEY_BULK_FLUSH_MAX_ACTIONS

public static final String CONFIG_KEY_BULK_FLUSH_MAX_ACTIONS

另请参阅:

常量字段值

CONFIG_KEY_BULK_FLUSH_MAX_SIZE_MB

public static final String CONFIG_KEY_BULK_FLUSH_MAX_SIZE_MB

另请参阅:

常量字段值

CONFIG_KEY_BULK_FLUSH_INTERVAL_MS

public static final String CONFIG_KEY_BULK_FLUSH_INTERVAL_MS

另请参阅:

常量字段值

CONFIG_KEY_BULK_FLUSH_BACKOFF_ENABLE

public static final String CONFIG_KEY_BULK_FLUSH_BACKOFF_ENABLE

另请参阅:

常量字段值

CONFIG_KEY_BULK_FLUSH_BACKOFF_TYPE

public static final String CONFIG_KEY_BULK_FLUSH_BACKOFF_TYPE

另请参阅:

常量字段值

CONFIG_KEY_BULK_FLUSH_BACKOFF_RETRIES

public static final String CONFIG_KEY_BULK_FLUSH_BACKOFF_RETRIES

另请参阅:

常量字段值

CONFIG_KEY_BULK_FLUSH_BACKOFF_DELAY

public static final String CONFIG_KEY_BULK_FLUSH_BACKOFF_DELAY

另请参阅:

常量字段值

构造器详细资料

ElasticsearchSinkBase

public ElasticsearchSinkBase(ElasticsearchApiCallBridge<C> callBridge,
                             Map<String,String> userConfig,
                             ElasticsearchSinkFunction<T> elasticsearchSinkFunction,
                             ActionRequestFailureHandler failureHandler)

方法详细资料

disableFlushOnCheckpoint

public void disableFlushOnCheckpoint()

Disable flushing on checkpoint. When disabled, the sink will not wait for all pending action requests to be acknowledged by Elasticsearch on checkpoints.

NOTE: If flushing on checkpoint is disabled, the Flink Elasticsearch Sink does NOT provide any strong guarantees for at-least-once delivery of action requests.

open

public void open(org.apache.flink.configuration.Configuration parameters)
          throws Exception

指定者:

open 在接口中 org.apache.flink.api.common.functions.RichFunction

覆盖:

open 在类中 org.apache.flink.api.common.functions.AbstractRichFunction

抛出:

Exception

invoke

public void invoke(T value,
                   org.apache.flink.streaming.api.functions.sink.SinkFunction.Context context)
            throws Exception

指定者:

invoke 在接口中 org.apache.flink.streaming.api.functions.sink.SinkFunction<T>

抛出:

Exception

initializeState

public void initializeState(org.apache.flink.runtime.state.FunctionInitializationContext context)
                     throws Exception

指定者:

initializeState 在接口中 org.apache.flink.streaming.api.checkpoint.CheckpointedFunction

抛出:

Exception

snapshotState

public void snapshotState(org.apache.flink.runtime.state.FunctionSnapshotContext context)
                   throws Exception

指定者:

snapshotState 在接口中 org.apache.flink.streaming.api.checkpoint.CheckpointedFunction

抛出:

Exception

close

public void close()
           throws Exception

指定者:

close 在接口中 org.apache.flink.api.common.functions.RichFunction

覆盖:

close 在类中 org.apache.flink.api.common.functions.AbstractRichFunction

抛出:

Exception

buildBulkProcessor

@VisibleForTesting
protected org.elasticsearch.action.bulk.BulkProcessor buildBulkProcessor(org.elasticsearch.action.bulk.BulkProcessor.Listener listener)

Build the BulkProcessor.

Note: this is exposed for testing purposes.