T - Type of the elements handled by this sinkC - Type of the Elasticsearch client, which implements AutoCloseable@Internal
public abstract class ElasticsearchSinkBase<T,C extends AutoCloseable>
extends org.apache.flink.streaming.api.functions.sink.RichSinkFunction<T>
implements org.apache.flink.streaming.api.checkpoint.CheckpointedFunction
This class implements the common behaviour across Elasticsearch versions, such as
the use of an internal BulkProcessor to buffer multiple ActionRequests before
sending the requests to the cluster, as well as passing input records to the user provided
ElasticsearchSinkFunction for processing.
The version specific API calls for different Elasticsearch versions should be defined by a concrete implementation of
a ElasticsearchApiCallBridge, which is provided to the constructor of this class. This call bridge is used,
for example, to create a Elasticsearch Client, handle failed item responses, etc.
| 限定符和类型 | 类和说明 |
|---|---|
static class |
ElasticsearchSinkBase.BulkFlushBackoffPolicy
Provides a backoff policy for bulk requests.
|
static class |
ElasticsearchSinkBase.FlushBackoffType
Used to control whether the retry delay should increase exponentially or remain constant.
|
| 限定符和类型 | 字段和说明 |
|---|---|
static String |
CONFIG_KEY_BULK_FLUSH_BACKOFF_DELAY |
static String |
CONFIG_KEY_BULK_FLUSH_BACKOFF_ENABLE |
static String |
CONFIG_KEY_BULK_FLUSH_BACKOFF_RETRIES |
static String |
CONFIG_KEY_BULK_FLUSH_BACKOFF_TYPE |
static String |
CONFIG_KEY_BULK_FLUSH_INTERVAL_MS |
static String |
CONFIG_KEY_BULK_FLUSH_MAX_ACTIONS |
static String |
CONFIG_KEY_BULK_FLUSH_MAX_SIZE_MB |
| 构造器和说明 |
|---|
ElasticsearchSinkBase(ElasticsearchApiCallBridge<C> callBridge,
Map<String,String> userConfig,
ElasticsearchSinkFunction<T> elasticsearchSinkFunction,
ActionRequestFailureHandler failureHandler) |
| 限定符和类型 | 方法和说明 |
|---|---|
protected org.elasticsearch.action.bulk.BulkProcessor |
buildBulkProcessor(org.elasticsearch.action.bulk.BulkProcessor.Listener listener)
Build the
BulkProcessor. |
void |
close() |
void |
disableFlushOnCheckpoint()
Disable flushing on checkpoint.
|
void |
initializeState(org.apache.flink.runtime.state.FunctionInitializationContext context) |
void |
invoke(T value,
org.apache.flink.streaming.api.functions.sink.SinkFunction.Context context) |
void |
open(org.apache.flink.configuration.Configuration parameters) |
void |
snapshotState(org.apache.flink.runtime.state.FunctionSnapshotContext context) |
getIterationRuntimeContext, getRuntimeContext, setRuntimeContextpublic static final String CONFIG_KEY_BULK_FLUSH_MAX_ACTIONS
public static final String CONFIG_KEY_BULK_FLUSH_MAX_SIZE_MB
public static final String CONFIG_KEY_BULK_FLUSH_INTERVAL_MS
public static final String CONFIG_KEY_BULK_FLUSH_BACKOFF_ENABLE
public static final String CONFIG_KEY_BULK_FLUSH_BACKOFF_TYPE
public static final String CONFIG_KEY_BULK_FLUSH_BACKOFF_RETRIES
public ElasticsearchSinkBase(ElasticsearchApiCallBridge<C> callBridge, Map<String,String> userConfig, ElasticsearchSinkFunction<T> elasticsearchSinkFunction, ActionRequestFailureHandler failureHandler)
public void disableFlushOnCheckpoint()
NOTE: If flushing on checkpoint is disabled, the Flink Elasticsearch Sink does NOT provide any strong guarantees for at-least-once delivery of action requests.
public void open(org.apache.flink.configuration.Configuration parameters)
throws Exception
open 在接口中 org.apache.flink.api.common.functions.RichFunctionopen 在类中 org.apache.flink.api.common.functions.AbstractRichFunctionExceptionpublic void invoke(T value, org.apache.flink.streaming.api.functions.sink.SinkFunction.Context context) throws Exception
public void initializeState(org.apache.flink.runtime.state.FunctionInitializationContext context)
throws Exception
initializeState 在接口中 org.apache.flink.streaming.api.checkpoint.CheckpointedFunctionExceptionpublic void snapshotState(org.apache.flink.runtime.state.FunctionSnapshotContext context)
throws Exception
snapshotState 在接口中 org.apache.flink.streaming.api.checkpoint.CheckpointedFunctionExceptionpublic void close()
throws Exception
close 在接口中 org.apache.flink.api.common.functions.RichFunctionclose 在类中 org.apache.flink.api.common.functions.AbstractRichFunctionException@VisibleForTesting protected org.elasticsearch.action.bulk.BulkProcessor buildBulkProcessor(org.elasticsearch.action.bulk.BulkProcessor.Listener listener)
BulkProcessor.
Note: this is exposed for testing purposes.
Copyright © 2014–2019 The Apache Software Foundation. All rights reserved.