org.apache.flink.table.filesystem.stream.compact

Class CompactFileWriter<T>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

org.apache.flink.table.filesystem.stream.AbstractStreamingWriter<T,CompactMessages.CoordinatorInput>

org.apache.flink.table.filesystem.stream.compact.CompactFileWriter<T>

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.operators.BoundedOneInput, org.apache.flink.streaming.api.operators.Input<T>, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.OneInputStreamOperator<T,CompactMessages.CoordinatorInput>, org.apache.flink.streaming.api.operators.SetupableStreamOperator<CompactMessages.CoordinatorInput>, org.apache.flink.streaming.api.operators.StreamOperator<CompactMessages.CoordinatorInput>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.util.Disposable

public class CompactFileWriter<T>
extends AbstractStreamingWriter<T,CompactMessages.CoordinatorInput>

Writer for emitting CompactMessages.InputFile and CompactMessages.EndCheckpoint to downstream.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

chainingStrategy, config, latencyStats, LOG, metrics, output, processingTimeService

Constructor Summary

Constructors

Constructor and Description
CompactFileWriter(long bucketCheckInterval, org.apache.flink.streaming.api.functions.sink.filesystem.StreamingFileSink.BucketsBuilder<T,String,? extends org.apache.flink.streaming.api.functions.sink.filesystem.StreamingFileSink.BucketsBuilder<T,String,?>> bucketsBuilder)

Method Summary

Modifier and Type	Method and Description
void	notifyCheckpointComplete(long checkpointId)
protected void	onPartFileOpened(String partition, org.apache.flink.core.fs.Path newPath) Notifies a new file has been opened.
protected void	partitionInactive(String partition) Notifies a partition become inactive.

Methods inherited from class org.apache.flink.table.filesystem.stream.AbstractStreamingWriter

commitUpToCheckpoint, dispose, endInput, initializeState, processElement, processWatermark, snapshotState

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

close, getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, open, prepareSnapshotPreBarrier, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.streaming.api.operators.OneInputStreamOperator

setKeyContextElement

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

close, getMetricGroup, getOperatorID, initializeState, open, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

Methods inherited from interface org.apache.flink.streaming.api.operators.Input

processLatencyMarker

Constructor Detail

CompactFileWriter

public CompactFileWriter(long bucketCheckInterval,
                         org.apache.flink.streaming.api.functions.sink.filesystem.StreamingFileSink.BucketsBuilder<T,String,? extends org.apache.flink.streaming.api.functions.sink.filesystem.StreamingFileSink.BucketsBuilder<T,String,?>> bucketsBuilder)

Method Detail

partitionInactive

protected void partitionInactive(String partition)

Description copied from class: AbstractStreamingWriter

Notifies a partition become inactive. A partition becomes inactive after all the records received so far have been committed.

Specified by:

partitionInactive in class AbstractStreamingWriter<T,CompactMessages.CoordinatorInput>

onPartFileOpened

protected void onPartFileOpened(String partition,
                                org.apache.flink.core.fs.Path newPath)

Description copied from class: AbstractStreamingWriter

Notifies a new file has been opened.

Note that this does not mean that the file has been created in the file system. It is only created logically and the actual file will be generated after it is committed.

Specified by:

onPartFileOpened in class AbstractStreamingWriter<T,CompactMessages.CoordinatorInput>

notifyCheckpointComplete

public void notifyCheckpointComplete(long checkpointId)
                              throws Exception

Specified by:

notifyCheckpointComplete in interface org.apache.flink.api.common.state.CheckpointListener

Overrides:

notifyCheckpointComplete in class AbstractStreamingWriter<T,CompactMessages.CoordinatorInput>

Throws:

Exception