A B C D F G H I K M N O P R S T V W

A

AvroKeyValue(GenericRecord) - Constructor for class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter.AvroKeyValue

Wraps a GenericRecord that is a key value pair.

AvroKeyValueSinkWriter<K,V> - Class in org.apache.flink.streaming.connectors.fs

Implementation of AvroKeyValue writer that can be used in Sink.

AvroKeyValueSinkWriter(Map<String, String>) - Constructor for class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter

C'tor for the writer

AvroKeyValueSinkWriter.AvroKeyValue<K,V> - Class in org.apache.flink.streaming.connectors.fs

B

BasePathBucketer<T> - Class in org.apache.flink.streaming.connectors.fs.bucketing

A Bucketer that does not perform any bucketing of files.

BasePathBucketer() - Constructor for class org.apache.flink.streaming.connectors.fs.bucketing.BasePathBucketer

Bucketer - Interface in org.apache.flink.streaming.connectors.fs

Deprecated.

use Bucketer instead.

Bucketer<T> - Interface in org.apache.flink.streaming.connectors.fs.bucketing

A bucketer is used with a BucketingSink to put emitted elements into rolling files.

BucketingSink<T> - Class in org.apache.flink.streaming.connectors.fs.bucketing

Sink that emits its input elements to FileSystem files within buckets.

BucketingSink(String) - Constructor for class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

Creates a new BucketingSink that writes files to the given base directory.

BucketState() - Constructor for class org.apache.flink.streaming.connectors.fs.RollingSink.BucketState

Deprecated.

C

Clock - Interface in org.apache.flink.streaming.connectors.fs

A clock that can provide the current time.

close() - Method in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter

close() - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

close() - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

close() - Method in class org.apache.flink.streaming.connectors.fs.SequenceFileWriter

close() - Method in class org.apache.flink.streaming.connectors.fs.StreamWriterBase

close() - Method in interface org.apache.flink.streaming.connectors.fs.Writer

Closes the Writer.

CONF_COMPRESS - Static variable in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter

CONF_COMPRESS_CODEC - Static variable in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter

CONF_DEFLATE_LEVEL - Static variable in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter

CONF_OUTPUT_KEY_SCHEMA - Static variable in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter

CONF_OUTPUT_VALUE_SCHEMA - Static variable in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter

CONF_XZ_LEVEL - Static variable in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter

currentFile - Variable in class org.apache.flink.streaming.connectors.fs.RollingSink.BucketState

Deprecated.

The file that was in-progress when the last checkpoint occurred.

currentFileValidLength - Variable in class org.apache.flink.streaming.connectors.fs.RollingSink.BucketState

Deprecated.

The valid length of the in-progress file at the time of the last checkpoint.

currentTimeMillis() - Method in interface org.apache.flink.streaming.connectors.fs.Clock

Return the current system time in milliseconds.

currentTimeMillis() - Method in class org.apache.flink.streaming.connectors.fs.SystemClock

D

DateTimeBucketer<T> - Class in org.apache.flink.streaming.connectors.fs.bucketing

A Bucketer that assigns to buckets based on current system time.

DateTimeBucketer() - Constructor for class org.apache.flink.streaming.connectors.fs.bucketing.DateTimeBucketer

Creates a new DateTimeBucketer with format string "yyyy-MM-dd--HH".

DateTimeBucketer(String) - Constructor for class org.apache.flink.streaming.connectors.fs.bucketing.DateTimeBucketer

Creates a new DateTimeBucketer with the given date/time format string.

DateTimeBucketer - Class in org.apache.flink.streaming.connectors.fs

Deprecated.

use DateTimeBucketer instead.

DateTimeBucketer() - Constructor for class org.apache.flink.streaming.connectors.fs.DateTimeBucketer

Deprecated.

Creates a new DateTimeBucketer with format string "yyyy-MM-dd--HH".

DateTimeBucketer(String) - Constructor for class org.apache.flink.streaming.connectors.fs.DateTimeBucketer

Deprecated.

Creates a new DateTimeBucketer with the given date/time format string.

disableCleanupOnOpen() - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

Deprecated.

This option is deprecated and remains only for backwards compatibility. We do not clean up lingering files anymore.

disableCleanupOnOpen() - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

This option is deprecated and remains only for backwards compatibility. We do not clean up lingering files anymore.

duplicate() - Method in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter

duplicate() - Method in class org.apache.flink.streaming.connectors.fs.SequenceFileWriter

duplicate() - Method in class org.apache.flink.streaming.connectors.fs.StringWriter

duplicate() - Method in interface org.apache.flink.streaming.connectors.fs.Writer

Duplicates the Writer.

F

flush() - Method in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter

flush() - Method in class org.apache.flink.streaming.connectors.fs.StreamWriterBase

flush() - Method in interface org.apache.flink.streaming.connectors.fs.Writer

Flushes out any internally held data, and returns the offset that the file must be truncated to at recovery.

G

get() - Method in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter.AvroKeyValue

getBucketPath(Clock, Path, T) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BasePathBucketer

getBucketPath(Clock, Path, T) - Method in interface org.apache.flink.streaming.connectors.fs.bucketing.Bucketer

Returns the Path of a bucket file.

getBucketPath(Clock, Path, T) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.DateTimeBucketer

getKey() - Method in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter.AvroKeyValue

getNextBucketPath(Path) - Method in interface org.apache.flink.streaming.connectors.fs.Bucketer

Deprecated.

Returns the Path of a new bucket file.

getNextBucketPath(Path) - Method in class org.apache.flink.streaming.connectors.fs.DateTimeBucketer

Deprecated.

getNextBucketPath(Path) - Method in class org.apache.flink.streaming.connectors.fs.NonRollingBucketer

Deprecated.

getPos() - Method in class org.apache.flink.streaming.connectors.fs.StreamWriterBase

getPos() - Method in interface org.apache.flink.streaming.connectors.fs.Writer

Retrieves the current position, and thus size, of the output file.

getSchema(Schema, Schema) - Static method in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter.AvroKeyValue

Creates a KeyValuePair generic record schema.

getState() - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

getStream() - Method in class org.apache.flink.streaming.connectors.fs.StreamWriterBase

Returns the current output stream, if the stream is open.

getValue() - Method in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter.AvroKeyValue

H

hflushOrSync(FSDataOutputStream) - Method in class org.apache.flink.streaming.connectors.fs.StreamWriterBase

If hflush is available in this version of HDFS, then this method calls hflush, else it calls sync.

I

initializeState(FunctionInitializationContext) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

initializeState(FunctionInitializationContext) - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

invoke(T) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

invoke(T) - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

K

KEY_FIELD - Static variable in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter.AvroKeyValue

The name of the generic record field containing the key.

KEY_VALUE_PAIR_RECORD_NAME - Static variable in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter.AvroKeyValue

The name of the key value pair generic record.

KEY_VALUE_PAIR_RECORD_NAMESPACE - Static variable in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter.AvroKeyValue

The namespace of the key value pair generic record.

M

mKeyValueRecord - Variable in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter.AvroKeyValue

The key/value generic record wrapped by this class.

N

NonRollingBucketer - Class in org.apache.flink.streaming.connectors.fs

Deprecated.

use BasePathBucketer instead.

NonRollingBucketer() - Constructor for class org.apache.flink.streaming.connectors.fs.NonRollingBucketer

Deprecated.

notifyCheckpointComplete(long) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

notifyCheckpointComplete(long) - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

O

onProcessingTime(long) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

open(FileSystem, Path) - Method in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter

open(Configuration) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

open(Configuration) - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

open(FileSystem, Path) - Method in class org.apache.flink.streaming.connectors.fs.SequenceFileWriter

open(FileSystem, Path) - Method in class org.apache.flink.streaming.connectors.fs.StreamWriterBase

open(FileSystem, Path) - Method in class org.apache.flink.streaming.connectors.fs.StringWriter

open(FileSystem, Path) - Method in interface org.apache.flink.streaming.connectors.fs.Writer

Initializes the Writer for a newly opened bucket file.

org.apache.flink.streaming.connectors.fs - package org.apache.flink.streaming.connectors.fs

org.apache.flink.streaming.connectors.fs.bucketing - package org.apache.flink.streaming.connectors.fs.bucketing

P

pendingFiles - Variable in class org.apache.flink.streaming.connectors.fs.RollingSink.BucketState

Deprecated.

Pending files that accumulated since the last checkpoint.

pendingFilesPerCheckpoint - Variable in class org.apache.flink.streaming.connectors.fs.RollingSink.BucketState

Deprecated.

When doing a checkpoint we move the pending files since the last checkpoint to this map with the id of the checkpoint.

R

restoreState(RollingSink.BucketState) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

restoreState(RollingSink.BucketState) - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

RollingSink<T> - Class in org.apache.flink.streaming.connectors.fs

Deprecated.

use BucketingSink instead.

RollingSink(String) - Constructor for class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

Creates a new RollingSink that writes files to the given base directory.

RollingSink.BucketState - Class in org.apache.flink.streaming.connectors.fs

Deprecated.

This is used for keeping track of the current in-progress files and files that we mark for moving from pending to final location after we get a checkpoint-complete notification.

S

SequenceFileWriter<K extends org.apache.hadoop.io.Writable,V extends org.apache.hadoop.io.Writable> - Class in org.apache.flink.streaming.connectors.fs

A Writer that writes the bucket files as Hadoop SequenceFiles.

SequenceFileWriter() - Constructor for class org.apache.flink.streaming.connectors.fs.SequenceFileWriter

Creates a new SequenceFileWriter that writes sequence files without compression.

SequenceFileWriter(String, SequenceFile.CompressionType) - Constructor for class org.apache.flink.streaming.connectors.fs.SequenceFileWriter

Creates a new SequenceFileWriter that writes sequence with the given compression codec and compression type.

setAsyncTimeout(long) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

Sets the default timeout for asynchronous operations such as recoverLease and truncate.

setAsyncTimeout(long) - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

Sets the default timeout for asynchronous operations such as recoverLease and truncate.

setBatchSize(long) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

Sets the maximum bucket size in bytes.

setBatchSize(long) - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

Sets the maximum bucket size in bytes.

setBucketer(Bucketer<T>) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

Sets the Bucketer to use for determining the bucket files to write to.

setBucketer(Bucketer) - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

Sets the Bucketer to use for determining the bucket files to write to.

setClock(Clock) - Static method in class org.apache.flink.streaming.connectors.fs.DateTimeBucketer

Deprecated.

This sets the internal Clock implementation.

setFSConfig(Configuration) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

Specify a custom Configuration that will be used when creating the FileSystem for writing.

setFSConfig(Configuration) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

Specify a custom Configuration that will be used when creating the FileSystem for writing.

setFSConfig(Configuration) - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

Specify a custom Configuration that will be used when creating the FileSystem for writing.

setFSConfig(Configuration) - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

Specify a custom Configuration that will be used when creating the FileSystem for writing.

setInactiveBucketCheckInterval(long) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

Sets the default time between checks for inactive buckets.

setInactiveBucketThreshold(long) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

Sets the default threshold for marking a bucket as inactive and closing its part files.

setInProgressPrefix(String) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

Sets the prefix of in-progress part files.

setInProgressPrefix(String) - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

Sets the prefix of in-progress part files.

setInProgressSuffix(String) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

Sets the suffix of in-progress part files.

setInProgressSuffix(String) - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

Sets the suffix of in-progress part files.

setInputType(TypeInformation<?>, ExecutionConfig) - Method in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter

setInputType(TypeInformation<?>, ExecutionConfig) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

setInputType(TypeInformation<?>, ExecutionConfig) - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

setInputType(TypeInformation<?>, ExecutionConfig) - Method in class org.apache.flink.streaming.connectors.fs.SequenceFileWriter

setKey(K) - Method in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter.AvroKeyValue

setPartPrefix(String) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

Sets the prefix of part files.

setPartPrefix(String) - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

Sets the prefix of part files.

setPendingPrefix(String) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

Sets the prefix of pending part files.

setPendingPrefix(String) - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

Sets the prefix of pending part files.

setPendingSuffix(String) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

Sets the suffix of pending part files.

setPendingSuffix(String) - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

Sets the suffix of pending part files.

setValidLengthPrefix(String) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

Sets the prefix of valid-length files.

setValidLengthPrefix(String) - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

Sets the prefix of valid-length files.

setValidLengthSuffix(String) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

Sets the suffix of valid-length files.

setValidLengthSuffix(String) - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

Sets the suffix of valid-length files.

setValue(V) - Method in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter.AvroKeyValue

setWriter(Writer<T>) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

Sets the Writer to be used for writing the incoming elements to bucket files.

setWriter(Writer<T>) - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

Sets the Writer to be used for writing the incoming elements to bucket files.

shouldStartNewBucket(Path, Path) - Method in interface org.apache.flink.streaming.connectors.fs.Bucketer

Deprecated.

Returns true when a new bucket should be started.

shouldStartNewBucket(Path, Path) - Method in class org.apache.flink.streaming.connectors.fs.DateTimeBucketer

Deprecated.

shouldStartNewBucket(Path, Path) - Method in class org.apache.flink.streaming.connectors.fs.NonRollingBucketer

Deprecated.

snapshotState(FunctionSnapshotContext) - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink

snapshotState(FunctionSnapshotContext) - Method in class org.apache.flink.streaming.connectors.fs.RollingSink

Deprecated.

StreamWriterBase<T> - Class in org.apache.flink.streaming.connectors.fs

Base class for Writers that write to a FSDataOutputStream.

StreamWriterBase() - Constructor for class org.apache.flink.streaming.connectors.fs.StreamWriterBase

StringWriter<T> - Class in org.apache.flink.streaming.connectors.fs

A Writer that uses toString() on the input elements and writes them to the output bucket file separated by newline.

StringWriter() - Constructor for class org.apache.flink.streaming.connectors.fs.StringWriter

Creates a new StringWriter that uses "UTF-8" charset to convert strings to bytes.

StringWriter(String) - Constructor for class org.apache.flink.streaming.connectors.fs.StringWriter

Creates a new StringWriter that uses the given charset to convert strings to bytes.

SystemClock - Class in org.apache.flink.streaming.connectors.fs

A Clock that uses System.currentTimeMillis() to determine the system time.

SystemClock() - Constructor for class org.apache.flink.streaming.connectors.fs.SystemClock

T

toString() - Method in class org.apache.flink.streaming.connectors.fs.bucketing.BasePathBucketer

toString() - Method in class org.apache.flink.streaming.connectors.fs.bucketing.DateTimeBucketer

toString() - Method in class org.apache.flink.streaming.connectors.fs.DateTimeBucketer

Deprecated.

toString() - Method in class org.apache.flink.streaming.connectors.fs.NonRollingBucketer

Deprecated.

toString() - Method in class org.apache.flink.streaming.connectors.fs.RollingSink.BucketState

Deprecated.

V

VALUE_FIELD - Static variable in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter.AvroKeyValue

The name of the generic record field containing the value.

W

write(Tuple2<K, V>) - Method in class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter

write(Tuple2<K, V>) - Method in class org.apache.flink.streaming.connectors.fs.SequenceFileWriter

write(T) - Method in class org.apache.flink.streaming.connectors.fs.StringWriter

write(T) - Method in interface org.apache.flink.streaming.connectors.fs.Writer

Writes one element to the bucket file.

Writer<T> - Interface in org.apache.flink.streaming.connectors.fs

An implementation of Writer is used in conjunction with a BucketingSink to perform the actual writing to the bucket files.

A B C D F G H I K M N O P R S T V W