Serialized Form (flink-connector-filesystem 1.3.1 API)

Package org.apache.flink.streaming.connectors.fs
- Class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter extends StreamWriterBase<org.apache.flink.api.java.tuple.Tuple2<K,V>> implements Serializable
  
  serialVersionUID:
  
  1L
  - Serialized Fields
    - properties
```
Map<K,V> properties
```
- Class org.apache.flink.streaming.connectors.fs.DateTimeBucketer extends Object implements Serializable
  
  serialVersionUID:
  
  1L
  - Serialization Methods
    - readObject
```
private void readObject(ObjectInputStream in)
                 throws IOException,
                        ClassNotFoundException
```
      Deprecated.
      
      Throws:
      
      IOException
      
      ClassNotFoundException
  - Serialized Fields
    - formatString
```
String formatString
```
      Deprecated.
- Class org.apache.flink.streaming.connectors.fs.NonRollingBucketer extends Object implements Serializable
  
  serialVersionUID:
  
  1L
- Class org.apache.flink.streaming.connectors.fs.RollingSink extends org.apache.flink.streaming.api.functions.sink.RichSinkFunction<T> implements Serializable
  
  serialVersionUID:
  
  1L
  - Serialized Fields
    - DEFAULT_BATCH_SIZE
```
long DEFAULT_BATCH_SIZE
```
      Deprecated.
      
      The default maximum size of part files (currently 384 MB).
    - DEFAULT_IN_PROGRESS_SUFFIX
```
String DEFAULT_IN_PROGRESS_SUFFIX
```
      Deprecated.
      
      This is used for part files that we are writing to but which where not yet confirmed by a checkpoint.
    - DEFAULT_IN_PROGRESS_PREFIX
```
String DEFAULT_IN_PROGRESS_PREFIX
```
      Deprecated.
      
      See above, but for prefix
    - DEFAULT_PENDING_SUFFIX
```
String DEFAULT_PENDING_SUFFIX
```
      Deprecated.
      
      This is used for part files that we are not writing to but which are not yet confirmed by checkpoint.
    - DEFAULT_PENDING_PREFIX
```
String DEFAULT_PENDING_PREFIX
```
      Deprecated.
      
      See above, but for prefix.
    - DEFAULT_VALID_SUFFIX
```
String DEFAULT_VALID_SUFFIX
```
      Deprecated.
      
      When truncate() is not supported on the used FileSystem we instead write a file along the part file with this ending that contains the length up to which the part file is valid.
    - DEFAULT_VALID_PREFIX
```
String DEFAULT_VALID_PREFIX
```
      Deprecated.
      
      See above, but for prefix.
    - DEFAULT_PART_REFIX
```
String DEFAULT_PART_REFIX
```
      Deprecated.
      
      The default prefix for part files.
    - DEFAULT_ASYNC_TIMEOUT_MS
```
long DEFAULT_ASYNC_TIMEOUT_MS
```
      Deprecated.
      
      The default timeout for asynchronous operations such as recoverLease and truncate. In milliseconds.
    - basePath
```
String basePath
```
      Deprecated.
      
      The base Path that stores all bucket directories.
    - bucketer
```
Bucketer bucketer
```
      Deprecated.
      
      The Bucketer that is used to determine the path of bucket directories.
    - writerTemplate
```
Writer<T> writerTemplate
```
      Deprecated.
      
      We have a template and call duplicate() for each parallel writer in open() to get the actual writer that is used for the part files.
    - writer
```
Writer<T> writer
```
      Deprecated.
      
      The actual writer that we user for writing the part files.
    - batchSize
```
long batchSize
```
      Deprecated.
      
      Maximum size of part files. If files exceed this we close and create a new one in the same bucket directory.
    - inProgressSuffix
```
String inProgressSuffix
```
      Deprecated.
    - inProgressPrefix
```
String inProgressPrefix
```
      Deprecated.
    - pendingSuffix
```
String pendingSuffix
```
      Deprecated.
    - pendingPrefix
```
String pendingPrefix
```
      Deprecated.
    - validLengthSuffix
```
String validLengthSuffix
```
      Deprecated.
    - validLengthPrefix
```
String validLengthPrefix
```
      Deprecated.
    - partPrefix
```
String partPrefix
```
      Deprecated.
    - asyncTimeout
```
long asyncTimeout
```
      Deprecated.
      
      The timeout for asynchronous operations such as recoverLease and truncate. In milliseconds.
    - fsConfig
```
org.apache.flink.configuration.Configuration fsConfig
```
      Deprecated.
      
      User-defined FileSystem parameters.
- Class org.apache.flink.streaming.connectors.fs.RollingSink.BucketState extends Object implements Serializable
  
  serialVersionUID:
  
  1L
  - Serialized Fields
    - currentFile
```
String currentFile
```
      The file that was in-progress when the last checkpoint occurred.
    - currentFileValidLength
```
long currentFileValidLength
```
      The valid length of the in-progress file at the time of the last checkpoint.
    - pendingFiles
```
List<E> pendingFiles
```
      Pending files that accumulated since the last checkpoint.
    - pendingFilesPerCheckpoint
```
Map<K,V> pendingFilesPerCheckpoint
```
      When doing a checkpoint we move the pending files since the last checkpoint to this map with the id of the checkpoint. When we get the checkpoint-complete notification we move pending files of completed checkpoints to their final location.
- Class org.apache.flink.streaming.connectors.fs.SequenceFileWriter extends StreamWriterBase<org.apache.flink.api.java.tuple.Tuple2<K extends org.apache.hadoop.io.Writable,V extends org.apache.hadoop.io.Writable>> implements Serializable
  
  serialVersionUID:
  
  1L
  - Serialized Fields
    - compressionCodecName
```
String compressionCodecName
```
    - compressionType
```
org.apache.hadoop.io.SequenceFile.CompressionType compressionType
```
    - keyClass
```
Class<T> keyClass
```
    - valueClass
```
Class<T> valueClass
```
- Class org.apache.flink.streaming.connectors.fs.StreamWriterBase extends Object implements Serializable
- Class org.apache.flink.streaming.connectors.fs.StringWriter extends StreamWriterBase<T> implements Serializable
  
  serialVersionUID:
  
  1L
  - Serialized Fields
    - charsetName
```
String charsetName
```
Package org.apache.flink.streaming.connectors.fs.bucketing
- Class org.apache.flink.streaming.connectors.fs.bucketing.BasePathBucketer extends Object implements Serializable
  
  serialVersionUID:
  
  1L
- Class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink extends org.apache.flink.streaming.api.functions.sink.RichSinkFunction<T> implements Serializable
  
  serialVersionUID:
  
  1L
  - Serialized Fields
    - DEFAULT_BATCH_SIZE
```
long DEFAULT_BATCH_SIZE
```
      The default maximum size of part files (currently 384 MB).
    - DEFAULT_INACTIVE_BUCKET_CHECK_INTERVAL_MS
```
long DEFAULT_INACTIVE_BUCKET_CHECK_INTERVAL_MS
```
      The default time between checks for inactive buckets. By default, {60 sec}.
    - DEFAULT_INACTIVE_BUCKET_THRESHOLD_MS
```
long DEFAULT_INACTIVE_BUCKET_THRESHOLD_MS
```
      The default threshold (in ms) for marking a bucket as inactive and closing its part files. By default, {60 sec}.
    - DEFAULT_IN_PROGRESS_SUFFIX
```
String DEFAULT_IN_PROGRESS_SUFFIX
```
      The suffix for in-progress part files. These are files we are currently writing to, but which were not yet confirmed by a checkpoint.
    - DEFAULT_IN_PROGRESS_PREFIX
```
String DEFAULT_IN_PROGRESS_PREFIX
```
      The prefix for in-progress part files. These are files we are currently writing to, but which were not yet confirmed by a checkpoint.
    - DEFAULT_PENDING_SUFFIX
```
String DEFAULT_PENDING_SUFFIX
```
      The suffix for pending part files. These are closed files that we are not currently writing to (inactive or reached BucketingSink.batchSize), but which were not yet confirmed by a checkpoint.
    - DEFAULT_PENDING_PREFIX
```
String DEFAULT_PENDING_PREFIX
```
      The prefix for pending part files. These are closed files that we are not currently writing to (inactive or reached BucketingSink.batchSize), but which were not yet confirmed by a checkpoint.
    - DEFAULT_VALID_SUFFIX
```
String DEFAULT_VALID_SUFFIX
```
      When truncate() is not supported by the used FileSystem, we create a file along the part file with this suffix that contains the length up to which the part file is valid.
    - DEFAULT_VALID_PREFIX
```
String DEFAULT_VALID_PREFIX
```
      When truncate() is not supported by the used FileSystem, we create a file along the part file with this preffix that contains the length up to which the part file is valid.
    - DEFAULT_PART_REFIX
```
String DEFAULT_PART_REFIX
```
      The default prefix for part files.
    - DEFAULT_ASYNC_TIMEOUT_MS
```
long DEFAULT_ASYNC_TIMEOUT_MS
```
      The default timeout for asynchronous operations such as recoverLease and truncate (in ms).
    - basePath
```
String basePath
```
      The base Path that stores all bucket directories.
    - bucketer
```
Bucketer<T> bucketer
```
      The Bucketer that is used to determine the path of bucket directories.
    - writerTemplate
```
Writer<T> writerTemplate
```
      We have a template and call duplicate() for each parallel writer in open() to get the actual writer that is used for the part files.
    - batchSize
```
long batchSize
```
    - inactiveBucketCheckInterval
```
long inactiveBucketCheckInterval
```
    - inactiveBucketThreshold
```
long inactiveBucketThreshold
```
    - inProgressSuffix
```
String inProgressSuffix
```
    - inProgressPrefix
```
String inProgressPrefix
```
    - pendingSuffix
```
String pendingSuffix
```
    - pendingPrefix
```
String pendingPrefix
```
    - validLengthSuffix
```
String validLengthSuffix
```
    - validLengthPrefix
```
String validLengthPrefix
```
    - partPrefix
```
String partPrefix
```
    - asyncTimeout
```
long asyncTimeout
```
      The timeout for asynchronous operations such as recoverLease and truncate (in ms).
    - fsConfig
```
org.apache.flink.configuration.Configuration fsConfig
```
      User-defined FileSystem parameters
- Class org.apache.flink.streaming.connectors.fs.bucketing.DateTimeBucketer extends Object implements Serializable
  
  serialVersionUID:
  
  1L
  - Serialization Methods
    - readObject
```
private void readObject(ObjectInputStream in)
                 throws IOException,
                        ClassNotFoundException
```
      Throws:
      
      IOException
      
      ClassNotFoundException
  - Serialized Fields
    - formatString
```
String formatString
```

Serialized Form

Package org.apache.flink.streaming.connectors.fs

Class org.apache.flink.streaming.connectors.fs.AvroKeyValueSinkWriter extends StreamWriterBase<org.apache.flink.api.java.tuple.Tuple2<K,V>> implements Serializable

Serialized Fields

properties

Class org.apache.flink.streaming.connectors.fs.DateTimeBucketer extends Object implements Serializable

Serialization Methods

readObject

Serialized Fields

formatString

Class org.apache.flink.streaming.connectors.fs.NonRollingBucketer extends Object implements Serializable

Class org.apache.flink.streaming.connectors.fs.RollingSink extends org.apache.flink.streaming.api.functions.sink.RichSinkFunction<T> implements Serializable

Serialized Fields

DEFAULT_BATCH_SIZE

DEFAULT_IN_PROGRESS_SUFFIX

DEFAULT_IN_PROGRESS_PREFIX

DEFAULT_PENDING_SUFFIX

DEFAULT_PENDING_PREFIX

DEFAULT_VALID_SUFFIX

DEFAULT_VALID_PREFIX

DEFAULT_PART_REFIX

DEFAULT_ASYNC_TIMEOUT_MS

basePath

bucketer

writerTemplate

writer

batchSize

inProgressSuffix

inProgressPrefix

pendingSuffix

pendingPrefix

validLengthSuffix

validLengthPrefix

partPrefix

asyncTimeout

fsConfig

Class org.apache.flink.streaming.connectors.fs.RollingSink.BucketState extends Object implements Serializable

Serialized Fields

currentFile

currentFileValidLength

pendingFiles

pendingFilesPerCheckpoint

Class org.apache.flink.streaming.connectors.fs.SequenceFileWriter extends StreamWriterBase<org.apache.flink.api.java.tuple.Tuple2<K extends org.apache.hadoop.io.Writable,V extends org.apache.hadoop.io.Writable>> implements Serializable

Serialized Fields

compressionCodecName

compressionType

keyClass

valueClass

Class org.apache.flink.streaming.connectors.fs.StreamWriterBase extends Object implements Serializable

Class org.apache.flink.streaming.connectors.fs.StringWriter extends StreamWriterBase<T> implements Serializable

Serialized Fields

charsetName

Package org.apache.flink.streaming.connectors.fs.bucketing

Class org.apache.flink.streaming.connectors.fs.bucketing.BasePathBucketer extends Object implements Serializable

Class org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink extends org.apache.flink.streaming.api.functions.sink.RichSinkFunction<T> implements Serializable

Serialized Fields

DEFAULT_BATCH_SIZE

DEFAULT_INACTIVE_BUCKET_CHECK_INTERVAL_MS

DEFAULT_INACTIVE_BUCKET_THRESHOLD_MS

DEFAULT_IN_PROGRESS_SUFFIX

DEFAULT_IN_PROGRESS_PREFIX

DEFAULT_PENDING_SUFFIX

DEFAULT_PENDING_PREFIX

DEFAULT_VALID_SUFFIX

DEFAULT_VALID_PREFIX

DEFAULT_PART_REFIX

DEFAULT_ASYNC_TIMEOUT_MS

basePath

bucketer

writerTemplate

batchSize

inactiveBucketCheckInterval

inactiveBucketThreshold

inProgressSuffix

inProgressPrefix

pendingSuffix

pendingPrefix

validLengthSuffix

validLengthPrefix

partPrefix