Skip navigation links
A B C E F G I J L O P S V X 

A

AvroReadSchemaTransformFormatProvider - Class in org.apache.beam.sdk.io.fileschematransform
 
AvroReadSchemaTransformFormatProvider() - Constructor for class org.apache.beam.sdk.io.fileschematransform.AvroReadSchemaTransformFormatProvider
 
AvroWriteSchemaTransformFormatProvider - Class in org.apache.beam.sdk.io.fileschematransform
AvroWriteSchemaTransformFormatProvider() - Constructor for class org.apache.beam.sdk.io.fileschematransform.AvroWriteSchemaTransformFormatProvider
 

B

build() - Method in class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformConfiguration.Builder
 
build() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.Builder
 
build() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.CsvConfiguration.Builder
 
build() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.ParquetConfiguration.Builder
 
build() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.XmlConfiguration.Builder
 
builder() - Static method in class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformConfiguration
 
Builder() - Constructor for class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformConfiguration.Builder
 
builder() - Static method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration
 
Builder() - Constructor for class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.Builder
 
Builder() - Constructor for class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.CsvConfiguration.Builder
 
Builder() - Constructor for class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.ParquetConfiguration.Builder
 
Builder() - Constructor for class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.XmlConfiguration.Builder
 
buildTransform(FileReadSchemaTransformConfiguration) - Method in class org.apache.beam.sdk.io.fileschematransform.AvroReadSchemaTransformFormatProvider
 
buildTransform(FileWriteSchemaTransformConfiguration, Schema) - Method in class org.apache.beam.sdk.io.fileschematransform.AvroWriteSchemaTransformFormatProvider
Builds a PTransform that transforms a Row PCollection into result PCollectionTuple with two tags, one for file names written using AvroIO.Write, another for errored-out rows.
buildTransform(FileWriteSchemaTransformConfiguration, Schema) - Method in class org.apache.beam.sdk.io.fileschematransform.CsvWriteSchemaTransformFormatProvider
 
buildTransform(FileReadSchemaTransformConfiguration) - Method in interface org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformFormatProvider
 
buildTransform(FileWriteSchemaTransformConfiguration, Schema) - Method in interface org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformFormatProvider
Builds a PTransform that writes a Row PCollection and outputs the resulting PCollectionTuple with two tags, one for the file names, and another errored-out rows.
buildTransform(FileReadSchemaTransformConfiguration) - Method in class org.apache.beam.sdk.io.fileschematransform.JsonReadSchemaTransformFormatProvider
 
buildTransform(FileWriteSchemaTransformConfiguration, Schema) - Method in class org.apache.beam.sdk.io.fileschematransform.JsonWriteSchemaTransformFormatProvider
Builds a PTransform that transforms a Row PCollection into result PCollectionTuple with two tags, one for file names written using TextIO.Write, another for errored-out rows.
buildTransform(FileReadSchemaTransformConfiguration) - Method in class org.apache.beam.sdk.io.fileschematransform.LineReadSchemaTransformFormatProvider
 
buildTransform(FileReadSchemaTransformConfiguration) - Method in class org.apache.beam.sdk.io.fileschematransform.ParquetReadSchemaTransformFormatProvider
 
buildTransform(FileWriteSchemaTransformConfiguration, Schema) - Method in class org.apache.beam.sdk.io.fileschematransform.ParquetWriteSchemaTransformFormatProvider
Builds a PTransform that transforms a Row PCollection into result PCollectionTuple with two tags, one for file names written using ParquetIO.Sink and FileIO.Write, another for errored-out rows.
buildTransform(FileWriteSchemaTransformConfiguration, Schema) - Method in class org.apache.beam.sdk.io.fileschematransform.XmlWriteSchemaTransformFormatProvider
Builds a PTransform that transforms a Row PCollection into result PCollectionTuple with two tags, one for file names written using XmlIO.Sink and FileIO.Write, another for errored-out rows.

C

configurationClass() - Method in class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformProvider
 
configurationClass() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformProvider
Provides the required TypedSchemaTransformProvider.configurationClass().
CsvConfiguration() - Constructor for class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.CsvConfiguration
 
csvConfigurationBuilder() - Static method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration
 
CsvWriteSchemaTransformFormatProvider - Class in org.apache.beam.sdk.io.fileschematransform
CsvWriteSchemaTransformFormatProvider() - Constructor for class org.apache.beam.sdk.io.fileschematransform.CsvWriteSchemaTransformFormatProvider
 

E

ERROR_SCHEMA - Static variable in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformProvider
 

F

FILE_NAME_FIELD - Static variable in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformProvider
 
FileReadSchemaTransformConfiguration - Class in org.apache.beam.sdk.io.fileschematransform
 
FileReadSchemaTransformConfiguration() - Constructor for class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformConfiguration
 
FileReadSchemaTransformConfiguration.Builder - Class in org.apache.beam.sdk.io.fileschematransform
 
FileReadSchemaTransformFormatProvider - Interface in org.apache.beam.sdk.io.fileschematransform
Interface that provides a PTransform that reads in a PCollection of FileIO.ReadableFiles and outputs the data represented as a PCollection of Rows.
FileReadSchemaTransformProvider - Class in org.apache.beam.sdk.io.fileschematransform
 
FileReadSchemaTransformProvider() - Constructor for class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformProvider
 
FileWriteSchemaTransformConfiguration - Class in org.apache.beam.sdk.io.fileschematransform
The configuration for building file writing transforms using SchemaTransform and SchemaTransformProvider.
FileWriteSchemaTransformConfiguration() - Constructor for class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration
 
FileWriteSchemaTransformConfiguration.Builder - Class in org.apache.beam.sdk.io.fileschematransform
 
FileWriteSchemaTransformConfiguration.CsvConfiguration - Class in org.apache.beam.sdk.io.fileschematransform
Configures extra details related to writing CSV formatted files.
FileWriteSchemaTransformConfiguration.CsvConfiguration.Builder - Class in org.apache.beam.sdk.io.fileschematransform
 
FileWriteSchemaTransformConfiguration.ParquetConfiguration - Class in org.apache.beam.sdk.io.fileschematransform
Configures extra details related to writing Parquet formatted files.
FileWriteSchemaTransformConfiguration.ParquetConfiguration.Builder - Class in org.apache.beam.sdk.io.fileschematransform
 
FileWriteSchemaTransformConfiguration.XmlConfiguration - Class in org.apache.beam.sdk.io.fileschematransform
Configures extra details related to writing XML formatted files.
FileWriteSchemaTransformConfiguration.XmlConfiguration.Builder - Class in org.apache.beam.sdk.io.fileschematransform
 
FileWriteSchemaTransformFormatProvider - Interface in org.apache.beam.sdk.io.fileschematransform
Provides a PTransform that writes a PCollection of Rows and outputs a PCollection of the file names according to a registered AutoService FileWriteSchemaTransformFormatProvider implementation.
FileWriteSchemaTransformFormatProviders - Class in org.apache.beam.sdk.io.fileschematransform
FileWriteSchemaTransformFormatProviders() - Constructor for class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformFormatProviders
 
FileWriteSchemaTransformProvider - Class in org.apache.beam.sdk.io.fileschematransform
A TypedSchemaTransformProvider implementation for writing a Row PCollection to file systems, driven by a FileWriteSchemaTransformConfiguration.
FileWriteSchemaTransformProvider() - Constructor for class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformProvider
 
from(FileReadSchemaTransformConfiguration) - Method in class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformProvider
 
from(FileWriteSchemaTransformConfiguration) - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformProvider
Builds a SchemaTransform from a FileWriteSchemaTransformConfiguration.

G

getCharset() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.XmlConfiguration
 
getCompression() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration
See Compression for expected values.
getCompressionCodecName() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.ParquetConfiguration
 
getCsvConfiguration() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration
 
getFilenamePrefix() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration
 
getFilenameSuffix() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration
 
getFilepattern() - Method in class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformConfiguration
The filepattern used to match and read files.
getFormat() - Method in class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformConfiguration
The format of the file(s) to read.
getFormat() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration
 
getNumShards() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration
 
getParquetConfiguration() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration
 
getPollIntervalMillis() - Method in class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformConfiguration
The time, in milliseconds, to wait before polling for new files.
getPredefinedCsvFormat() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.CsvConfiguration
See CSVFormat.Predefined#values() for a list of allowed values.
getRootElement() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.XmlConfiguration
 
getRowGroupSize() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.ParquetConfiguration
 
getSafeFilepattern() - Method in class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformConfiguration
 
getSafeSchema() - Method in class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformConfiguration
 
getSchema() - Method in class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformConfiguration
The schema used by sources to deserialize data and create Beam Rows.
getShardNameTemplate() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration
See ShardNameTemplate for the expected values.
getTerminateAfterSecondsSinceNewOutput() - Method in class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformConfiguration
If no new files are found after this many seconds, this transform will cease to watch for new files.
getXmlConfiguration() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration
 

I

identifier() - Method in class org.apache.beam.sdk.io.fileschematransform.AvroReadSchemaTransformFormatProvider
 
identifier() - Method in class org.apache.beam.sdk.io.fileschematransform.AvroWriteSchemaTransformFormatProvider
 
identifier() - Method in class org.apache.beam.sdk.io.fileschematransform.CsvWriteSchemaTransformFormatProvider
 
identifier() - Method in class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformProvider
 
identifier() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformProvider
Returns the SchemaTransformProvider.identifier() required for registration.
identifier() - Method in class org.apache.beam.sdk.io.fileschematransform.JsonReadSchemaTransformFormatProvider
 
identifier() - Method in class org.apache.beam.sdk.io.fileschematransform.JsonWriteSchemaTransformFormatProvider
 
identifier() - Method in class org.apache.beam.sdk.io.fileschematransform.LineReadSchemaTransformFormatProvider
 
identifier() - Method in class org.apache.beam.sdk.io.fileschematransform.ParquetReadSchemaTransformFormatProvider
 
identifier() - Method in class org.apache.beam.sdk.io.fileschematransform.ParquetWriteSchemaTransformFormatProvider
 
identifier() - Method in class org.apache.beam.sdk.io.fileschematransform.XmlWriteSchemaTransformFormatProvider
 
inputCollectionNames() - Method in class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformProvider
 
inputCollectionNames() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformProvider
The expected PCollectionRowTuple input tags.

J

JsonReadSchemaTransformFormatProvider - Class in org.apache.beam.sdk.io.fileschematransform
A FileReadSchemaTransformFormatProvider that reads newline-delimited JSONs.
JsonReadSchemaTransformFormatProvider() - Constructor for class org.apache.beam.sdk.io.fileschematransform.JsonReadSchemaTransformFormatProvider
 
JsonWriteSchemaTransformFormatProvider - Class in org.apache.beam.sdk.io.fileschematransform
JsonWriteSchemaTransformFormatProvider() - Constructor for class org.apache.beam.sdk.io.fileschematransform.JsonWriteSchemaTransformFormatProvider
 

L

LineReadSchemaTransformFormatProvider - Class in org.apache.beam.sdk.io.fileschematransform
A FileReadSchemaTransformFormatProvider that reads lines as Strings.
LineReadSchemaTransformFormatProvider() - Constructor for class org.apache.beam.sdk.io.fileschematransform.LineReadSchemaTransformFormatProvider
 
loadProviders() - Static method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformFormatProviders
Load all FileWriteSchemaTransformFormatProvider implementations.

O

org.apache.beam.sdk.io.fileschematransform - package org.apache.beam.sdk.io.fileschematransform
Defines transforms for File reading and writing support with Schema Transform.
OUTPUT_SCHEMA - Static variable in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformProvider
 
outputCollectionNames() - Method in class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformProvider
 
outputCollectionNames() - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformProvider
The expected PCollectionRowTuple output tags.

P

ParquetConfiguration() - Constructor for class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.ParquetConfiguration
 
parquetConfigurationBuilder() - Static method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration
 
ParquetReadSchemaTransformFormatProvider - Class in org.apache.beam.sdk.io.fileschematransform
 
ParquetReadSchemaTransformFormatProvider() - Constructor for class org.apache.beam.sdk.io.fileschematransform.ParquetReadSchemaTransformFormatProvider
 
ParquetWriteSchemaTransformFormatProvider - Class in org.apache.beam.sdk.io.fileschematransform
ParquetWriteSchemaTransformFormatProvider() - Constructor for class org.apache.beam.sdk.io.fileschematransform.ParquetWriteSchemaTransformFormatProvider
 

S

setCharset(String) - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.XmlConfiguration.Builder
The charset used to write the file.
setCompression(String) - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.Builder
The file Compression See FileWriteSchemaTransformConfiguration.getCompression() for more details.
setCompressionCodecName(String) - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.ParquetConfiguration.Builder
Specifies compression codec.
setCsvConfiguration(FileWriteSchemaTransformConfiguration.CsvConfiguration) - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.Builder
Configures extra details related to writing CSV formatted files.
setFilenamePrefix(String) - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.Builder
A common prefix to use for all generated filenames.
setFilenameSuffix(String) - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.Builder
Configures the filename suffix for written files.
setFilepattern(String) - Method in class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformConfiguration.Builder
 
setFormat(String) - Method in class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformConfiguration.Builder
 
setFormat(String) - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.Builder
The format of the file content.
setNumShards(Integer) - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.Builder
The number of output shards produced; a value of 1 disables sharding.
setParquetConfiguration(FileWriteSchemaTransformConfiguration.ParquetConfiguration) - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.Builder
Configures extra details related to writing Parquet formatted files.
setPollIntervalMillis(Long) - Method in class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformConfiguration.Builder
 
setPredefinedCsvFormat(String) - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.CsvConfiguration.Builder
The CSVFormat.Predefined#name() of the written CSV file.
setRootElement(String) - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.XmlConfiguration.Builder
Sets the enclosing root element for the generated XML files.
setRowGroupSize(Integer) - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.ParquetConfiguration.Builder
Specify row-group size; if not set or zero, a default is used by the underlying writer.
setSchema(String) - Method in class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformConfiguration.Builder
 
setShardNameTemplate(String) - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.Builder
Uses the given ShardNameTemplate for naming output files.
setTerminateAfterSecondsSinceNewOutput(Long) - Method in class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformConfiguration.Builder
 
setXmlConfiguration(FileWriteSchemaTransformConfiguration.XmlConfiguration) - Method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.Builder
Configures extra details related to writing XML formatted files.

V

VALID_PROVIDERS - Static variable in class org.apache.beam.sdk.io.fileschematransform.FileReadSchemaTransformConfiguration
 

X

XmlConfiguration() - Constructor for class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration.XmlConfiguration
 
xmlConfigurationBuilder() - Static method in class org.apache.beam.sdk.io.fileschematransform.FileWriteSchemaTransformConfiguration
 
XmlWriteSchemaTransformFormatProvider - Class in org.apache.beam.sdk.io.fileschematransform
XmlWriteSchemaTransformFormatProvider() - Constructor for class org.apache.beam.sdk.io.fileschematransform.XmlWriteSchemaTransformFormatProvider
 
A B C E F G I J L O P S V X 
Skip navigation links