org.apache.beam.sdk.io.csv

Class CsvIO.Write<T>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<T>,org.apache.beam.sdk.io.WriteFilesResult<java.lang.String>>

org.apache.beam.sdk.io.csv.CsvIO.Write<T>

All Implemented Interfaces:

java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

Enclosing class:

CsvIO

public abstract static class CsvIO.Write<T>
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<T>,org.apache.beam.sdk.io.WriteFilesResult<java.lang.String>>
implements org.apache.beam.sdk.transforms.display.HasDisplayData

PTransform for writing CSV files.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

annotations, displayData, name, resourceHints

Constructor Summary

Constructors

Constructor and Description
Write()

Method Summary

Modifier and Type	Method and Description
org.apache.beam.sdk.io.WriteFilesResult<java.lang.String>	expand(org.apache.beam.sdk.values.PCollection<T> input)
void	populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)
CsvIO.Write<T>	withCompression(org.apache.beam.sdk.io.Compression compression) Specifies the Compression of all generated shard files.
CsvIO.Write<T>	withNoSpilling() Whether to skip the spilling of data.
CsvIO.Write<T>	withNumShards(java.lang.Integer numShards) Specifies to use a given fixed number of shards per window.
CsvIO.Write<T>	withoutSharding() Forces a single file as output and empty shard name template.
CsvIO.Write<T>	withShardTemplate(java.lang.String shardTemplate) Uses the given ShardNameTemplate for naming output files.
CsvIO.Write<T>	withSuffix(java.lang.String suffix) Configures the filename suffix for written files.
CsvIO.Write<T>	withTempDirectory(org.apache.beam.sdk.io.fs.ResourceId tempDirectory) Set the base directory used to generate temporary files.
CsvIO.Write<T>	withWindowedWrites() Preserves windowing of input elements and writes them to files based on the element's window.
CsvIO.Write<T>	withWritableByteChannelFactory(org.apache.beam.sdk.io.FileBasedSink.WritableByteChannelFactory writableByteChannelFactory) Returns a transform for writing to text files like this one but that has the given FileBasedSink.WritableByteChannelFactory to be used by the FileBasedSink during output.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, setDisplayData, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Write

public Write()

Method Detail

populateDisplayData

public void populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)

Specified by:

populateDisplayData in interface org.apache.beam.sdk.transforms.display.HasDisplayData

Overrides:

populateDisplayData in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<T>,org.apache.beam.sdk.io.WriteFilesResult<java.lang.String>>

withCompression

public CsvIO.Write<T> withCompression(org.apache.beam.sdk.io.Compression compression)

Specifies the Compression of all generated shard files.

withNoSpilling

public CsvIO.Write<T> withNoSpilling()

Whether to skip the spilling of data. See WriteFiles.withNoSpilling().

withNumShards

public CsvIO.Write<T> withNumShards(java.lang.Integer numShards)

Specifies to use a given fixed number of shards per window. See TextIO.Write#withNumShards.

withoutSharding

public CsvIO.Write<T> withoutSharding()

Forces a single file as output and empty shard name template. See TextIO.Write#withoutSharding.

withShardTemplate

public CsvIO.Write<T> withShardTemplate(java.lang.String shardTemplate)

Uses the given ShardNameTemplate for naming output files. See TextIO.Write#withShardNameTemplate.

withSuffix

public CsvIO.Write<T> withSuffix(java.lang.String suffix)

Configures the filename suffix for written files. See TextIO.Write#withSuffix.

withTempDirectory

public CsvIO.Write<T> withTempDirectory(org.apache.beam.sdk.io.fs.ResourceId tempDirectory)

Set the base directory used to generate temporary files. See TextIO.Write#withTempDirectory.

withWindowedWrites

public CsvIO.Write<T> withWindowedWrites()

Preserves windowing of input elements and writes them to files based on the element's window. See TextIO.Write#withWindowedWrites.

withWritableByteChannelFactory

public CsvIO.Write<T> withWritableByteChannelFactory(org.apache.beam.sdk.io.FileBasedSink.WritableByteChannelFactory writableByteChannelFactory)

Returns a transform for writing to text files like this one but that has the given FileBasedSink.WritableByteChannelFactory to be used by the FileBasedSink during output. See TextIO.Write#withWritableByteChannelFactory.

expand

public org.apache.beam.sdk.io.WriteFilesResult<java.lang.String> expand(org.apache.beam.sdk.values.PCollection<T> input)

Specified by:

expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<T>,org.apache.beam.sdk.io.WriteFilesResult<java.lang.String>>