org.apache.beam.sdk.io.hadoop.format

Class HadoopFormatIO.Write<KeyT,ValueT>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<KeyT,ValueT>>,org.apache.beam.sdk.values.PDone>

org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Write<KeyT,ValueT>

Type Parameters:

KeyT - Type of keys to be written.

ValueT - Type of values to be written.

All Implemented Interfaces:

java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

Enclosing class:

HadoopFormatIO

public static class HadoopFormatIO.Write<KeyT,ValueT>
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<KeyT,ValueT>>,org.apache.beam.sdk.values.PDone>

A PTransform that writes to any data sink which implements Hadoop OutputFormat. For e.g. Cassandra, Elasticsearch, HBase, Redis, Postgres, etc. See the class-level Javadoc on HadoopFormatIO for more information.

See Also:

HadoopFormatIO, Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	HadoopFormatIO.Write.ExternalSynchronizationBuilder<KeyT,ValueT> Builder for External Synchronization defining.
static interface	HadoopFormatIO.Write.PartitionedWriterBuilder<KeyT,ValueT> Builder for partitioning determining.
static interface	HadoopFormatIO.Write.WriteBuilder<KeyT,ValueT> Main builder of Write transformation.

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

annotations, name, resourceHints

Method Summary

Modifier and Type	Method and Description
org.apache.beam.sdk.values.PDone	expand(org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<KeyT,ValueT>> input)
void	populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)
void	validate(org.apache.beam.sdk.options.PipelineOptions pipelineOptions)

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, setResourceHints, toString, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

validate

public void validate(org.apache.beam.sdk.options.PipelineOptions pipelineOptions)

Overrides:

validate in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<KeyT,ValueT>>,org.apache.beam.sdk.values.PDone>

populateDisplayData

public void populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)

Specified by:

populateDisplayData in interface org.apache.beam.sdk.transforms.display.HasDisplayData

Overrides:

populateDisplayData in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<KeyT,ValueT>>,org.apache.beam.sdk.values.PDone>

expand

public org.apache.beam.sdk.values.PDone expand(org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<KeyT,ValueT>> input)

Specified by:

expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<KeyT,ValueT>>,org.apache.beam.sdk.values.PDone>