A C E G H J N O P R S T V W

A

acquireTaskAttemptIdLock(Configuration, int) - Method in interface org.apache.beam.sdk.io.hadoop.format.ExternalSynchronization

Creates unique TaskAttemptID for given taskId.

acquireTaskAttemptIdLock(Configuration, int) - Method in class org.apache.beam.sdk.io.hadoop.format.HDFSSynchronization

acquireTaskIdLock(Configuration) - Method in interface org.apache.beam.sdk.io.hadoop.format.ExternalSynchronization

Creates TaskID with unique id among given job.

acquireTaskIdLock(Configuration) - Method in class org.apache.beam.sdk.io.hadoop.format.HDFSSynchronization

apply(T1) - Method in interface org.apache.beam.sdk.io.hadoop.format.HDFSSynchronization.ThrowingFunction

C

createInputFormatInstance() - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.HadoopInputFormatBoundedSource

Creates instance of InputFormat class.

createReader(PipelineOptions) - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.HadoopInputFormatBoundedSource

E

expand(PBegin) - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Read

expand(PCollection<KV<KeyT, ValueT>>) - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Write

ExternalSynchronization - Interface in org.apache.beam.sdk.io.hadoop.format

Provides mechanism for acquiring locks related to the job.

G

getConfiguration() - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.HadoopInputFormatBoundedSource

getConfiguration() - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Read

getDefaultCoder(TypeDescriptor<?>, CoderRegistry) - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Read

Returns the default coder for a given type descriptor.

getEstimatedSizeBytes(PipelineOptions) - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.HadoopInputFormatBoundedSource

getinputFormatClass() - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Read

getinputFormatKeyClass() - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Read

getinputFormatValueClass() - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Read

getKeyTranslationFunction() - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Read

getKeyTypeDescriptor() - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Read

getOutputCoder() - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.HadoopInputFormatBoundedSource

getSplit() - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.SerializableSplit

getValueTranslationFunction() - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Read

getValueTypeDescriptor() - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Read

H

HadoopFormatIO - Class in org.apache.beam.sdk.io.hadoop.format

A HadoopFormatIO is a Transform for reading data from any source or writing data to any sink which implements Hadoop InputFormat or OutputFormat.

HadoopFormatIO() - Constructor for class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO

HadoopFormatIO.HadoopInputFormatBoundedSource<K,V> - Class in org.apache.beam.sdk.io.hadoop.format

Bounded source implementation for HadoopFormatIO.

HadoopFormatIO.Read<K,V> - Class in org.apache.beam.sdk.io.hadoop.format

A PTransform that reads from any data source which implements Hadoop InputFormat.

HadoopFormatIO.SerializableSplit - Class in org.apache.beam.sdk.io.hadoop.format

A wrapper to allow Hadoop InputSplit to be serialized using Java's standard serialization mechanisms.

HadoopFormatIO.Write<KeyT,ValueT> - Class in org.apache.beam.sdk.io.hadoop.format

A PTransform that writes to any data sink which implements Hadoop OutputFormat.

HadoopFormatIO.Write.ExternalSynchronizationBuilder<KeyT,ValueT> - Interface in org.apache.beam.sdk.io.hadoop.format

Builder for External Synchronization defining.

HadoopFormatIO.Write.PartitionedWriterBuilder<KeyT,ValueT> - Interface in org.apache.beam.sdk.io.hadoop.format

Builder for partitioning determining.

HadoopFormatIO.Write.WriteBuilder<KeyT,ValueT> - Interface in org.apache.beam.sdk.io.hadoop.format

Main builder of Write transformation.

HadoopInputFormatBoundedSource(SerializableConfiguration, Coder<K>, Coder<V>, SimpleFunction<?, K>, SimpleFunction<?, V>, HadoopFormatIO.SerializableSplit) - Constructor for class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.HadoopInputFormatBoundedSource

HDFSSynchronization - Class in org.apache.beam.sdk.io.hadoop.format

Implementation of ExternalSynchronization which registers locks in the HDFS.

HDFSSynchronization(String) - Constructor for class org.apache.beam.sdk.io.hadoop.format.HDFSSynchronization

Creates instance of HDFSSynchronization.

HDFSSynchronization.ThrowingFunction<T1,T2,X extends java.lang.Exception> - Interface in org.apache.beam.sdk.io.hadoop.format

Function which can throw exception.

J

JOB_ID - Static variable in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO

MRJobConfig.ID.

N

NUM_REDUCES - Static variable in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO

MRJobConfig.NUM_REDUCES.

O

org.apache.beam.sdk.io.hadoop.format - package org.apache.beam.sdk.io.hadoop.format

Defines transforms for writing to Data sinks that implement HadoopFormatIO .

OUTPUT_DIR - Static variable in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO

MRJobConfig.MAPREDUCE_JOB_DIR.

OUTPUT_FORMAT_CLASS_ATTR - Static variable in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO

MRJobConfig.OUTPUT_FORMAT_CLASS_ATTR.

OUTPUT_KEY_CLASS - Static variable in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO

MRJobConfig.OUTPUT_KEY_CLASS.

OUTPUT_VALUE_CLASS - Static variable in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO

MRJobConfig.OUTPUT_VALUE_CLASS.

P

PARTITIONER_CLASS_ATTR - Static variable in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO

MRJobConfig.PARTITIONER_CLASS_ATTR.

populateDisplayData(DisplayData.Builder) - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.HadoopInputFormatBoundedSource

populateDisplayData(DisplayData.Builder) - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Write

R

read() - Static method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO

Creates an uninitialized HadoopFormatIO.Read.

Read() - Constructor for class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Read

releaseJobIdLock(Configuration) - Method in interface org.apache.beam.sdk.io.hadoop.format.ExternalSynchronization

Deletes lock ids bounded with given job if any exists.

releaseJobIdLock(Configuration) - Method in class org.apache.beam.sdk.io.hadoop.format.HDFSSynchronization

S

SerializableSplit() - Constructor for class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.SerializableSplit

SerializableSplit(InputSplit) - Constructor for class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.SerializableSplit

split(long, PipelineOptions) - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.HadoopInputFormatBoundedSource

T

toBuilder() - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Read

tryAcquireJobLock(Configuration) - Method in interface org.apache.beam.sdk.io.hadoop.format.ExternalSynchronization

Tries to acquire lock for given job.

tryAcquireJobLock(Configuration) - Method in class org.apache.beam.sdk.io.hadoop.format.HDFSSynchronization

V

validate() - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.HadoopInputFormatBoundedSource

validate(PipelineOptions) - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Write

validateTransform() - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Read

Validates construction of this transform.

W

withConfiguration(Configuration) - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Read

Reads from the source using the options provided by the given configuration.

withConfiguration(Configuration) - Method in interface org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Write.WriteBuilder

Writes to the sink using the options provided by the given hadoop configuration.

withConfigurationTransform(PTransform<PCollection<? extends KV<KeyT, ValueT>>, PCollectionView<Configuration>>) - Method in interface org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Write.WriteBuilder

Writes to the sink using configuration created by provided configurationTransformation.

withExternalSynchronization(ExternalSynchronization) - Method in interface org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Write.ExternalSynchronizationBuilder

Specifies class which will provide external synchronization required for hadoop write operation.

withKeyTranslation(SimpleFunction<?, K>) - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Read

Transforms the keys read from the source using the given key translation function.

withoutPartitioning() - Method in interface org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Write.PartitionedWriterBuilder

Writes to the sink without need to partition output into specified number of partitions.

withPartitioning() - Method in interface org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Write.PartitionedWriterBuilder

Writes to the sink with partitioning by Task Id.

withValueTranslation(SimpleFunction<?, V>) - Method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Read

Transforms the values read from the source using the given value translation function.

write() - Static method in class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO

Creates an Write.Builder for creation of Write Transformation.

Write(Configuration, PTransform<PCollection<? extends KV<KeyT, ValueT>>, PCollectionView<Configuration>>, ExternalSynchronization, boolean) - Constructor for class org.apache.beam.sdk.io.hadoop.format.HadoopFormatIO.Write

A C E G H J N O P R S T V W