org.apache.beam.sdk.io.synthetic

Class SyntheticUnboundedSource

java.lang.Object

org.apache.beam.sdk.io.Source<OutputT>

org.apache.beam.sdk.io.UnboundedSource<org.apache.beam.sdk.values.KV<byte[],byte[]>,org.apache.beam.sdk.io.synthetic.SyntheticRecordsCheckpoint>

org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource

All Implemented Interfaces:

java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

@Experimental(value=SOURCE_SINK)
public class SyntheticUnboundedSource
extends org.apache.beam.sdk.io.UnboundedSource<org.apache.beam.sdk.values.KV<byte[],byte[]>,org.apache.beam.sdk.io.synthetic.SyntheticRecordsCheckpoint>

A SyntheticUnboundedSource that reads KV<byte[], byte[]>.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.beam.sdk.io.UnboundedSource

org.apache.beam.sdk.io.UnboundedSource.CheckpointMark, org.apache.beam.sdk.io.UnboundedSource.UnboundedReader<OutputT>

Nested classes/interfaces inherited from class org.apache.beam.sdk.io.Source

org.apache.beam.sdk.io.Source.Reader<T>

Constructor Summary

Constructors

Constructor and Description
SyntheticUnboundedSource(SyntheticSourceOptions sourceOptions)

Method Summary

Modifier and Type	Method and Description
org.apache.beam.sdk.io.UnboundedSource.UnboundedReader<org.apache.beam.sdk.values.KV<byte[],byte[]>>	createReader(org.apache.beam.sdk.options.PipelineOptions options, @Nullable org.apache.beam.sdk.io.synthetic.SyntheticRecordsCheckpoint checkpoint)
org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.io.synthetic.SyntheticRecordsCheckpoint>	getCheckpointMarkCoder()
org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.values.KV<byte[],byte[]>>	getOutputCoder()
java.util.List<SyntheticUnboundedSource>	split(int desiredNumSplits, org.apache.beam.sdk.options.PipelineOptions options)
void	validate()

Methods inherited from class org.apache.beam.sdk.io.UnboundedSource

requiresDeduping

Methods inherited from class org.apache.beam.sdk.io.Source

getDefaultOutputCoder, populateDisplayData

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SyntheticUnboundedSource

public SyntheticUnboundedSource(SyntheticSourceOptions sourceOptions)

Method Detail

getOutputCoder

public org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.values.KV<byte[],byte[]>> getOutputCoder()

Overrides:

getOutputCoder in class org.apache.beam.sdk.io.Source<org.apache.beam.sdk.values.KV<byte[],byte[]>>

validate

public void validate()

Overrides:

validate in class org.apache.beam.sdk.io.Source<org.apache.beam.sdk.values.KV<byte[],byte[]>>

getCheckpointMarkCoder

public org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.io.synthetic.SyntheticRecordsCheckpoint> getCheckpointMarkCoder()

Specified by:

getCheckpointMarkCoder in class org.apache.beam.sdk.io.UnboundedSource<org.apache.beam.sdk.values.KV<byte[],byte[]>,org.apache.beam.sdk.io.synthetic.SyntheticRecordsCheckpoint>

createReader

public org.apache.beam.sdk.io.UnboundedSource.UnboundedReader<org.apache.beam.sdk.values.KV<byte[],byte[]>> createReader(org.apache.beam.sdk.options.PipelineOptions options,
                                                                                                                         @Nullable org.apache.beam.sdk.io.synthetic.SyntheticRecordsCheckpoint checkpoint)

Specified by:

createReader in class org.apache.beam.sdk.io.UnboundedSource<org.apache.beam.sdk.values.KV<byte[],byte[]>,org.apache.beam.sdk.io.synthetic.SyntheticRecordsCheckpoint>

split

public java.util.List<SyntheticUnboundedSource> split(int desiredNumSplits,
                                                      org.apache.beam.sdk.options.PipelineOptions options)

Specified by:

split in class org.apache.beam.sdk.io.UnboundedSource<org.apache.beam.sdk.values.KV<byte[],byte[]>,org.apache.beam.sdk.io.synthetic.SyntheticRecordsCheckpoint>