Package org.apache.beam.sdk.io.sparkreceiver

Class SparkReceiverIO.Read<V>

  • All Implemented Interfaces:
    java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData
    Enclosing class:
    SparkReceiverIO
    public abstract static class SparkReceiverIO.Read<V>
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,​org.apache.beam.sdk.values.PCollection<V>>
    A PTransform to read from Spark Receiver.
    See Also:
    Serialized Form

    • Field Summary

      • Fields inherited from class org.apache.beam.sdk.transforms.PTransform

        name, resourceHints

    • Constructor Summary

      Constructors 
      Constructor Description
      Read()  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      org.apache.beam.sdk.values.PCollection<V> expand​(org.apache.beam.sdk.values.PBegin input)  
      void validateTransform()  
      SparkReceiverIO.Read<V> withGetOffsetFn​(org.apache.beam.sdk.transforms.SerializableFunction<V,​java.lang.Long> getOffsetFn)
      A function to get offset in order to start Receiver from it.
      SparkReceiverIO.Read<V> withSparkReceiverBuilder​(ReceiverBuilder<V,​? extends org.apache.spark.streaming.receiver.Receiver<V>> sparkReceiverBuilder)
      Sets ReceiverBuilder with value and custom Spark Receiver class.
      SparkReceiverIO.Read<V> withTimestampFn​(org.apache.beam.sdk.transforms.SerializableFunction<V,​org.joda.time.Instant> timestampFn)
      A function to calculate timestamp for a record.

      • Methods inherited from class org.apache.beam.sdk.transforms.PTransform

        compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setResourceHints, toString, validate, validate

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    • Constructor Detail

      • Read

        public Read()

    • Method Detail

      • withGetOffsetFn

        public SparkReceiverIO.Read<V> withGetOffsetFn​(org.apache.beam.sdk.transforms.SerializableFunction<V,​java.lang.Long> getOffsetFn)
        A function to get offset in order to start Receiver from it.

      • withTimestampFn

        public SparkReceiverIO.Read<V> withTimestampFn​(org.apache.beam.sdk.transforms.SerializableFunction<V,​org.joda.time.Instant> timestampFn)
        A function to calculate timestamp for a record.

      • expand

        public org.apache.beam.sdk.values.PCollection<V> expand​(org.apache.beam.sdk.values.PBegin input)
        Specified by:
        expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,​org.apache.beam.sdk.values.PCollection<V>>

      • validateTransform

        public void validateTransform()