Package org.apache.beam.sdk.io.sparkreceiver

Class WrappedSupervisor

  • All Implemented Interfaces:
    org.apache.spark.internal.Logging
    public class WrappedSupervisor
extends org.apache.spark.streaming.receiver.ReceiverSupervisor
    Wrapper class for ReceiverSupervisor that doesn't use Spark Environment.

    • Nested Class Summary

      • Nested classes/interfaces inherited from class org.apache.spark.streaming.receiver.ReceiverSupervisor

        org.apache.spark.streaming.receiver.ReceiverSupervisor.ReceiverState$

    • Constructor Summary

      Constructors 
      Constructor Description
      WrappedSupervisor​(org.apache.spark.streaming.receiver.Receiver<?> receiver, org.apache.spark.SparkConf conf, org.apache.beam.sdk.transforms.SerializableFunction<java.lang.Object[],​java.lang.Void> storeFn)  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      org.apache.spark.streaming.receiver.BlockGenerator createBlockGenerator​(org.apache.spark.streaming.receiver.BlockGeneratorListener blockGeneratorListener)  
      long getCurrentRateLimit()  
      boolean isReceiverStopped()  
      void logInfo​(scala.Function0<java.lang.String> msg)  
      boolean onReceiverStart()  
      void pushArrayBuffer​(scala.collection.mutable.ArrayBuffer<?> arrayBuffer, scala.Option<java.lang.Object> option, scala.Option<org.apache.spark.storage.StreamBlockId> option1)  
      void pushBytes​(java.nio.ByteBuffer byteBuffer, scala.Option<java.lang.Object> option, scala.Option<org.apache.spark.storage.StreamBlockId> option1)  
      void pushIterator​(scala.collection.Iterator<?> iterator, scala.Option<java.lang.Object> option, scala.Option<org.apache.spark.storage.StreamBlockId> option1)  
      void pushSingle​(java.lang.Object o)  
      void reportError​(java.lang.String s, java.lang.Throwable throwable)  

      • Methods inherited from class org.apache.spark.streaming.receiver.ReceiverSupervisor

        awaitTermination, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, isReceiverStarted, isTraceEnabled, log, logDebug, logDebug, logError, logError, logInfo, logName, logTrace, logTrace, logWarning, logWarning, onReceiverStop, onStart, onStop, org$apache$spark$internal$Logging$$log_, org$apache$spark$internal$Logging$$log__$eq, receiverState, ReceiverState, receiverState_$eq, restartReceiver, restartReceiver, restartReceiver$default$2, start, startReceiver, stop, stoppingError, stoppingError_$eq, stopReceiver, streamId

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • WrappedSupervisor

        public WrappedSupervisor​(org.apache.spark.streaming.receiver.Receiver<?> receiver,
                         org.apache.spark.SparkConf conf,
                         org.apache.beam.sdk.transforms.SerializableFunction<java.lang.Object[],​java.lang.Void> storeFn)

    • Method Detail

      • pushSingle

        public void pushSingle​(java.lang.Object o)
        Specified by:
        pushSingle in class org.apache.spark.streaming.receiver.ReceiverSupervisor

      • pushBytes

        public void pushBytes​(java.nio.ByteBuffer byteBuffer,
                      scala.Option<java.lang.Object> option,
                      scala.Option<org.apache.spark.storage.StreamBlockId> option1)
        Specified by:
        pushBytes in class org.apache.spark.streaming.receiver.ReceiverSupervisor

      • pushIterator

        public void pushIterator​(scala.collection.Iterator<?> iterator,
                         scala.Option<java.lang.Object> option,
                         scala.Option<org.apache.spark.storage.StreamBlockId> option1)
        Specified by:
        pushIterator in class org.apache.spark.streaming.receiver.ReceiverSupervisor

      • pushArrayBuffer

        public void pushArrayBuffer​(scala.collection.mutable.ArrayBuffer<?> arrayBuffer,
                            scala.Option<java.lang.Object> option,
                            scala.Option<org.apache.spark.storage.StreamBlockId> option1)
        Specified by:
        pushArrayBuffer in class org.apache.spark.streaming.receiver.ReceiverSupervisor

      • createBlockGenerator

        public org.apache.spark.streaming.receiver.BlockGenerator createBlockGenerator​(org.apache.spark.streaming.receiver.BlockGeneratorListener blockGeneratorListener)
        Specified by:
        createBlockGenerator in class org.apache.spark.streaming.receiver.ReceiverSupervisor

      • reportError

        public void reportError​(java.lang.String s,
                        java.lang.Throwable throwable)
        Specified by:
        reportError in class org.apache.spark.streaming.receiver.ReceiverSupervisor

      • onReceiverStart

        public boolean onReceiverStart()
        Specified by:
        onReceiverStart in class org.apache.spark.streaming.receiver.ReceiverSupervisor

      • getCurrentRateLimit

        public long getCurrentRateLimit()
        Overrides:
        getCurrentRateLimit in class org.apache.spark.streaming.receiver.ReceiverSupervisor

      • isReceiverStopped

        public boolean isReceiverStopped()
        Overrides:
        isReceiverStopped in class org.apache.spark.streaming.receiver.ReceiverSupervisor

      • logInfo

        public void logInfo​(scala.Function0<java.lang.String> msg)
        Specified by:
        logInfo in interface org.apache.spark.internal.Logging
        Overrides:
        logInfo in class org.apache.spark.streaming.receiver.ReceiverSupervisor