c

org.apache.spark.network.client

TransportClientFactory

class TransportClientFactory extends Closeable

Factory for creating TransportClients by using createClient.

The factory maintains a connection pool to other hosts and should return the same TransportClient for the same remote host. It also shares a single worker thread pool for all TransportClients.

TransportClients will be reused whenever possible. Prior to completing the creation of a new TransportClient, all given TransportClientBootstraps will be run.

Linear Supertypes
Closeable, AutoCloseable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. TransportClientFactory
  2. Closeable
  3. AutoCloseable
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new TransportClientFactory(context: TransportContext, clientBootstraps: List[TransportClientBootstrap])

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. def close(): Unit

    Close all connections in the connection pool, and shutdown the worker thread pool.

    Close all connections in the connection pool, and shutdown the worker thread pool.

    Definition Classes
    TransportClientFactory → Closeable → AutoCloseable
    Annotations
    @Override()
  7. def createClient(remoteHost: String, remotePort: Int): TransportClient
  8. def createClient(remoteHost: String, remotePort: Int, fastFail: Boolean): TransportClient

    Create a TransportClient connecting to the given remote host / port.

    Create a TransportClient connecting to the given remote host / port.

    We maintain an array of clients (size determined by spark.shuffle.io.numConnectionsPerPeer) and randomly picks one to use. If no client was previously created in the randomly selected spot, this function creates a new client and places it there.

    If the fastFail parameter is true, fail immediately when the last attempt to the same address failed within the fast fail time window (95 percent of the io wait retry timeout). The assumption is the caller will handle retrying.

    Prior to the creation of a new TransportClient, we will execute all TransportClientBootstraps that are registered with this factory.

    This blocks until a connection is successfully established and fully bootstrapped.

    Concurrency: This method is safe to call from multiple threads.

    remoteHost

    remote address host

    remotePort

    remote address port

    fastFail

    whether this call should fail immediately when the last attempt to the same address failed with in the last fast fail time window.

  9. def createUnmanagedClient(remoteHost: String, remotePort: Int): TransportClient

    Create a completely new TransportClient to the given remote host / port.

    Create a completely new TransportClient to the given remote host / port. This connection is not pooled.

    As with int), this method is blocking.

  10. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  11. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  12. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  13. def getAllMetrics(): MetricSet
  14. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  15. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  16. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  17. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  18. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  19. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  20. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  21. def toString(): String
    Definition Classes
    AnyRef → Any
  22. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  23. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  24. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from Closeable

Inherited from AutoCloseable

Inherited from AnyRef

Inherited from Any

Ungrouped