Packages

c

net.snowflake.spark.snowflake.s3upload

MultiPartOutputStream

class MultiPartOutputStream extends OutputStream

An OutputStream which packages data written to it into discrete StreamParts which can be obtained in a separate thread via iteration and uploaded to S3.

A single MultiPartOutputStream is allocated a range of part numbers it can assign to the StreamParts it produces, which is determined at construction.

Creating a StreamPart is triggered when MultiPartOutputStream#checkSize() is called and the stream holds enough data, so be sure to call this regularly when writing data. It's also essential to call MultiPartOutputStream#close() when finished so that it can create the final StreamPart and consumers can finish.

Linear Supertypes
OutputStream, Flushable, Closeable, AutoCloseable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. MultiPartOutputStream
  2. OutputStream
  3. Flushable
  4. Closeable
  5. AutoCloseable
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new MultiPartOutputStream(partNumberStart: Int, partNumberEnd: Int, partSize: Int, queue: BlockingQueue[StreamPart])

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def checkSize(): Unit

    Checks if the stream currently contains enough data to create a new part.

    Checks if the stream currently contains enough data to create a new part.

    Internally this puts the part on a queue, which if full will cause this method to block and potentially throw an InterruptedException, in which case retrying is acceptable.

  6. def clone(): AnyRef
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @native() @throws( ... )
  7. def close(): Unit

    Packages any remaining data into a StreamPart and signals to the StreamTransferManager that there are no more parts afterwards.

    Packages any remaining data into a StreamPart and signals to the StreamTransferManager that there are no more parts afterwards. You cannot write to the stream after it has been closed.

    Definition Classes
    MultiPartOutputStream → OutputStream → Closeable → AutoCloseable
  8. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  9. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  10. def finalize(): Unit
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  11. def flush(): Unit
    Definition Classes
    OutputStream → Flushable
    Annotations
    @throws( classOf[java.io.IOException] )
  12. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  13. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  14. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  15. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  16. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  17. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  18. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  19. def toString(): String
    Definition Classes
    MultiPartOutputStream → AnyRef → Any
  20. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  21. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  22. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @throws( ... )
  23. def write(b: Array[Byte]): Unit
    Definition Classes
    MultiPartOutputStream → OutputStream
  24. def write(b: Array[Byte], off: Int, len: Int): Unit
    Definition Classes
    MultiPartOutputStream → OutputStream
  25. def write(b: Int): Unit
    Definition Classes
    MultiPartOutputStream → OutputStream

Inherited from OutputStream

Inherited from Flushable

Inherited from Closeable

Inherited from AutoCloseable

Inherited from AnyRef

Inherited from Any

Ungrouped