org.apache.flink.fs.s3hadoop

Class HadoopS3MultiPartUploader

java.lang.Object

org.apache.flink.fs.s3hadoop.HadoopS3MultiPartUploader

All Implemented Interfaces:

org.apache.flink.fs.s3.common.writer.S3MultiPartUploader

public class HadoopS3MultiPartUploader
extends Object
implements org.apache.flink.fs.s3.common.writer.S3MultiPartUploader

An implementation of the S3MultiPartUploader for the Hadoop S3A filesystem.

Constructor Summary

Constructors

Constructor and Description
HadoopS3MultiPartUploader(org.apache.hadoop.fs.s3a.S3AFileSystem s3a, org.apache.hadoop.conf.Configuration conf)

Method Summary

Modifier and Type	Method and Description
com.amazonaws.services.s3.model.CompleteMultipartUploadResult	commitMultiPartUpload(String destKey, String uploadId, List<com.amazonaws.services.s3.model.PartETag> partETags, long length, AtomicInteger errorCount)
com.amazonaws.services.s3.model.ObjectMetadata	getObjectMetadata(String key)
String	startMultiPartUpload(String key)
com.amazonaws.services.s3.model.PutObjectResult	uploadIncompletePart(String key, InputStream inputStream, long length)
com.amazonaws.services.s3.model.UploadPartResult	uploadPart(String key, String uploadId, int partNumber, InputStream inputStream, long length)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HadoopS3MultiPartUploader

public HadoopS3MultiPartUploader(org.apache.hadoop.fs.s3a.S3AFileSystem s3a,
                                 org.apache.hadoop.conf.Configuration conf)

Method Detail

startMultiPartUpload

public String startMultiPartUpload(String key)
                            throws IOException

Specified by:

startMultiPartUpload in interface org.apache.flink.fs.s3.common.writer.S3MultiPartUploader

Throws:

IOException

uploadPart

public com.amazonaws.services.s3.model.UploadPartResult uploadPart(String key,
                                                                   String uploadId,
                                                                   int partNumber,
                                                                   InputStream inputStream,
                                                                   long length)
                                                            throws IOException

Specified by:

uploadPart in interface org.apache.flink.fs.s3.common.writer.S3MultiPartUploader

Throws:

IOException

uploadIncompletePart

public com.amazonaws.services.s3.model.PutObjectResult uploadIncompletePart(String key,
                                                                            InputStream inputStream,
                                                                            long length)
                                                                     throws IOException

Specified by:

uploadIncompletePart in interface org.apache.flink.fs.s3.common.writer.S3MultiPartUploader

Throws:

IOException

commitMultiPartUpload

public com.amazonaws.services.s3.model.CompleteMultipartUploadResult commitMultiPartUpload(String destKey,
                                                                                           String uploadId,
                                                                                           List<com.amazonaws.services.s3.model.PartETag> partETags,
                                                                                           long length,
                                                                                           AtomicInteger errorCount)
                                                                                    throws IOException

Specified by:

commitMultiPartUpload in interface org.apache.flink.fs.s3.common.writer.S3MultiPartUploader

Throws:

IOException

getObjectMetadata

public com.amazonaws.services.s3.model.ObjectMetadata getObjectMetadata(String key)
                                                                 throws IOException

Specified by:

getObjectMetadata in interface org.apache.flink.fs.s3.common.writer.S3MultiPartUploader

Throws:

IOException