org.apache.flink.fs.s3hadoop

Class HadoopS3AccessHelper

java.lang.Object

org.apache.flink.fs.s3hadoop.HadoopS3AccessHelper

All Implemented Interfaces:

org.apache.flink.fs.s3.common.writer.S3AccessHelper

public class HadoopS3AccessHelper
extends Object
implements org.apache.flink.fs.s3.common.writer.S3AccessHelper

An implementation of the S3AccessHelper for the Hadoop S3A filesystem.

Constructor Summary

Constructors

Constructor and Description
HadoopS3AccessHelper(org.apache.hadoop.fs.s3a.S3AFileSystem s3a, org.apache.hadoop.conf.Configuration conf)

Method Summary

Modifier and Type	Method and Description
com.amazonaws.services.s3.model.CompleteMultipartUploadResult	commitMultiPartUpload(String destKey, String uploadId, List<com.amazonaws.services.s3.model.PartETag> partETags, long length, AtomicInteger errorCount)
boolean	deleteObject(String key)
long	getObject(String key, File targetLocation)
com.amazonaws.services.s3.model.ObjectMetadata	getObjectMetadata(String key)
com.amazonaws.services.s3.model.PutObjectResult	putObject(String key, File inputFile)
String	startMultiPartUpload(String key)
com.amazonaws.services.s3.model.UploadPartResult	uploadPart(String key, String uploadId, int partNumber, File inputFile, long length)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HadoopS3AccessHelper

public HadoopS3AccessHelper(org.apache.hadoop.fs.s3a.S3AFileSystem s3a,
                            org.apache.hadoop.conf.Configuration conf)

Method Detail

startMultiPartUpload

public String startMultiPartUpload(String key)
                            throws IOException

Specified by:

startMultiPartUpload in interface org.apache.flink.fs.s3.common.writer.S3AccessHelper

Throws:

IOException

uploadPart

public com.amazonaws.services.s3.model.UploadPartResult uploadPart(String key,
                                                                   String uploadId,
                                                                   int partNumber,
                                                                   File inputFile,
                                                                   long length)
                                                            throws IOException

Specified by:

uploadPart in interface org.apache.flink.fs.s3.common.writer.S3AccessHelper

Throws:

IOException

putObject

public com.amazonaws.services.s3.model.PutObjectResult putObject(String key,
                                                                 File inputFile)
                                                          throws IOException

Specified by:

putObject in interface org.apache.flink.fs.s3.common.writer.S3AccessHelper

Throws:

IOException

commitMultiPartUpload

public com.amazonaws.services.s3.model.CompleteMultipartUploadResult commitMultiPartUpload(String destKey,
                                                                                           String uploadId,
                                                                                           List<com.amazonaws.services.s3.model.PartETag> partETags,
                                                                                           long length,
                                                                                           AtomicInteger errorCount)
                                                                                    throws IOException

Specified by:

commitMultiPartUpload in interface org.apache.flink.fs.s3.common.writer.S3AccessHelper

Throws:

IOException

deleteObject

public boolean deleteObject(String key)
                     throws IOException

Specified by:

deleteObject in interface org.apache.flink.fs.s3.common.writer.S3AccessHelper

Throws:

IOException

getObject

public long getObject(String key,
                      File targetLocation)
               throws IOException

Specified by:

getObject in interface org.apache.flink.fs.s3.common.writer.S3AccessHelper

Throws:

IOException

getObjectMetadata

public com.amazonaws.services.s3.model.ObjectMetadata getObjectMetadata(String key)
                                                                 throws IOException

Specified by:

getObjectMetadata in interface org.apache.flink.fs.s3.common.writer.S3AccessHelper

Throws:

IOException