org.apache.hadoop.fs.s3a

Class DefaultS3ClientFactory

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.fs.s3a.DefaultS3ClientFactory

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, S3ClientFactory

Direct Known Subclasses:

InconsistentS3ClientFactory

public class DefaultS3ClientFactory
extends org.apache.hadoop.conf.Configured
implements S3ClientFactory

The default factory implementation, which calls the AWS SDK to configure and create an AmazonS3Client that communicates with the S3 service.

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor and Description
DefaultS3ClientFactory()

Method Summary

Modifier and Type	Method and Description
static com.amazonaws.ClientConfiguration	createAwsConf(org.apache.hadoop.conf.Configuration conf) Create a new ClientConfiguration.
com.amazonaws.services.s3.AmazonS3	createS3Client(URI name) Creates a new AmazonS3 client.
protected com.amazonaws.services.s3.AmazonS3	newAmazonS3Client(com.amazonaws.auth.AWSCredentialsProvider credentials, com.amazonaws.ClientConfiguration awsConf) Wrapper around constructor for AmazonS3 client.

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected static final org.slf4j.Logger LOG

Constructor Detail

DefaultS3ClientFactory

public DefaultS3ClientFactory()

Method Detail

createS3Client

public com.amazonaws.services.s3.AmazonS3 createS3Client(URI name)
                                                  throws IOException

Description copied from interface: S3ClientFactory

Creates a new AmazonS3 client.

Specified by:

createS3Client in interface S3ClientFactory

Parameters:

name - raw input S3A file system URI

Returns:

S3 client

Throws:

IOException - IO problem

createAwsConf

public static com.amazonaws.ClientConfiguration createAwsConf(org.apache.hadoop.conf.Configuration conf)

Create a new ClientConfiguration.

Parameters:

conf - The Hadoop configuration

Returns:

new AWS client configuration

newAmazonS3Client

protected com.amazonaws.services.s3.AmazonS3 newAmazonS3Client(com.amazonaws.auth.AWSCredentialsProvider credentials,
                                                               com.amazonaws.ClientConfiguration awsConf)

Wrapper around constructor for AmazonS3 client. Override this to provide an extended version of the client

Parameters:

credentials - credentials to use

awsConf - AWS configuration

Returns:

new AmazonS3 client