Skip navigation links
A B C D E F G H I K M N O P Q R S T U V W 

A

addBatchWriteRequest(long, boolean) - Method in interface org.apache.beam.sdk.io.aws2.common.AsyncBatchWriteHandler.Stats
 
algorithm(String) - Method in class org.apache.beam.sdk.io.aws2.s3.SSECustomerKey.Builder
 
apply(String, T) - Method in interface org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches.EntryMapperFn.Builder
 
AsyncBatchWriteHandler<RecT,ResT> - Class in org.apache.beam.sdk.io.aws2.common
Async handler that automatically retries unprocessed records in case of a partial success.
AsyncBatchWriteHandler(int, FluentBackoff, AsyncBatchWriteHandler.Stats, Function<ResT, String>, BiFunction<String, List<RecT>, CompletableFuture<List<ResT>>>) - Constructor for class org.apache.beam.sdk.io.aws2.common.AsyncBatchWriteHandler
 
AsyncBatchWriteHandler.Stats - Interface in org.apache.beam.sdk.io.aws2.common
Statistics on the batch request.
AttributeValueCoder - Class in org.apache.beam.sdk.io.aws2.dynamodb
A Coder that serializes and deserializes the AttributeValue objects.
AwsBuilderFactory<PojoT extends software.amazon.awssdk.core.SdkPojo,BuilderT extends software.amazon.awssdk.utils.builder.SdkBuilder<BuilderT,PojoT> & software.amazon.awssdk.core.SdkPojo> - Class in org.apache.beam.sdk.io.aws2.schemas
Builder factory for AWS SdkPojo to avoid using reflection to instantiate a builder.
AwsBuilderFactory() - Constructor for class org.apache.beam.sdk.io.aws2.schemas.AwsBuilderFactory
 
AwsModule - Class in org.apache.beam.sdk.io.aws2.options
A Jackson Module that registers a JsonSerializer and JsonDeserializer for AwsCredentialsProvider and some subclasses.
AwsModule() - Constructor for class org.apache.beam.sdk.io.aws2.options.AwsModule
 
AwsOptions - Interface in org.apache.beam.sdk.io.aws2.options
Options used to configure Amazon Web Services specific options such as credentials and region.
AwsOptions.AwsRegionFactory - Class in org.apache.beam.sdk.io.aws2.options
Attempt to load default region.
AwsOptions.AwsUserCredentialsFactory - Class in org.apache.beam.sdk.io.aws2.options
Return DefaultCredentialsProvider as default provider.
AwsPipelineOptionsRegistrar - Class in org.apache.beam.sdk.io.aws2.options
A registrar containing the default AWS options.
AwsPipelineOptionsRegistrar() - Constructor for class org.apache.beam.sdk.io.aws2.options.AwsPipelineOptionsRegistrar
 
AwsRegionFactory() - Constructor for class org.apache.beam.sdk.io.aws2.options.AwsOptions.AwsRegionFactory
 
AwsSchemaProvider - Class in org.apache.beam.sdk.io.aws2.schemas
Schema provider for AWS SdkPojo models using the provided field metadata (@see SdkPojo.sdkFields()) rather than reflection.
AwsSchemaProvider() - Constructor for class org.apache.beam.sdk.io.aws2.schemas.AwsSchemaProvider
 
AwsSchemaRegistrar - Class in org.apache.beam.sdk.io.aws2.schemas
 
AwsSchemaRegistrar() - Constructor for class org.apache.beam.sdk.io.aws2.schemas.AwsSchemaRegistrar
 
AwsSerializableUtils - Class in org.apache.beam.sdk.io.aws2.options
Utilities for working with AWS Serializables.
AwsSerializableUtils() - Constructor for class org.apache.beam.sdk.io.aws2.options.AwsSerializableUtils
 
AwsTypes - Class in org.apache.beam.sdk.io.aws2.schemas
 
AwsTypes() - Constructor for class org.apache.beam.sdk.io.aws2.schemas.AwsTypes
 
AwsUserCredentialsFactory() - Constructor for class org.apache.beam.sdk.io.aws2.options.AwsOptions.AwsUserCredentialsFactory
 

B

baseBackoff() - Method in class org.apache.beam.sdk.io.aws2.common.RetryConfiguration
 
baseBackoff(Duration) - Method in class org.apache.beam.sdk.io.aws2.common.RetryConfiguration.Builder
 
batchWrite(String, List<RecT>) - Method in class org.apache.beam.sdk.io.aws2.common.AsyncBatchWriteHandler
Asynchronously trigger a batch write request (unless already in error state).
batchWrite(String, List<RecT>, boolean) - Method in class org.apache.beam.sdk.io.aws2.common.AsyncBatchWriteHandler
Asynchronously trigger a batch write request (unless already in error state).
build() - Method in class org.apache.beam.sdk.io.aws2.common.ClientConfiguration.Builder
 
build() - Method in class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration.Builder
 
build() - Method in class org.apache.beam.sdk.io.aws2.common.RetryConfiguration.Builder
 
build() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.RecordAggregation.Builder
 
build() - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration.Builder
 
build() - Method in class org.apache.beam.sdk.io.aws2.s3.SSECustomerKey.Builder
 
buildClient(AwsOptions, BuilderT, ClientConfiguration) - Static method in interface org.apache.beam.sdk.io.aws2.common.ClientBuilderFactory
Utility to directly build a client of type ClientT using builder of BuilderT.
builder() - Static method in class org.apache.beam.sdk.io.aws2.common.ClientConfiguration
 
Builder() - Constructor for class org.apache.beam.sdk.io.aws2.common.ClientConfiguration.Builder
 
builder() - Static method in class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration
 
Builder() - Constructor for class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration.Builder
 
builder() - Static method in class org.apache.beam.sdk.io.aws2.common.RetryConfiguration
 
Builder() - Constructor for class org.apache.beam.sdk.io.aws2.common.RetryConfiguration.Builder
 
builder() - Static method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.RecordAggregation
 
Builder() - Constructor for class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.RecordAggregation.Builder
 
builder() - Static method in class org.apache.beam.sdk.io.aws2.kinesis.WatermarkParameters
 
builder() - Static method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
Creates a new uninitialized S3FileSystemConfiguration.Builder.
Builder() - Constructor for class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration.Builder
 
builder() - Static method in class org.apache.beam.sdk.io.aws2.s3.SSECustomerKey
 
builderFrom(S3Options) - Static method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
Creates a new S3FileSystemConfiguration.Builder with values initialized by the properties of s3Options.
byId(int, int, RetryConfiguration, AsyncBatchWriteHandler.Stats, BiFunction<String, List<RecT>, CompletableFuture<List<ErrT>>>, Function<ErrT, String>, Function<RecT, String>, Function<ErrT, String>) - Static method in class org.apache.beam.sdk.io.aws2.common.AsyncBatchWriteHandler
AsyncBatchWriteHandler that correlates records and results by id, all results are erroneous.
byId(int, FluentBackoff, AsyncBatchWriteHandler.Stats, BiFunction<String, List<RecT>, CompletableFuture<List<ErrT>>>, Function<ErrT, String>, Function<RecT, String>, Function<ErrT, String>) - Static method in class org.apache.beam.sdk.io.aws2.common.AsyncBatchWriteHandler
AsyncBatchWriteHandler that correlates records and results by id, all results are erroneous.
byPosition(int, int, RetryConfiguration, AsyncBatchWriteHandler.Stats, BiFunction<String, List<RecT>, CompletableFuture<List<ResT>>>, Function<ResT, String>) - Static method in class org.apache.beam.sdk.io.aws2.common.AsyncBatchWriteHandler
AsyncBatchWriteHandler that correlates records and results by position in the respective list.
byPosition(int, FluentBackoff, AsyncBatchWriteHandler.Stats, BiFunction<String, List<RecT>, CompletableFuture<List<ResT>>>, Function<ResT, String>) - Static method in class org.apache.beam.sdk.io.aws2.common.AsyncBatchWriteHandler
AsyncBatchWriteHandler that correlates records and results by position in the respective list.

C

checkConfiguration(ClientConfiguration, AwsOptions) - Method in interface org.apache.beam.sdk.io.aws2.common.ClientBuilderFactory
Check if all necessary configuration is available to create clients.
checkConfiguration(ClientConfiguration, AwsOptions) - Method in class org.apache.beam.sdk.io.aws2.common.ClientBuilderFactory.DefaultClientBuilder
 
checkForAsyncFailure() - Method in class org.apache.beam.sdk.io.aws2.common.AsyncBatchWriteHandler
Check if any failure happened async.
ClientBuilderFactory - Interface in org.apache.beam.sdk.io.aws2.common
Factory to build and configure any AwsClientBuilder using a specific ClientConfiguration or the globally provided settings in AwsOptions as fallback.
ClientBuilderFactory.DefaultClientBuilder - Class in org.apache.beam.sdk.io.aws2.common
Default implementation of ClientBuilderFactory.
ClientConfiguration - Class in org.apache.beam.sdk.io.aws2.common
AWS client configuration.
ClientConfiguration() - Constructor for class org.apache.beam.sdk.io.aws2.common.ClientConfiguration
 
ClientConfiguration.Builder - Class in org.apache.beam.sdk.io.aws2.common
 
connectionAcquisitionTimeout(Integer) - Method in class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration.Builder
Milliseconds to wait when acquiring a connection from the pool before giving up and timing out.
connectionAcquisitionTimeout() - Method in class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration
Milliseconds to wait when acquiring a connection from the pool before giving up and timing out.
connectionMaxIdleTime(Integer) - Method in class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration.Builder
Maximum milliseconds a connection should be allowed to remain open while idle.
connectionMaxIdleTime() - Method in class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration
Maximum milliseconds a connection should be allowed to remain open while idle.
connectionTimeout(Integer) - Method in class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration.Builder
Milliseconds to wait when initially establishing a connection before giving up and timing out.
connectionTimeout() - Method in class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration
Milliseconds to wait when initially establishing a connection before giving up and timing out.
connectionTimeToLive(Integer) - Method in class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration.Builder
Maximum milliseconds a connection should be allowed to remain open, regardless of usage frequency.
connectionTimeToLive() - Method in class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration
Maximum milliseconds a connection should be allowed to remain open, regardless of usage frequency.
create(BuilderT, AwsOptions) - Method in interface org.apache.beam.sdk.io.aws2.common.ClientBuilderFactory
Configure a client builder BuilderT using the global defaults in AwsOptions.
create(BuilderT, ClientConfiguration, AwsOptions) - Method in interface org.apache.beam.sdk.io.aws2.common.ClientBuilderFactory
Configure a client builder BuilderT using the provided ClientConfiguration and fall back to the global defaults in AwsOptions where necessary.
create(BuilderT, ClientConfiguration, AwsOptions) - Method in class org.apache.beam.sdk.io.aws2.common.ClientBuilderFactory.DefaultClientBuilder
 
create(AwsCredentialsProvider, Region, URI) - Static method in class org.apache.beam.sdk.io.aws2.common.ClientConfiguration
 
create(PipelineOptions) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIOOptions.MapFactory
 
create() - Static method in class org.apache.beam.sdk.io.aws2.kinesis.WatermarkParameters
 
create(PipelineOptions) - Method in class org.apache.beam.sdk.io.aws2.options.AwsOptions.AwsRegionFactory
 
create(PipelineOptions) - Method in class org.apache.beam.sdk.io.aws2.options.AwsOptions.AwsUserCredentialsFactory
 
create(PipelineOptions) - Method in class org.apache.beam.sdk.io.aws2.options.S3Options.S3UploadBufferSizeBytesFactory
 
create(PipelineOptions) - Method in class org.apache.beam.sdk.io.aws2.options.S3Options.SSECustomerKeyFactory
 
create(String, String, String, long, long) - Static method in class org.apache.beam.sdk.io.aws2.sqs.SqsMessage
 
createBuilder(S3Options) - Method in interface org.apache.beam.sdk.io.aws2.options.S3ClientBuilderFactory
 
createBuilder(S3Options) - Method in class org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory
 
createWatermarkPolicy() - Method in interface org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicyFactory
 
credentialsProvider(AwsCredentialsProvider) - Method in class org.apache.beam.sdk.io.aws2.common.ClientConfiguration.Builder
Optional AwsCredentialsProvider.
credentialsProvider() - Method in class org.apache.beam.sdk.io.aws2.common.ClientConfiguration
Optional AwsCredentialsProvider.

D

decode(InputStream) - Method in class org.apache.beam.sdk.io.aws2.dynamodb.AttributeValueCoder
 
DEFAULT_UPLOAD_BUFFER_SIZE_BYTES - Static variable in class org.apache.beam.sdk.io.aws2.options.S3Options.S3UploadBufferSizeBytesFactory
 
defaultFactory() - Static method in interface org.apache.beam.sdk.io.aws2.common.ClientBuilderFactory
The default ClientBuilderFactory instance.
DefaultRateLimiter(BackOff, BackOff) - Constructor for class org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory.DefaultRateLimiter
 
DefaultRateLimiter(Duration, Duration, Duration) - Constructor for class org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory.DefaultRateLimiter
 
DefaultS3ClientBuilderFactory - Class in org.apache.beam.sdk.io.aws2.s3
Construct S3ClientBuilder with default values of S3 client properties like path style access, accelerated mode, etc.
DefaultS3ClientBuilderFactory() - Constructor for class org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory
 
DefaultS3FileSystemSchemeRegistrar - Class in org.apache.beam.sdk.io.aws2.s3
Registers the "s3" uri schema to be handled by S3FileSystem.
DefaultS3FileSystemSchemeRegistrar() - Constructor for class org.apache.beam.sdk.io.aws2.s3.DefaultS3FileSystemSchemeRegistrar
 
DelayIntervalRateLimiter() - Constructor for class org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory.DelayIntervalRateLimiter
 
DelayIntervalRateLimiter(Supplier<Duration>) - Constructor for class org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory.DelayIntervalRateLimiter
 
deserializeAwsCredentialsProvider(String) - Static method in class org.apache.beam.sdk.io.aws2.options.AwsSerializableUtils
 
DynamoDBIO - Class in org.apache.beam.sdk.io.aws2.dynamodb
IO to read from and write to DynamoDB tables.
DynamoDBIO() - Constructor for class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO
 
DynamoDBIO.Read<T> - Class in org.apache.beam.sdk.io.aws2.dynamodb
Read data from DynamoDB using DynamoDBIO.Read.getScanRequestFn() and emit an element of type T for each ScanResponse using the mapping function DynamoDBIO.Read.getScanResponseMapperFn().
DynamoDBIO.Write<T> - Class in org.apache.beam.sdk.io.aws2.dynamodb
Write a PCollection data into DynamoDB.

E

EMPTY - Static variable in class org.apache.beam.sdk.io.aws2.common.ClientConfiguration
 
encode(AttributeValue, OutputStream) - Method in class org.apache.beam.sdk.io.aws2.dynamodb.AttributeValueCoder
 
endpoint(URI) - Method in class org.apache.beam.sdk.io.aws2.common.ClientConfiguration.Builder
Optional service endpoint to use AWS compatible services instead, e.g.
endpoint() - Method in class org.apache.beam.sdk.io.aws2.common.ClientConfiguration
Optional service endpoint to use AWS compatible services instead, e.g.
equals(Object) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
 
errorCodeFn - Variable in class org.apache.beam.sdk.io.aws2.common.AsyncBatchWriteHandler
 
expand(PBegin) - Method in class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO.Read
 
expand(PCollection<T>) - Method in class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO.Write
 
expand(PBegin) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
 
expand(PCollection<T>) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
 
expand() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write.Result
 
expand(PCollection<T>) - Method in class org.apache.beam.sdk.io.aws2.sns.SnsIO.Write
 
expand(PBegin) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.Read
 
expand(PCollection<SendMessageRequest>) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.Write
Deprecated.
 
expand(PCollection<T>) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches
 
expand() - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches.Result
 
explicitRandomPartitioner(int) - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.KinesisPartitioner
Explicit hash key partitioner that randomly returns one of x precalculated hash keys.

F

failedRecords(List<RecT>, List<ResT>) - Method in class org.apache.beam.sdk.io.aws2.common.AsyncBatchWriteHandler
 
fieldValueGetters(Class<?>, Schema) - Method in class org.apache.beam.sdk.io.aws2.schemas.AwsSchemaProvider
 
fieldValueTypeInformations(Class<?>, Schema) - Method in class org.apache.beam.sdk.io.aws2.schemas.AwsSchemaProvider
 
finishSpecifyingOutput(String, PInput, PTransform<?, ?>) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write.Result
 
finishSpecifyingOutput(String, PInput, PTransform<?, ?>) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches.Result
 
fromOptions(PipelineOptions) - Method in class org.apache.beam.sdk.io.aws2.s3.DefaultS3FileSystemSchemeRegistrar
 
fromOptions(PipelineOptions) - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemRegistrar
 
fromOptions(PipelineOptions) - Method in interface org.apache.beam.sdk.io.aws2.s3.S3FileSystemSchemeRegistrar
Create zero or more S3FileSystemConfiguration instances from the given PipelineOptions.
fromRowFunction(TypeDescriptor<T>) - Method in class org.apache.beam.sdk.io.aws2.schemas.AwsSchemaProvider
 
fromS3Options(S3Options) - Static method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
 

G

get() - Method in class org.apache.beam.sdk.io.aws2.schemas.AwsBuilderFactory
 
getAlgorithm() - Method in class org.apache.beam.sdk.io.aws2.s3.SSECustomerKey
 
getApproximateArrivalTimestamp() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
 
getAwsCredentialsProvider() - Method in interface org.apache.beam.sdk.io.aws2.options.AwsOptions
AwsCredentialsProvider used to configure AWS service clients.
getAwsRegion() - Method in interface org.apache.beam.sdk.io.aws2.options.AwsOptions
Region used to configure AWS service clients.
getBody() - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsMessage
Message body.
getBucketKeyEnabled() - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
 
getBucketKeyEnabled() - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
Whether to use an S3 Bucket Key for object encryption with server-side encryption using AWS KMS (SSE-KMS) or not.
getClientBuilderFactory() - Method in interface org.apache.beam.sdk.io.aws2.options.AwsOptions
 
getData() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
 
getDataAsBytes() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
 
getEndpoint() - Method in interface org.apache.beam.sdk.io.aws2.options.AwsOptions
Endpoint used to configure AWS service clients.
getExplicitHashKey(T) - Method in interface org.apache.beam.sdk.io.aws2.kinesis.KinesisPartitioner.ExplicitPartitioner
Required hash value (128-bit integer) to determine explicitly the shard a record is assigned to based on the hash key range of each shard.
getExplicitHashKey(T) - Method in interface org.apache.beam.sdk.io.aws2.kinesis.KinesisPartitioner
Optional hash value (128-bit integer) to determine explicitly the shard a record is assigned to based on the hash key range of each shard.
getExtendedSequenceNumber() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
 
getFactory(AwsOptions) - Static method in interface org.apache.beam.sdk.io.aws2.common.ClientBuilderFactory
getHttpClientConfiguration() - Method in interface org.apache.beam.sdk.io.aws2.options.AwsOptions
HttpClientConfiguration used to configure AWS service clients.
getKey() - Method in class org.apache.beam.sdk.io.aws2.s3.SSECustomerKey
 
getKinesisIOConsumerArns() - Method in interface org.apache.beam.sdk.io.aws2.kinesis.KinesisIOOptions
Used to enable / disable EFO.
getMD5() - Method in class org.apache.beam.sdk.io.aws2.s3.SSECustomerKey
 
getMessageId() - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsMessage
SQS message id.
getPartitionKey(T) - Method in interface org.apache.beam.sdk.io.aws2.kinesis.KinesisPartitioner.ExplicitPartitioner
 
getPartitionKey(T) - Method in interface org.apache.beam.sdk.io.aws2.kinesis.KinesisPartitioner
Determines which shard in the stream the record is assigned to.
getPartitionKey() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
 
getPipeline() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write.Result
 
getPipeline() - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches.Result
 
getPipelineOptions() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIOOptions.KinesisIOOptionsRegistrar
 
getPipelineOptions() - Method in class org.apache.beam.sdk.io.aws2.options.AwsPipelineOptionsRegistrar
 
getProxyConfiguration() - Method in interface org.apache.beam.sdk.io.aws2.options.AwsOptions
ProxyConfiguration used to configure AWS service clients.
getRateLimitPolicy() - Method in interface org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory
 
getReadTime() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
 
getReceiptHandle() - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsMessage
SQS receipt handle.
getRequestTimeStamp() - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsMessage
Timestamp the message was received at (in epoch millis).
getS3ClientBuilder() - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
Builder used to create the S3Client.
getS3ClientFactoryClass() - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
 
getS3StorageClass() - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
 
getS3StorageClass() - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
The AWS S3 storage class used for creating S3 objects.
getS3ThreadPoolSize() - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
 
getS3ThreadPoolSize() - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
Thread pool size, limiting the max concurrent S3 operations.
getS3UploadBufferSizeBytes() - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
 
getS3UploadBufferSizeBytes() - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
Size of S3 upload chnks.
getSchemaProviders() - Method in class org.apache.beam.sdk.io.aws2.schemas.AwsSchemaRegistrar
 
getScheme() - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
The uri scheme used by resources on this filesystem.
getSequenceNumber() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
 
getShardId() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
 
getSSEAlgorithm() - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
 
getSSEAlgorithm() - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
Algorithm for SSE-S3 encryption, e.g.
getSSECustomerKey() - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
 
getSSECustomerKey() - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
SSE key for SSE-C encryption, e.g.
getSSEKMSKeyId() - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
 
getSSEKMSKeyId() - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
KMS key id for SSE-KMS encyrption, e.g.
getStreamName() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
 
getSubSequenceNumber() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
 
getTimeStamp() - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsMessage
Timestamp the message was sent at (in epoch millis).
getUniqueId() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
 
getWatermark() - Method in interface org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicy
 
getWatermark() - Method in class org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicyFactory.ArrivalTimeWatermarkPolicy
 
getWatermark() - Method in class org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicyFactory.CustomWatermarkPolicy
 
getWatermark() - Method in class org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicyFactory.ProcessingTimeWatermarkPolicy
 

H

hasErrored() - Method in class org.apache.beam.sdk.io.aws2.common.AsyncBatchWriteHandler
If this handler has errored since it was last reset.
hasFailedRecords(List<ResT>) - Method in class org.apache.beam.sdk.io.aws2.common.AsyncBatchWriteHandler
 
hashCode() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
 
HttpClientConfiguration - Class in org.apache.beam.sdk.io.aws2.common
HTTP client configuration for both, sync and async AWS clients.
HttpClientConfiguration() - Constructor for class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration
 
HttpClientConfiguration.Builder - Class in org.apache.beam.sdk.io.aws2.common
 

I

items() - Method in class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO.Read
 

K

key(String) - Method in class org.apache.beam.sdk.io.aws2.s3.SSECustomerKey.Builder
 
KinesisClientThrottledException - Exception in org.apache.beam.sdk.io.aws2.kinesis
Thrown when the Kinesis client was throttled due to rate limits.
KinesisClientThrottledException(String, KinesisException) - Constructor for exception org.apache.beam.sdk.io.aws2.kinesis.KinesisClientThrottledException
 
KinesisIO - Class in org.apache.beam.sdk.io.aws2.kinesis
IO to read from Kinesis streams.
KinesisIO() - Constructor for class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO
 
KinesisIO.Read - Class in org.apache.beam.sdk.io.aws2.kinesis
Implementation of KinesisIO.read().
KinesisIO.RecordAggregation - Class in org.apache.beam.sdk.io.aws2.kinesis
Configuration of Kinesis record aggregation.
KinesisIO.RecordAggregation.Builder - Class in org.apache.beam.sdk.io.aws2.kinesis
 
KinesisIO.Write<T> - Class in org.apache.beam.sdk.io.aws2.kinesis
Implementation of KinesisIO.write().
KinesisIO.Write.Result - Class in org.apache.beam.sdk.io.aws2.kinesis
KinesisIOOptions - Interface in org.apache.beam.sdk.io.aws2.kinesis
PipelineOptions for KinesisIO.
KinesisIOOptions.KinesisIOOptionsRegistrar - Class in org.apache.beam.sdk.io.aws2.kinesis
A registrar containing the default KinesisIOOptions.
KinesisIOOptions.MapFactory - Class in org.apache.beam.sdk.io.aws2.kinesis
 
KinesisIOOptionsRegistrar() - Constructor for class org.apache.beam.sdk.io.aws2.kinesis.KinesisIOOptions.KinesisIOOptionsRegistrar
 
KinesisPartitioner<T> - Interface in org.apache.beam.sdk.io.aws2.kinesis
Kinesis interface for custom partitioner.
KinesisPartitioner.ExplicitPartitioner<T> - Interface in org.apache.beam.sdk.io.aws2.kinesis
An explicit partitioner that always returns a Nonnull explicit hash key.
KinesisRecord - Class in org.apache.beam.sdk.io.aws2.kinesis
KinesisClientRecord enhanced with utility methods.
KinesisRecord(KinesisClientRecord, String, String) - Constructor for class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
 
KinesisRecord(ByteBuffer, String, long, String, Instant, Instant, String, String) - Constructor for class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
 

M

MapFactory() - Constructor for class org.apache.beam.sdk.io.aws2.kinesis.KinesisIOOptions.MapFactory
 
MAX_HASH_KEY - Static variable in interface org.apache.beam.sdk.io.aws2.kinesis.KinesisPartitioner
 
maxBackoff(Duration) - Method in class org.apache.beam.sdk.io.aws2.common.RetryConfiguration.Builder
 
maxBackoff() - Method in class org.apache.beam.sdk.io.aws2.common.RetryConfiguration
 
maxBufferedTime(Duration) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.RecordAggregation.Builder
Buffer timeout for user records.
maxBytes(int) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.RecordAggregation.Builder
Max bytes per aggregated record.
maxConnections(Integer) - Method in class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration.Builder
The maximum number of connections allowed in the connection pool.
maxConnections() - Method in class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration
The maximum number of connections allowed in the connection pool.
md5(String) - Method in class org.apache.beam.sdk.io.aws2.s3.SSECustomerKey.Builder
 
MIN_HASH_KEY - Static variable in interface org.apache.beam.sdk.io.aws2.kinesis.KinesisPartitioner
 
MINIMUM_UPLOAD_BUFFER_SIZE_BYTES - Static variable in class org.apache.beam.sdk.io.aws2.options.S3Options.S3UploadBufferSizeBytesFactory
 
MINIMUM_UPLOAD_BUFFER_SIZE_BYTES - Static variable in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
 

N

NONE - Static variable in interface org.apache.beam.sdk.io.aws2.common.AsyncBatchWriteHandler.Stats
 
numRetries(int) - Method in class org.apache.beam.sdk.io.aws2.common.RetryConfiguration.Builder
 
numRetries() - Method in class org.apache.beam.sdk.io.aws2.common.RetryConfiguration
 

O

ObjectPool<KeyT,ObjectT> - Class in org.apache.beam.sdk.io.aws2.common
Reference counting object pool to easily share & destroy objects.
ObjectPool(Function<KeyT, ObjectT>) - Constructor for class org.apache.beam.sdk.io.aws2.common.ObjectPool
 
ObjectPool(Function<KeyT, ObjectT>, ThrowingConsumer<Exception, ObjectT>) - Constructor for class org.apache.beam.sdk.io.aws2.common.ObjectPool
 
ObjectPool.ClientPool<ClientT extends software.amazon.awssdk.core.SdkClient> - Class in org.apache.beam.sdk.io.aws2.common
Client pool to easily share AWS clients per configuration.
of() - Static method in class org.apache.beam.sdk.io.aws2.dynamodb.AttributeValueCoder
 
onSuccess(List<KinesisRecord>) - Method in interface org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicy
Called after Kinesis records are successfully retrieved.
onSuccess(List<KinesisRecord>) - Method in class org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory.DefaultRateLimiter
 
onSuccess(List<KinesisRecord>) - Method in class org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory.DelayIntervalRateLimiter
 
onThrottle(KinesisClientThrottledException) - Method in interface org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicy
Called after the Kinesis client is throttled.
onThrottle(KinesisClientThrottledException) - Method in class org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory.DefaultRateLimiter
 
org.apache.beam.sdk.io.aws2.common - package org.apache.beam.sdk.io.aws2.common
Common code for AWS sources and sinks such as retry configuration.
org.apache.beam.sdk.io.aws2.dynamodb - package org.apache.beam.sdk.io.aws2.dynamodb
Defines IO connectors for Amazon Web Services DynamoDB.
org.apache.beam.sdk.io.aws2.kinesis - package org.apache.beam.sdk.io.aws2.kinesis
Transforms for reading from Amazon Kinesis.
org.apache.beam.sdk.io.aws2.options - package org.apache.beam.sdk.io.aws2.options
Defines PipelineOptions for configuring pipeline execution for Amazon Web Services components.
org.apache.beam.sdk.io.aws2.s3 - package org.apache.beam.sdk.io.aws2.s3
Defines IO connectors for Amazon Web Services S3.
org.apache.beam.sdk.io.aws2.schemas - package org.apache.beam.sdk.io.aws2.schemas
Schemas for AWS model classes.
org.apache.beam.sdk.io.aws2.sns - package org.apache.beam.sdk.io.aws2.sns
Defines IO connectors for Amazon Web Services SNS.
org.apache.beam.sdk.io.aws2.sqs - package org.apache.beam.sdk.io.aws2.sqs
Defines IO connectors for Amazon Web Services SQS.

P

pooledClientFactory(BuilderT) - Static method in class org.apache.beam.sdk.io.aws2.common.ObjectPool
 
ProcessingTimeWatermarkPolicy() - Constructor for class org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicyFactory.ProcessingTimeWatermarkPolicy
 

Q

queueUrl(T) - Method in interface org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches.DynamicDestination
 

R

RateLimitPolicy - Interface in org.apache.beam.sdk.io.aws2.kinesis
 
RateLimitPolicyFactory - Interface in org.apache.beam.sdk.io.aws2.kinesis
Implement this interface to create a RateLimitPolicy.
RateLimitPolicyFactory.DefaultRateLimiter - Class in org.apache.beam.sdk.io.aws2.kinesis
Default rate limiter that throttles reading from a shard using an exponential backoff if the response is empty or if the consumer is throttled by AWS.
RateLimitPolicyFactory.DelayIntervalRateLimiter - Class in org.apache.beam.sdk.io.aws2.kinesis
 
read() - Static method in class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO
 
Read() - Constructor for class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO.Read
 
read() - Static method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO
Returns a new KinesisIO.Read transform for reading from Kinesis.
Read() - Constructor for class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
 
read() - Static method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO
 
Read() - Constructor for class org.apache.beam.sdk.io.aws2.sqs.SqsIO.Read
 
readTimeout(Integer) - Method in class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration.Builder
Milliseconds to wait for a read on a socket before an exception is thrown.
readTimeout() - Method in class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration
Milliseconds to wait for a read on a socket before an exception is thrown.
RecordAggregation() - Constructor for class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.RecordAggregation
 
region(Region) - Method in class org.apache.beam.sdk.io.aws2.common.ClientConfiguration.Builder
Optional Region.
region() - Method in class org.apache.beam.sdk.io.aws2.common.ClientConfiguration
Optional Region.
release(ObjectT) - Method in class org.apache.beam.sdk.io.aws2.common.ObjectPool
Release a reference to a shared client instance.
releaseByKey(KeyT) - Method in class org.apache.beam.sdk.io.aws2.common.ObjectPool
Release a reference to a shared object instance using KeyT.
requestsInProgress() - Method in class org.apache.beam.sdk.io.aws2.common.AsyncBatchWriteHandler
 
REQUIRED_MEMORY_FOR_DEFAULT_BUFFER_BYTES - Static variable in class org.apache.beam.sdk.io.aws2.options.S3Options.S3UploadBufferSizeBytesFactory
 
reset() - Method in class org.apache.beam.sdk.io.aws2.common.AsyncBatchWriteHandler
 
retain(AwsOptions, ClientConfiguration) - Method in class org.apache.beam.sdk.io.aws2.common.ObjectPool.ClientPool
Retain a reference to a shared client instance.
retain(KeyT) - Method in class org.apache.beam.sdk.io.aws2.common.ObjectPool
Retain a reference to a shared client instance.
retry(RetryConfiguration) - Method in class org.apache.beam.sdk.io.aws2.common.ClientConfiguration.Builder
Optional RetryConfiguration for AWS clients.
retry(Consumer<RetryConfiguration.Builder>) - Method in class org.apache.beam.sdk.io.aws2.common.ClientConfiguration.Builder
Optional RetryConfiguration for AWS clients.
retry() - Method in class org.apache.beam.sdk.io.aws2.common.ClientConfiguration
Optional RetryConfiguration for AWS clients.
RetryConfiguration - Class in org.apache.beam.sdk.io.aws2.common
Configuration of the retry behavior for AWS SDK clients.
RetryConfiguration() - Constructor for class org.apache.beam.sdk.io.aws2.common.RetryConfiguration
 
RetryConfiguration.Builder - Class in org.apache.beam.sdk.io.aws2.common
 

S

S3ClientBuilderFactory - Interface in org.apache.beam.sdk.io.aws2.options
Construct S3ClientBuilder from S3 pipeline options.
S3FileSystemConfiguration - Class in org.apache.beam.sdk.io.aws2.s3
Object used to configure S3FileSystem.
S3FileSystemConfiguration() - Constructor for class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
 
S3FileSystemConfiguration.Builder - Class in org.apache.beam.sdk.io.aws2.s3
 
S3FileSystemRegistrar - Class in org.apache.beam.sdk.io.aws2.s3
AutoService registrar for the S3FileSystem.
S3FileSystemRegistrar() - Constructor for class org.apache.beam.sdk.io.aws2.s3.S3FileSystemRegistrar
 
S3FileSystemSchemeRegistrar - Interface in org.apache.beam.sdk.io.aws2.s3
A registrar that creates S3FileSystemConfiguration instances from PipelineOptions.
S3Options - Interface in org.apache.beam.sdk.io.aws2.options
Options used to configure Amazon Web Services S3.
S3Options.S3UploadBufferSizeBytesFactory - Class in org.apache.beam.sdk.io.aws2.options
Provide the default s3 upload buffer size in bytes: 64MB if more than 512MB in RAM are available and 5MB otherwise.
S3Options.SSECustomerKeyFactory - Class in org.apache.beam.sdk.io.aws2.options
 
S3UploadBufferSizeBytesFactory() - Constructor for class org.apache.beam.sdk.io.aws2.options.S3Options.S3UploadBufferSizeBytesFactory
 
schemaFor(TypeDescriptor<T>) - Method in class org.apache.beam.sdk.io.aws2.schemas.AwsSchemaProvider
 
schemaTypeCreator(Class<?>, Schema) - Method in class org.apache.beam.sdk.io.aws2.schemas.AwsSchemaProvider
 
sdkFields() - Method in class org.apache.beam.sdk.io.aws2.schemas.AwsBuilderFactory
 
serializeAwsCredentialsProvider(AwsCredentialsProvider) - Static method in class org.apache.beam.sdk.io.aws2.options.AwsSerializableUtils
 
setAwsCredentialsProvider(AwsCredentialsProvider) - Method in interface org.apache.beam.sdk.io.aws2.options.AwsOptions
 
setAwsRegion(Region) - Method in interface org.apache.beam.sdk.io.aws2.options.AwsOptions
 
setBucketKeyEnabled(boolean) - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
 
setBucketKeyEnabled(boolean) - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration.Builder
 
setClientBuilderFactory(Class<? extends ClientBuilderFactory>) - Method in interface org.apache.beam.sdk.io.aws2.options.AwsOptions
 
setEndpoint(URI) - Method in interface org.apache.beam.sdk.io.aws2.options.AwsOptions
 
setHttpClientConfiguration(HttpClientConfiguration) - Method in interface org.apache.beam.sdk.io.aws2.options.AwsOptions
 
setKinesisIOConsumerArns(Map<String, String>) - Method in interface org.apache.beam.sdk.io.aws2.kinesis.KinesisIOOptions
 
setProxyConfiguration(ProxyConfiguration) - Method in interface org.apache.beam.sdk.io.aws2.options.AwsOptions
 
setS3ClientBuilder(S3ClientBuilder) - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration.Builder
 
setS3ClientFactoryClass(Class<? extends S3ClientBuilderFactory>) - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
 
setS3StorageClass(String) - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
 
setS3StorageClass(String) - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration.Builder
 
setS3ThreadPoolSize(int) - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
 
setS3ThreadPoolSize(int) - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration.Builder
 
setS3UploadBufferSizeBytes(Integer) - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
 
setS3UploadBufferSizeBytes(int) - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration.Builder
 
setScheme(String) - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration.Builder
 
setSSEAlgorithm(String) - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
 
setSSEAlgorithm(String) - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration.Builder
 
setSSECustomerKey(SSECustomerKey) - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
 
setSSECustomerKey(SSECustomerKey) - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration.Builder
 
setSSEKMSKeyId(String) - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
 
setSSEKMSKeyId(String) - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration.Builder
 
setupModule(Module.SetupContext) - Method in class org.apache.beam.sdk.io.aws2.options.AwsModule
 
shardRefreshInterval(Duration) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.RecordAggregation.Builder
Refresh interval for shards.
SnsIO - Class in org.apache.beam.sdk.io.aws2.sns
IO to send notifications via SNS.
SnsIO() - Constructor for class org.apache.beam.sdk.io.aws2.sns.SnsIO
 
SnsIO.Write<T> - Class in org.apache.beam.sdk.io.aws2.sns
Implementation of SnsIO.write().
socketTimeout(Integer) - Method in class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration.Builder
Milliseconds to wait for data to be transferred over an established, open connection before the connection is timed out.
socketTimeout() - Method in class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration
Milliseconds to wait for data to be transferred over an established, open connection before the connection is timed out.
SqsIO - Class in org.apache.beam.sdk.io.aws2.sqs
IO to read (unbounded) from and write to SQS queues.
SqsIO.Read - Class in org.apache.beam.sdk.io.aws2.sqs
A PTransform to read/receive messages from SQS.
SqsIO.Write - Class in org.apache.beam.sdk.io.aws2.sqs
Deprecated.
superseded by SqsIO.WriteBatches
SqsIO.WriteBatches<T> - Class in org.apache.beam.sdk.io.aws2.sqs
A PTransform to send messages to SQS.
SqsIO.WriteBatches.DynamicDestination<T> - Interface in org.apache.beam.sdk.io.aws2.sqs
 
SqsIO.WriteBatches.EntryMapperFn<T> - Interface in org.apache.beam.sdk.io.aws2.sqs
Mapper to create a SendMessageBatchRequestEntry from a unique batch entry id and the input T.
SqsIO.WriteBatches.EntryMapperFn.Builder<T> - Interface in org.apache.beam.sdk.io.aws2.sqs
A more convenient SqsIO.WriteBatches.EntryMapperFn variant that already sets the entry id.
SqsIO.WriteBatches.Result - Class in org.apache.beam.sdk.io.aws2.sqs
SqsMessage - Class in org.apache.beam.sdk.io.aws2.sqs
 
SqsMessage() - Constructor for class org.apache.beam.sdk.io.aws2.sqs.SqsMessage
 
SSECustomerKey - Class in org.apache.beam.sdk.io.aws2.s3
Customer provided key for use with Amazon S3 server-side encryption.
SSECustomerKey.Builder - Class in org.apache.beam.sdk.io.aws2.s3
 
SSECustomerKeyFactory() - Constructor for class org.apache.beam.sdk.io.aws2.options.S3Options.SSECustomerKeyFactory
 
submitFn - Variable in class org.apache.beam.sdk.io.aws2.common.AsyncBatchWriteHandler
 

T

throttledBaseBackoff(Duration) - Method in class org.apache.beam.sdk.io.aws2.common.RetryConfiguration.Builder
 
throttledBaseBackoff() - Method in class org.apache.beam.sdk.io.aws2.common.RetryConfiguration
 
TimeUtil - Class in org.apache.beam.sdk.io.aws2.kinesis
Time conversion utilities.
to(SqsIO.WriteBatches.DynamicDestination<T>) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches
Dynamic record based destination to write to.
to(String) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches
Queue url to write to.
toBuilder() - Method in class org.apache.beam.sdk.io.aws2.common.ClientConfiguration
 
toBuilder() - Method in class org.apache.beam.sdk.io.aws2.common.RetryConfiguration
 
toBuilder() - Method in class org.apache.beam.sdk.io.aws2.kinesis.WatermarkParameters
 
toBuilder() - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
Creates a new S3FileSystemConfiguration.Builder with values initialized by this instance's properties.
toJava(Instant) - Static method in class org.apache.beam.sdk.io.aws2.kinesis.TimeUtil
 
toJoda(Instant) - Static method in class org.apache.beam.sdk.io.aws2.kinesis.TimeUtil
 
toString() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
 

U

update(KinesisRecord) - Method in interface org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicy
 
update(KinesisRecord) - Method in class org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicyFactory.ArrivalTimeWatermarkPolicy
 
update(KinesisRecord) - Method in class org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicyFactory.CustomWatermarkPolicy
 
update(KinesisRecord) - Method in class org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicyFactory.ProcessingTimeWatermarkPolicy
 

V

validate(AwsOptions, ClientConfiguration) - Static method in interface org.apache.beam.sdk.io.aws2.common.ClientBuilderFactory
Utility to validate if all necessary configuration is available to create clients using the ClientBuilderFactory configured in AwsOptions.

W

waitForCompletion() - Method in class org.apache.beam.sdk.io.aws2.common.AsyncBatchWriteHandler
Wait for all pending requests to complete and check for failures.
WatermarkParameters - Class in org.apache.beam.sdk.io.aws2.kinesis
WatermarkParameters contains the parameters used for watermark computation.
WatermarkParameters() - Constructor for class org.apache.beam.sdk.io.aws2.kinesis.WatermarkParameters
 
WatermarkPolicy - Interface in org.apache.beam.sdk.io.aws2.kinesis
Implement this interface to define a custom watermark calculation heuristic.
WatermarkPolicyFactory - Interface in org.apache.beam.sdk.io.aws2.kinesis
Implement this interface to create a WatermarkPolicy.
WatermarkPolicyFactory.ArrivalTimeWatermarkPolicy - Class in org.apache.beam.sdk.io.aws2.kinesis
ArrivalTimeWatermarkPolicy uses WatermarkPolicyFactory.CustomWatermarkPolicy for watermark computation.
WatermarkPolicyFactory.CustomWatermarkPolicy - Class in org.apache.beam.sdk.io.aws2.kinesis
CustomWatermarkPolicy uses parameters defined in WatermarkParameters to compute watermarks.
WatermarkPolicyFactory.ProcessingTimeWatermarkPolicy - Class in org.apache.beam.sdk.io.aws2.kinesis
Watermark policy where the processing time is used as the event time.
withArrivalTimePolicy() - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicyFactory
Returns an ArrivalTimeWatermarkPolicy.
withArrivalTimePolicy(Duration) - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicyFactory
Returns an ArrivalTimeWatermarkPolicy.
withArrivalTimeWatermarkPolicy() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
Specifies the WatermarkPolicyFactory as ArrivalTimeWatermarkPolicyFactory.
withArrivalTimeWatermarkPolicy(Duration) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
Specifies the WatermarkPolicyFactory as ArrivalTimeWatermarkPolicyFactory.
withBatchMaxBytes(int) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
Max.
withBatchMaxRecords(int) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
Max.
withBatchSize(int) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches
The batch size to use, default (and AWS limit) is 10.
withBatchTimeout(Duration) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches
The duration to accumulate records before timing out, default is 3 secs.
withBatchTimeout(Duration, boolean) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches
The duration to accumulate records before timing out, default is 3 secs.
withClientConfiguration(ClientConfiguration) - Method in class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO.Read
Configuration of DynamoDB client.
withClientConfiguration(ClientConfiguration) - Method in class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO.Write
Configuration of DynamoDB client.
withClientConfiguration(ClientConfiguration) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
Configuration of Kinesis & Cloudwatch clients.
withClientConfiguration(ClientConfiguration) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
Configuration of Kinesis client.
withClientConfiguration(ClientConfiguration) - Method in class org.apache.beam.sdk.io.aws2.sns.SnsIO.Write
Configuration of SNS client.
withClientConfiguration(ClientConfiguration) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.Read
Configuration of SQS client.
withClientConfiguration(ClientConfiguration) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.Write
Deprecated.
Configuration of SQS client.
withClientConfiguration(ClientConfiguration) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches
Configuration of SQS client.
withCoder(Coder<T>) - Method in class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO.Read
 
withConcurrentRequests(int) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
Max number of concurrent batch write requests per bundle.
withConcurrentRequests(int) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches
Max number of concurrent batch write requests per bundle, default is 5.
withConsumerArn(String) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
Specify consumer ARN to enable Enhanced Fan-Out.
withCustomRateLimitPolicy(RateLimitPolicyFactory) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
Specifies the RateLimitPolicyFactory for a custom rate limiter.
withCustomWatermarkPolicy(WatermarkPolicyFactory) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
Specifies the WatermarkPolicyFactory as a custom watermarkPolicyFactory.
withCustomWatermarkPolicy(WatermarkParameters) - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicyFactory
Returns an custom WatermarkPolicyFactory.
withDeduplicateKeys(List<String>) - Method in class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO.Write
 
withDefaultRateLimiter() - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory
 
withDefaultRateLimiter(Duration, Duration, Duration) - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory
 
withDelay(Supplier<Duration>) - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory
 
withDynamicDelayRateLimitPolicy(Supplier<Duration>) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
Specifies a dynamic delay rate limit policy with the given function being called at each polling interval to get the next delay value.
withEntryMapper(SqsIO.WriteBatches.EntryMapperFn<T>) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches
Optional mapper to create a batch entry from a unique entry id and the input T, otherwise inferred from the schema.
withEntryMapper(SqsIO.WriteBatches.EntryMapperFn.Builder<T>) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches
Optional mapper to create a batch entry from the input T using a builder, otherwise inferred from the schema.
withFixedDelay() - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory
 
withFixedDelay(Duration) - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory
 
withFixedDelayRateLimitPolicy() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
Specifies a fixed delay rate limit policy with the default delay of 1 second.
withFixedDelayRateLimitPolicy(Duration) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
Specifies a fixed delay rate limit policy with the given delay.
withInitialPositionInStream(InitialPositionInStream) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
Specify reading from some initial position in stream.
withInitialTimestampInStream(Instant) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
Specify reading beginning at given Instant.
withMaxCapacityPerShard(Integer) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
Specifies the maximum number of messages per one shard.
withMaxNumRecords(long) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
Specifies to read at most a given number of records.
withMaxNumRecords(long) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.Read
Define the max number of records received by the SqsIO.Read.
withMaxReadTime(Duration) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
Specifies to read records during maxReadTime.
withMaxReadTime(Duration) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.Read
Define the max read time (duration) while the SqsIO.Read will receive messages.
withoutLimiter() - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory
 
withPartitioner(KinesisPartitioner<T>) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
Specify how to partition records among all stream shards (required).
withProcessingTimePolicy() - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicyFactory
Returns an ProcessingTimeWatermarkPolicy.
withProcessingTimeWatermarkPolicy() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
Specifies the WatermarkPolicyFactory as ProcessingTimeWatermarkPolicyFactory.
withPublishRequestBuilder(SerializableFunction<T, PublishRequest.Builder>) - Method in class org.apache.beam.sdk.io.aws2.sns.SnsIO.Write
Function to convert a message into a PublishRequest.Builder (mandatory).
withPublishRequestFn(SerializableFunction<T, PublishRequest>) - Method in class org.apache.beam.sdk.io.aws2.sns.SnsIO.Write
withQueueUrl(String) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.Read
Define the queueUrl used by the SqsIO.Read to receive messages from SQS.
withRecordAggregation(KinesisIO.RecordAggregation) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
Enable record aggregation that is compatible with the KPL / KCL.
withRecordAggregation(Consumer<KinesisIO.RecordAggregation.Builder>) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
Enable record aggregation that is compatible with the KPL / KCL.
withRecordAggregationDisabled() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
Disable KPL / KCL like record aggregation.
withRequestRecordsLimit(int) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
Specifies the maximum number of records in GetRecordsResult returned by GetRecords call which is limited by 10K records.
withScanRequestFn(SerializableFunction<Void, ScanRequest>) - Method in class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO.Read
Can't pass ScanRequest object directly from client since this object is not full serializable.
withScanResponseMapperFn(SerializableFunction<ScanResponse, T>) - Method in class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO.Read
 
withSerializer(SerializableFunction<T, byte[]>) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
Specify how to serialize records to bytes on the stream (required).
withStreamName(String) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
Specify reading from streamName.
withStreamName(String) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
Kinesis stream name which will be used for writing (required).
withTimestampFn(SerializableFunction<KinesisRecord, Instant>) - Method in class org.apache.beam.sdk.io.aws2.kinesis.WatermarkParameters
Specify the SerializableFunction to extract the event time from a KinesisRecord.
withTopicArn(String) - Method in class org.apache.beam.sdk.io.aws2.sns.SnsIO.Write
SNS topic ARN used for publishing to SNS.
withUpToDateThreshold(Duration) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
Specifies how late records consumed by this source can be to still be considered on time.
withWatermarkIdleDurationThreshold(Duration) - Method in class org.apache.beam.sdk.io.aws2.kinesis.WatermarkParameters
Specify the watermark idle duration to consider before advancing the watermark.
withWriteRequestMapperFn(SerializableFunction<T, KV<String, WriteRequest>>) - Method in class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO.Write
 
write() - Static method in class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO
 
Write() - Constructor for class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO.Write
 
write() - Static method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO
Returns a new KinesisIO.Write transform for writing to Kinesis.
Write() - Constructor for class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
 
write() - Static method in class org.apache.beam.sdk.io.aws2.sns.SnsIO
 
Write() - Constructor for class org.apache.beam.sdk.io.aws2.sns.SnsIO.Write
 
write() - Static method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO
Deprecated.
Use SqsIO.writeBatches() for more configuration options.
Write() - Constructor for class org.apache.beam.sdk.io.aws2.sqs.SqsIO.Write
Deprecated.
 
writeBatches() - Static method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO
 
WriteBatches() - Constructor for class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches
 
writeTimeout(Integer) - Method in class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration.Builder
Milliseconds to wait for a write on a socket before an exception is thrown.
writeTimeout() - Method in class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration
Milliseconds to wait for a write on a socket before an exception is thrown.
A B C D E F G H I K M N O P Q R S T U V W 
Skip navigation links