- get() - Method in class org.apache.beam.sdk.io.aws2.schemas.AwsBuilderFactory
-
- getAlgorithm() - Method in class org.apache.beam.sdk.io.aws2.s3.SSECustomerKey
-
- getApproximateArrivalTimestamp() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
-
- getAwsCredentialsProvider() - Method in interface org.apache.beam.sdk.io.aws2.options.AwsOptions
-
AwsCredentialsProvider used to configure AWS service clients.
- getAwsRegion() - Method in interface org.apache.beam.sdk.io.aws2.options.AwsOptions
-
Region used to configure AWS service clients.
- getBody() - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsMessage
-
Message body.
- getBucketKeyEnabled() - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
-
- getBucketKeyEnabled() - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
-
Whether to use an S3 Bucket Key for object encryption with server-side encryption using AWS KMS
(SSE-KMS) or not.
- getClientBuilderFactory() - Method in interface org.apache.beam.sdk.io.aws2.options.AwsOptions
-
- getData() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
-
- getDataAsBytes() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
-
- getEndpoint() - Method in interface org.apache.beam.sdk.io.aws2.options.AwsOptions
-
Endpoint used to configure AWS service clients.
- getExplicitHashKey(T) - Method in interface org.apache.beam.sdk.io.aws2.kinesis.KinesisPartitioner.ExplicitPartitioner
-
Required hash value (128-bit integer) to determine explicitly the shard a record is assigned
to based on the hash key range of each shard.
- getExplicitHashKey(T) - Method in interface org.apache.beam.sdk.io.aws2.kinesis.KinesisPartitioner
-
Optional hash value (128-bit integer) to determine explicitly the shard a record is assigned to
based on the hash key range of each shard.
- getExtendedSequenceNumber() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
-
- getFactory(AwsOptions) - Static method in interface org.apache.beam.sdk.io.aws2.common.ClientBuilderFactory
-
- getHttpClientConfiguration() - Method in interface org.apache.beam.sdk.io.aws2.options.AwsOptions
-
- getKey() - Method in class org.apache.beam.sdk.io.aws2.s3.SSECustomerKey
-
- getKinesisIOConsumerArns() - Method in interface org.apache.beam.sdk.io.aws2.kinesis.KinesisIOOptions
-
Used to enable / disable EFO.
- getMD5() - Method in class org.apache.beam.sdk.io.aws2.s3.SSECustomerKey
-
- getMessageId() - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsMessage
-
SQS message id.
- getPartitionKey(T) - Method in interface org.apache.beam.sdk.io.aws2.kinesis.KinesisPartitioner.ExplicitPartitioner
-
- getPartitionKey(T) - Method in interface org.apache.beam.sdk.io.aws2.kinesis.KinesisPartitioner
-
Determines which shard in the stream the record is assigned to.
- getPartitionKey() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
-
- getPipeline() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write.Result
-
- getPipeline() - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches.Result
-
- getPipelineOptions() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIOOptions.KinesisIOOptionsRegistrar
-
- getPipelineOptions() - Method in class org.apache.beam.sdk.io.aws2.options.AwsPipelineOptionsRegistrar
-
- getProxyConfiguration() - Method in interface org.apache.beam.sdk.io.aws2.options.AwsOptions
-
ProxyConfiguration used to configure AWS service clients.
- getRateLimitPolicy() - Method in interface org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory
-
- getReadTime() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
-
- getReceiptHandle() - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsMessage
-
SQS receipt handle.
- getRequestTimeStamp() - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsMessage
-
Timestamp the message was received at (in epoch millis).
- getS3ClientBuilder() - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
-
Builder used to create the S3Client.
- getS3ClientFactoryClass() - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
-
- getS3StorageClass() - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
-
- getS3StorageClass() - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
-
The AWS S3 storage class used for creating S3 objects.
- getS3ThreadPoolSize() - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
-
- getS3ThreadPoolSize() - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
-
Thread pool size, limiting the max concurrent S3 operations.
- getS3UploadBufferSizeBytes() - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
-
- getS3UploadBufferSizeBytes() - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
-
Size of S3 upload chnks.
- getSchemaProviders() - Method in class org.apache.beam.sdk.io.aws2.schemas.AwsSchemaRegistrar
-
- getScheme() - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
-
The uri scheme used by resources on this filesystem.
- getSequenceNumber() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
-
- getShardId() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
-
- getSSEAlgorithm() - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
-
- getSSEAlgorithm() - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
-
Algorithm for SSE-S3 encryption, e.g.
- getSSECustomerKey() - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
-
- getSSECustomerKey() - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
-
SSE key for SSE-C encryption, e.g.
- getSSEKMSKeyId() - Method in interface org.apache.beam.sdk.io.aws2.options.S3Options
-
- getSSEKMSKeyId() - Method in class org.apache.beam.sdk.io.aws2.s3.S3FileSystemConfiguration
-
KMS key id for SSE-KMS encyrption, e.g.
- getStreamName() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
-
- getSubSequenceNumber() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
-
- getTimeStamp() - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsMessage
-
Timestamp the message was sent at (in epoch millis).
- getUniqueId() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisRecord
-
- getWatermark() - Method in interface org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicy
-
- getWatermark() - Method in class org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicyFactory.ArrivalTimeWatermarkPolicy
-
- getWatermark() - Method in class org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicyFactory.CustomWatermarkPolicy
-
- getWatermark() - Method in class org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicyFactory.ProcessingTimeWatermarkPolicy
-
- waitForCompletion() - Method in class org.apache.beam.sdk.io.aws2.common.AsyncBatchWriteHandler
-
Wait for all pending requests to complete and check for failures.
- WatermarkParameters - Class in org.apache.beam.sdk.io.aws2.kinesis
-
WatermarkParameters contains the parameters used for watermark computation.
- WatermarkParameters() - Constructor for class org.apache.beam.sdk.io.aws2.kinesis.WatermarkParameters
-
- WatermarkPolicy - Interface in org.apache.beam.sdk.io.aws2.kinesis
-
Implement this interface to define a custom watermark calculation heuristic.
- WatermarkPolicyFactory - Interface in org.apache.beam.sdk.io.aws2.kinesis
-
Implement this interface to create a WatermarkPolicy.
- WatermarkPolicyFactory.ArrivalTimeWatermarkPolicy - Class in org.apache.beam.sdk.io.aws2.kinesis
-
- WatermarkPolicyFactory.CustomWatermarkPolicy - Class in org.apache.beam.sdk.io.aws2.kinesis
-
- WatermarkPolicyFactory.ProcessingTimeWatermarkPolicy - Class in org.apache.beam.sdk.io.aws2.kinesis
-
Watermark policy where the processing time is used as the event time.
- withArrivalTimePolicy() - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicyFactory
-
Returns an ArrivalTimeWatermarkPolicy.
- withArrivalTimePolicy(Duration) - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicyFactory
-
Returns an ArrivalTimeWatermarkPolicy.
- withArrivalTimeWatermarkPolicy() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
-
Specifies the WatermarkPolicyFactory as ArrivalTimeWatermarkPolicyFactory.
- withArrivalTimeWatermarkPolicy(Duration) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
-
Specifies the WatermarkPolicyFactory as ArrivalTimeWatermarkPolicyFactory.
- withBatchMaxBytes(int) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
-
Max.
- withBatchMaxRecords(int) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
-
Max.
- withBatchSize(int) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches
-
The batch size to use, default (and AWS limit) is 10.
- withBatchTimeout(Duration) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches
-
The duration to accumulate records before timing out, default is 3 secs.
- withBatchTimeout(Duration, boolean) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches
-
The duration to accumulate records before timing out, default is 3 secs.
- withClientConfiguration(ClientConfiguration) - Method in class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO.Read
-
Configuration of DynamoDB client.
- withClientConfiguration(ClientConfiguration) - Method in class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO.Write
-
Configuration of DynamoDB client.
- withClientConfiguration(ClientConfiguration) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
-
Configuration of Kinesis & Cloudwatch clients.
- withClientConfiguration(ClientConfiguration) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
-
Configuration of Kinesis client.
- withClientConfiguration(ClientConfiguration) - Method in class org.apache.beam.sdk.io.aws2.sns.SnsIO.Write
-
Configuration of SNS client.
- withClientConfiguration(ClientConfiguration) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.Read
-
Configuration of SQS client.
- withClientConfiguration(ClientConfiguration) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.Write
-
Deprecated.
Configuration of SQS client.
- withClientConfiguration(ClientConfiguration) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches
-
Configuration of SQS client.
- withCoder(Coder<T>) - Method in class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO.Read
-
- withConcurrentRequests(int) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
-
Max number of concurrent batch write requests per bundle.
- withConcurrentRequests(int) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches
-
Max number of concurrent batch write requests per bundle, default is 5.
- withConsumerArn(String) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
-
Specify consumer ARN to enable Enhanced Fan-Out.
- withCustomRateLimitPolicy(RateLimitPolicyFactory) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
-
Specifies the RateLimitPolicyFactory for a custom rate limiter.
- withCustomWatermarkPolicy(WatermarkPolicyFactory) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
-
Specifies the WatermarkPolicyFactory as a custom watermarkPolicyFactory.
- withCustomWatermarkPolicy(WatermarkParameters) - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicyFactory
-
Returns an custom WatermarkPolicyFactory.
- withDeduplicateKeys(List<String>) - Method in class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO.Write
-
- withDefaultRateLimiter() - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory
-
- withDefaultRateLimiter(Duration, Duration, Duration) - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory
-
- withDelay(Supplier<Duration>) - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory
-
- withDynamicDelayRateLimitPolicy(Supplier<Duration>) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
-
Specifies a dynamic delay rate limit policy with the given function being called at each
polling interval to get the next delay value.
- withEntryMapper(SqsIO.WriteBatches.EntryMapperFn<T>) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches
-
Optional mapper to create a batch entry from a unique entry id and the input T,
otherwise inferred from the schema.
- withEntryMapper(SqsIO.WriteBatches.EntryMapperFn.Builder<T>) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches
-
Optional mapper to create a batch entry from the input T using a builder, otherwise
inferred from the schema.
- withFixedDelay() - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory
-
- withFixedDelay(Duration) - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory
-
- withFixedDelayRateLimitPolicy() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
-
Specifies a fixed delay rate limit policy with the default delay of 1 second.
- withFixedDelayRateLimitPolicy(Duration) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
-
Specifies a fixed delay rate limit policy with the given delay.
- withInitialPositionInStream(InitialPositionInStream) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
-
Specify reading from some initial position in stream.
- withInitialTimestampInStream(Instant) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
-
Specify reading beginning at given Instant.
- withMaxCapacityPerShard(Integer) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
-
Specifies the maximum number of messages per one shard.
- withMaxNumRecords(long) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
-
Specifies to read at most a given number of records.
- withMaxNumRecords(long) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.Read
-
Define the max number of records received by the
SqsIO.Read.
- withMaxReadTime(Duration) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
-
Specifies to read records during maxReadTime.
- withMaxReadTime(Duration) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.Read
-
Define the max read time (duration) while the
SqsIO.Read will receive messages.
- withoutLimiter() - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.RateLimitPolicyFactory
-
- withPartitioner(KinesisPartitioner<T>) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
-
Specify how to partition records among all stream shards (required).
- withProcessingTimePolicy() - Static method in interface org.apache.beam.sdk.io.aws2.kinesis.WatermarkPolicyFactory
-
Returns an ProcessingTimeWatermarkPolicy.
- withProcessingTimeWatermarkPolicy() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
-
Specifies the WatermarkPolicyFactory as ProcessingTimeWatermarkPolicyFactory.
- withPublishRequestBuilder(SerializableFunction<T, PublishRequest.Builder>) - Method in class org.apache.beam.sdk.io.aws2.sns.SnsIO.Write
-
Function to convert a message into a PublishRequest.Builder (mandatory).
- withPublishRequestFn(SerializableFunction<T, PublishRequest>) - Method in class org.apache.beam.sdk.io.aws2.sns.SnsIO.Write
-
- withQueueUrl(String) - Method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO.Read
-
Define the queueUrl used by the
SqsIO.Read to receive messages from SQS.
- withRecordAggregation(KinesisIO.RecordAggregation) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
-
Enable record aggregation that is compatible with the KPL / KCL.
- withRecordAggregation(Consumer<KinesisIO.RecordAggregation.Builder>) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
-
Enable record aggregation that is compatible with the KPL / KCL.
- withRecordAggregationDisabled() - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
-
Disable KPL / KCL like record aggregation.
- withRequestRecordsLimit(int) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
-
Specifies the maximum number of records in GetRecordsResult returned by GetRecords call which
is limited by 10K records.
- withScanRequestFn(SerializableFunction<Void, ScanRequest>) - Method in class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO.Read
-
Can't pass ScanRequest object directly from client since this object is not full
serializable.
- withScanResponseMapperFn(SerializableFunction<ScanResponse, T>) - Method in class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO.Read
-
- withSerializer(SerializableFunction<T, byte[]>) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
-
Specify how to serialize records to bytes on the stream (required).
- withStreamName(String) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
-
Specify reading from streamName.
- withStreamName(String) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
-
Kinesis stream name which will be used for writing (required).
- withTimestampFn(SerializableFunction<KinesisRecord, Instant>) - Method in class org.apache.beam.sdk.io.aws2.kinesis.WatermarkParameters
-
Specify the SerializableFunction to extract the event time from a KinesisRecord.
- withTopicArn(String) - Method in class org.apache.beam.sdk.io.aws2.sns.SnsIO.Write
-
SNS topic ARN used for publishing to SNS.
- withUpToDateThreshold(Duration) - Method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Read
-
Specifies how late records consumed by this source can be to still be considered on time.
- withWatermarkIdleDurationThreshold(Duration) - Method in class org.apache.beam.sdk.io.aws2.kinesis.WatermarkParameters
-
Specify the watermark idle duration to consider before advancing the watermark.
- withWriteRequestMapperFn(SerializableFunction<T, KV<String, WriteRequest>>) - Method in class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO.Write
-
- write() - Static method in class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO
-
- Write() - Constructor for class org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIO.Write
-
- write() - Static method in class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO
-
- Write() - Constructor for class org.apache.beam.sdk.io.aws2.kinesis.KinesisIO.Write
-
- write() - Static method in class org.apache.beam.sdk.io.aws2.sns.SnsIO
-
- Write() - Constructor for class org.apache.beam.sdk.io.aws2.sns.SnsIO.Write
-
- write() - Static method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO
-
- Write() - Constructor for class org.apache.beam.sdk.io.aws2.sqs.SqsIO.Write
-
Deprecated.
- writeBatches() - Static method in class org.apache.beam.sdk.io.aws2.sqs.SqsIO
-
- WriteBatches() - Constructor for class org.apache.beam.sdk.io.aws2.sqs.SqsIO.WriteBatches
-
- writeTimeout(Integer) - Method in class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration.Builder
-
Milliseconds to wait for a write on a socket before an exception is thrown.
- writeTimeout() - Method in class org.apache.beam.sdk.io.aws2.common.HttpClientConfiguration
-
Milliseconds to wait for a write on a socket before an exception is thrown.