Package org.apache.pinot.plugin.stream.kinesis

Class KinesisStreamMetadataProvider

java.lang.Object

org.apache.pinot.plugin.stream.kinesis.KinesisStreamMetadataProvider

All Implemented Interfaces:

Closeable, AutoCloseable, StreamMetadataProvider

public class KinesisStreamMetadataProvider
extends Object
implements StreamMetadataProvider

A StreamMetadataProvider implementation for the Kinesis stream

Constructor Summary

Constructors

Constructor	Description
KinesisStreamMetadataProvider(String clientId, StreamConfig streamConfig)
KinesisStreamMetadataProvider(String clientId, StreamConfig streamConfig, KinesisConnectionHandler kinesisConnectionHandler, StreamConsumerFactory streamConsumerFactory)

Method Summary

Modifier and Type	Method	Description
void	close()
List<PartitionGroupMetadata>	computePartitionGroupMetadata(String clientId, StreamConfig streamConfig, List<PartitionGroupConsumptionStatus> partitionGroupConsumptionStatuses, int timeoutMillis)	This call returns all active shards, taking into account the consumption status for those shards.
int	fetchPartitionCount(long timeoutMillis)
StreamPartitionMsgOffset	fetchStreamPartitionOffset(OffsetCriteria offsetCriteria, long timeoutMillis)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

KinesisStreamMetadataProvider

public KinesisStreamMetadataProvider(String clientId,
                                     StreamConfig streamConfig)

KinesisStreamMetadataProvider

public KinesisStreamMetadataProvider(String clientId,
                                     StreamConfig streamConfig,
                                     KinesisConnectionHandler kinesisConnectionHandler,
                                     StreamConsumerFactory streamConsumerFactory)

Method Detail

fetchPartitionCount

public int fetchPartitionCount(long timeoutMillis)

Specified by:

fetchPartitionCount in interface StreamMetadataProvider

fetchStreamPartitionOffset

public StreamPartitionMsgOffset fetchStreamPartitionOffset(OffsetCriteria offsetCriteria,
                                                           long timeoutMillis)

Specified by:

fetchStreamPartitionOffset in interface StreamMetadataProvider

computePartitionGroupMetadata

public List<PartitionGroupMetadata> computePartitionGroupMetadata(String clientId,
                                                                  StreamConfig streamConfig,
                                                                  List<PartitionGroupConsumptionStatus> partitionGroupConsumptionStatuses,
                                                                  int timeoutMillis)
                                                           throws IOException,
                                                                  TimeoutException

This call returns all active shards, taking into account the consumption status for those shards. PartitionGroupMetadata is returned for a shard if: 1. It is a branch new shard AND its parent has been consumed completely 2. It is still being actively consumed from i.e. the consuming partition has not reached the end of the shard

Specified by:

computePartitionGroupMetadata in interface StreamMetadataProvider

Throws:

IOException

TimeoutException

close

public void close()

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable