Package org.apache.hudi.common.util.queue

Interface Summary

Interface	Description
HoodieConsumer<I,O>	HoodieConsumer is used to consume records/messages from hoodie inner message queue and write into DFS.
HoodieExecutor<E>	HoodieExecutor which orchestrates concurrent producers and consumers communicating.
HoodieMessageQueue<I,O>	HoodieMessageQueue holds an internal message queue, and control the behavior of 1.
HoodieProducer<I>	Producer for HoodieMessageQueue.

Class Summary

Class	Description
BaseHoodieQueueBasedExecutor<I,O,E>	Base class for multi-threaded queue-based executor which Can be ingesting instances of type I from multiple HoodieProducers into the HoodieMessageQueue Can be ingesting instances of type I into an (optional) HoodieConsumer from the internal HoodieMessageQueue (when no consumer is provided records are simply accumulated into internal queue) Such executors are allowing to setup an ingestion pipeline w/ N:1 configuration, where data is ingested from multiple sources (ie producers) into a singular sink (ie consumer), using an internal queue to stage the records ingested from producers before these are consumed
BoundedInMemoryExecutor<I,O,E>	Executor which orchestrates concurrent producers and consumers communicating through 'BoundedInMemoryQueue'.
BoundedInMemoryQueue<I,O>	Used for enqueueing input records.
DisruptorExecutor<I,O,E>	Executor which orchestrates concurrent producers and consumers communicating through 'DisruptorMessageQueue'.
DisruptorMessageQueue<I,O>	Implementation of HoodieMessageQueue based on Disruptor Queue.
FunctionBasedQueueProducer<I>	Buffer producer which allows custom functions to insert entries to queue.
IteratorBasedQueueProducer<I>	Iterator based producer which pulls entry from iterator and produces items for the queue.
SimpleExecutor<I,O,E>	Simple implementation of the HoodieExecutor interface assuming single-writer/single-reader mode allowing it to consume from the input Iterator directly avoiding the need for any internal materialization (ie queueing).
WaitStrategyFactory	Factory for creating wait strategy for a queue.

Enum Summary

Enum	Description
DisruptorWaitStrategyType	Enum for the type of waiting strategy in Disruptor Queue.
ExecutorType	Types of HoodieExecutor.