Package org.apache.pulsar.transaction.coordinator.impl

Class TxnLogBufferedWriter<T>

java.lang.Object

org.apache.pulsar.transaction.coordinator.impl.TxnLogBufferedWriter<T>

public class TxnLogBufferedWriter<T>
extends Object

See PIP-160: https://github.com/apache/pulsar/issues/15516. Buffer requests and flush to Managed Ledger. Transaction Log Store And Pending Ack Store will no longer write to Managed Ledger directly, Change to using this class to write Ledger data. Caches “write requests” for a certain number or a certain size of request data and then writes them to the Managed Ledger in one go. After Managed Ledger has written complete, responds to each request-caller. In this process, Managed Ledger doesn't care how many records(or what to be written) in the Entry, it just treats them as a single block of data. The first write-request by transaction components will take a long time to receive a response, because we must wait for subsequent requests to accumulate enough data to actually start writing to the Managed Ledger. To control the maximum latency, we will mark the first request time for each batch, and additional timing triggers writes. You can enable or disabled the batch feature, will use Managed Ledger directly and without batching when disabled.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	TxnLogBufferedWriter.AddDataCallback
static interface	TxnLogBufferedWriter.DataSerializer<T>	The serializer for the object which called by asyncAddData(T, org.apache.pulsar.transaction.coordinator.impl.TxnLogBufferedWriter.AddDataCallback, java.lang.Object).

Field Summary

Fields

Modifier and Type	Field	Description
static final short	BATCHED_ENTRY_DATA_PREFIX_MAGIC_NUMBER
static final int	BATCHED_ENTRY_DATA_PREFIX_MAGIC_NUMBER_LEN
static final short	BATCHED_ENTRY_DATA_PREFIX_VERSION
static final short	BATCHED_ENTRY_DATA_PREFIX_VERSION_LEN

Constructor Summary

Constructors

Constructor	Description
TxnLogBufferedWriter(ManagedLedger managedLedger, Executor executor, io.netty.util.Timer timer, TxnLogBufferedWriter.DataSerializer<T> dataSerializer, int batchedWriteMaxRecords, int batchedWriteMaxSize, int batchedWriteMaxDelayInMillis, boolean batchEnabled, TxnLogBufferedWriterMetricsStats metrics)	Constructor.

Method Summary

Modifier and Type	Method	Description
void	asyncAddData(T data, TxnLogBufferedWriter.AddDataCallback callback, Object ctx)	Append a new entry to the end of a managed ledger.
CompletableFuture<Void>	close()	Release resources and cancel pending tasks.
TxnLogBufferedWriterMetricsStats	getMetrics()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

BATCHED_ENTRY_DATA_PREFIX_MAGIC_NUMBER

public static final short BATCHED_ENTRY_DATA_PREFIX_MAGIC_NUMBER

See Also:

• Constant Field Values

BATCHED_ENTRY_DATA_PREFIX_MAGIC_NUMBER_LEN

public static final int BATCHED_ENTRY_DATA_PREFIX_MAGIC_NUMBER_LEN

See Also:

• Constant Field Values

BATCHED_ENTRY_DATA_PREFIX_VERSION

public static final short BATCHED_ENTRY_DATA_PREFIX_VERSION

See Also:

• Constant Field Values

BATCHED_ENTRY_DATA_PREFIX_VERSION_LEN

public static final short BATCHED_ENTRY_DATA_PREFIX_VERSION_LEN

See Also:

• Constant Field Values

Constructor Details

TxnLogBufferedWriter

public TxnLogBufferedWriter(ManagedLedger managedLedger,
 Executor executor,
 io.netty.util.Timer timer,
 TxnLogBufferedWriter.DataSerializer<T> dataSerializer,
 int batchedWriteMaxRecords,
 int batchedWriteMaxSize,
 int batchedWriteMaxDelayInMillis,
 boolean batchEnabled,
 TxnLogBufferedWriterMetricsStats metrics)

Constructor.

Parameters:

dataSerializer - The serializer for the object which called by asyncAddData(T, org.apache.pulsar.transaction.coordinator.impl.TxnLogBufferedWriter.AddDataCallback, java.lang.Object).

batchedWriteMaxRecords - Caches “write requests” for a certain number, if reach this threshold, will trig Bookie writes.

batchedWriteMaxSize - Caches “write requests” for a certain size of request data, if reach this threshold, will trig Bookie writes.

batchedWriteMaxDelayInMillis - Maximum delay for writing to bookie for the earliest request in the batch.

batchEnabled - Enable or disabled the batch feature, will use Managed Ledger directly and without batching when disabled.

timer - Used for periodic flush.

Method Details

asyncAddData

public void asyncAddData(T data,
 TxnLogBufferedWriter.AddDataCallback callback,
 Object ctx)

Append a new entry to the end of a managed ledger. All writes will be performed in the same thread. Callbacks are executed in strict write order，but after close(), callbacks that fail by state check will execute earlier, and successful callbacks will not be affected.

Parameters:

data - data entry to be persisted.

callback - Will call TxnLogBufferedWriter.AddDataCallback.addComplete(Position, Object) when add complete. Will call TxnLogBufferedWriter.AddDataCallback.addFailed(ManagedLedgerException, Object) when add failure.

Throws:

ManagedLedgerException

close

public CompletableFuture<Void> close()

Release resources and cancel pending tasks.

getMetrics

public TxnLogBufferedWriterMetricsStats getMetrics()