Package org.apache.druid.server.coordinator.loading

Class LoadingRateTracker

java.lang.Object
org.apache.druid.server.coordinator.loading.LoadingRateTracker
@NotThreadSafe public class LoadingRateTracker extends Object
Tracks the current segment loading rate for a single server.

The loading rate is computed as a moving average of the last MOVING_AVERAGE_WINDOW_SIZE segment batches (or more if any batch was smaller than MIN_ENTRY_SIZE_BYTES). A batch is defined as a set of segments added to the load queue together. Usage: 

   batchDurationMillis
   = t(load queue becomes empty) - t(first load request in batch is sent to server)

   batchBytes = total bytes successfully loaded in batch

   avg loading rate in batch (kbps) = (8 * batchBytes) / batchDurationMillis

   overall avg loading rate (kbps)
   = (8 * sumOverWindow(batchBytes)) / sumOverWindow(batchDurationMillis)

This class is currently not required to be thread-safe as the caller HttpLoadQueuePeon itself ensures that the write methods of this class are only accessed by one thread at a time.

  • Field Details

    • MOVING_AVERAGE_WINDOW_SIZE

      public static final int MOVING_AVERAGE_WINDOW_SIZE
      See Also:

    • MIN_ENTRY_SIZE_BYTES

      public static final long MIN_ENTRY_SIZE_BYTES
      Minimum size of a single entry in the moving average window = 1 GiB.
      See Also:

  • Constructor Details

    • LoadingRateTracker

      public LoadingRateTracker()

  • Method Details

    • markBatchLoadingStarted

      public void markBatchLoadingStarted()
      Marks the start of loading of a batch of segments. This should be called when the first request in a batch is sent to the server.

    • isLoadingBatch

      public boolean isLoadingBatch()
      Returns:
      if a batch of segments is currently being loaded.

    • incrementBytesLoadedInBatch

      public void incrementBytesLoadedInBatch(long loadedBytes)
      Adds the given number of bytes to the total data successfully loaded in the current batch. This causes an update of the current load rate.
      Throws:
      DruidException - if called without making a prior call to markBatchLoadingStarted().

    • markBatchLoadingFinished

      public void markBatchLoadingFinished()
      Marks the end of loading of a batch of segments. This method should be called when all the requests in the batch have been processed by the server.

    • stop

      public void stop()
      Stops this rate tracker and resets its current state.

    • getMovingAverageLoadRateKbps

      public long getMovingAverageLoadRateKbps()
      Moving average load rate in kbps (1000 bits per second).