Package org.apache.druid.rpc.indexing

Class OverlordClientImpl

java.lang.Object
org.apache.druid.rpc.indexing.OverlordClientImpl
All Implemented Interfaces:
OverlordClient
public class OverlordClientImpl extends Object implements OverlordClient
Production implementation of OverlordClient.

  • Constructor Details

    • OverlordClientImpl

      public OverlordClientImpl(ServiceClient client, com.fasterxml.jackson.databind.ObjectMapper jsonMapper)

  • Method Details

    • findCurrentLeader

      public com.google.common.util.concurrent.ListenableFuture<URI> findCurrentLeader()
      Description copied from interface: OverlordClient
      Contact the Overlord that we believe to be the leader, and return the result of its /druid/indexer/v1/leader API. This may be a different Overlord server than the one we contacted, if a leader change happened since the last time we updated our sense of who the leader is.
      Specified by:
      findCurrentLeader in interface OverlordClient

    • runTask

      public com.google.common.util.concurrent.ListenableFuture<Void> runTask(String taskId, Object taskObject)
      Description copied from interface: OverlordClient
      Run a task with the provided ID and payload. The payload must be convertible by an ObjectMapper into a Task object. This method does not take Task objects directly, because Task is in the indexing-service package.
      Specified by:
      runTask in interface OverlordClient
      Parameters:
      taskId - task ID
      taskObject - task payload

    • cancelTask

      public com.google.common.util.concurrent.ListenableFuture<Void> cancelTask(String taskId)
      Description copied from interface: OverlordClient
      Cancel a task.
      Specified by:
      cancelTask in interface OverlordClient
      Parameters:
      taskId - task ID

    • taskStatuses

      public com.google.common.util.concurrent.ListenableFuture<CloseableIterator<TaskStatusPlus>> taskStatuses(@Nullable String state, @Nullable String dataSource, @Nullable Integer maxCompletedTasks)
      Description copied from interface: OverlordClient
      Return TaskStatusPlus for all tasks matching a set of optional search parameters. Complete tasks are returned in descending order by creation timestamp. Active tasks are returned in no particular order.
      Specified by:
      taskStatuses in interface OverlordClient
      Parameters:
      state - task state: may be "pending", "waiting", "running", or "complete"
      dataSource - datasource
      maxCompletedTasks - maximum number of completed tasks to return. If zero, no complete tasks are returned. If null, all complete tasks within druid.indexer.storage.recentlyFinishedThreshold are returned. This parameter does not affect the number of active tasks returned.
      Returns:
      list of tasks that match the search parameters

    • taskStatuses

      public com.google.common.util.concurrent.ListenableFuture<Map<String,TaskStatus>> taskStatuses(Set<String> taskIds)
      Description copied from interface: OverlordClient
      Return TaskStatus for a set of task IDs.
      Specified by:
      taskStatuses in interface OverlordClient
      Parameters:
      taskIds - task IDs
      Returns:
      map of task ID to status for known tasks. Unknown tasks are not included in the returned map.

    • taskStatus

      public com.google.common.util.concurrent.ListenableFuture<TaskStatusResponse> taskStatus(String taskId)
      Description copied from interface: OverlordClient
      Returns TaskStatusResponse for a particular task ID. This includes somewhat more information than the TaskStatus returned by OverlordClient.taskStatuses(Set).
      Specified by:
      taskStatus in interface OverlordClient

    • findLockedIntervals

      public com.google.common.util.concurrent.ListenableFuture<Map<String,List<org.joda.time.Interval>>> findLockedIntervals(List<LockFilterPolicy> lockFilterPolicies)
      Description copied from interface: OverlordClient
      Returns a list of intervals locked by higher priority conflicting lock types
      Specified by:
      findLockedIntervals in interface OverlordClient
      Parameters:
      lockFilterPolicies - List of all filters for different datasources
      Returns:
      Map from datasource name to list of intervals locked by tasks that have a conflicting lock type with priority greater than or equal to the minTaskPriority for that datasource.

    • taskReportAsMap

      public com.google.common.util.concurrent.ListenableFuture<TaskReport.ReportMap> taskReportAsMap(String taskId)
      Description copied from interface: OverlordClient
      Returns the report object for a task as a map. Certain task types offer live reports; for these task types, this method may return a task report while the task is running. Certain task types only write reports upon successful completion. Certain other task types do not write reports at all. Returns a HttpResponseException with code Response.Status.NOT_FOUND if there is no report available for some reason.
      Specified by:
      taskReportAsMap in interface OverlordClient

    • postSupervisor

      public com.google.common.util.concurrent.ListenableFuture<Map<String,String>> postSupervisor(SupervisorSpec supervisor)
      Description copied from interface: OverlordClient
      Submits a supervisor spec to the Overlord.

      API: /druid/indexer/v1/supervisor

      Specified by:
      postSupervisor in interface OverlordClient
      Returns:
      Map containing a single entry "id"

    • terminateSupervisor

      public com.google.common.util.concurrent.ListenableFuture<Map<String,String>> terminateSupervisor(String supervisorId)
      Description copied from interface: OverlordClient
      Shuts down a supervisor.

      API: /druid/indexer/v1/supervisor/<id>/terminate

      Specified by:
      terminateSupervisor in interface OverlordClient
      Returns:
      Map containing a single entry "id"

    • supervisorStatuses

      public com.google.common.util.concurrent.ListenableFuture<CloseableIterator<SupervisorStatus>> supervisorStatuses()
      Description copied from interface: OverlordClient
      Returns all current supervisor statuses.
      Specified by:
      supervisorStatuses in interface OverlordClient

    • getWorkers

      public com.google.common.util.concurrent.ListenableFuture<List<IndexingWorkerInfo>> getWorkers()
      Description copied from interface: OverlordClient
      Returns information about workers.
      Specified by:
      getWorkers in interface OverlordClient

    • getTotalWorkerCapacity

      public com.google.common.util.concurrent.ListenableFuture<IndexingTotalWorkerCapacityInfo> getTotalWorkerCapacity()
      Description copied from interface: OverlordClient
      Returns total worker capacity details.
      Specified by:
      getTotalWorkerCapacity in interface OverlordClient

    • killPendingSegments

      public com.google.common.util.concurrent.ListenableFuture<Integer> killPendingSegments(String dataSource, org.joda.time.Interval interval)
      Description copied from interface: OverlordClient
      Deletes pending segment records from the metadata store for a particular datasource. Records with created_date within the provided interval are deleted; other records are left alone. Deletion is done synchronously with the API call. When the future resolves, the deletion is complete.
      Specified by:
      killPendingSegments in interface OverlordClient
      Parameters:
      dataSource - datasource name
      interval - created time interval
      Returns:
      number of pending segments deleted

    • taskPayload

      public com.google.common.util.concurrent.ListenableFuture<TaskPayloadResponse> taskPayload(String taskId)
      Description copied from interface: OverlordClient
      Returns the payload for a task as an instance of ClientTaskQuery. This method only works for tasks that have a ClientTaskQuery model or are subclasses of ClientTaskQuery.
      Specified by:
      taskPayload in interface OverlordClient

    • markNonOvershadowedSegmentsAsUsed

      public com.google.common.util.concurrent.ListenableFuture<SegmentUpdateResponse> markNonOvershadowedSegmentsAsUsed(String dataSource)
      Description copied from interface: OverlordClient
      Marks all non-overshadowed segments of the datasource as used.

      API: POST /druid/indexer/v1/datasources/{dataSourceName}

      Specified by:
      markNonOvershadowedSegmentsAsUsed in interface OverlordClient

    • markNonOvershadowedSegmentsAsUsed

      public com.google.common.util.concurrent.ListenableFuture<SegmentUpdateResponse> markNonOvershadowedSegmentsAsUsed(String dataSource, SegmentsToUpdateFilter filter)
      Description copied from interface: OverlordClient
      Marks non-overshadowed segments that satisfy the given filter as used.

      API: POST /druid/indexer/v1/datasources/{dataSourceName}/markUsed

      Specified by:
      markNonOvershadowedSegmentsAsUsed in interface OverlordClient
      filter - Must be non-null

    • markSegmentAsUsed

      public com.google.common.util.concurrent.ListenableFuture<SegmentUpdateResponse> markSegmentAsUsed(SegmentId segmentId)
      Description copied from interface: OverlordClient
      Marks the given segment as used.

      API: POST /druid/indexer/v1/datasources/{dataSourceName}/segments/{segmentId}

      Specified by:
      markSegmentAsUsed in interface OverlordClient

    • markSegmentsAsUnused

      public com.google.common.util.concurrent.ListenableFuture<SegmentUpdateResponse> markSegmentsAsUnused(String dataSource)
      Description copied from interface: OverlordClient
      Marks all non-overshadowed segments of the datasource as unused.

      API: DELETE /druid/indexer/v1/datasources/{dataSourceName}

      Specified by:
      markSegmentsAsUnused in interface OverlordClient

    • markSegmentsAsUnused

      public com.google.common.util.concurrent.ListenableFuture<SegmentUpdateResponse> markSegmentsAsUnused(String dataSource, SegmentsToUpdateFilter filter)
      Description copied from interface: OverlordClient
      Marks non-overshadowed segments that satisfy the given filter as unused.

      API: POST /druid/indexer/v1/datasources/{dataSourceName}/markUnused

      Specified by:
      markSegmentsAsUnused in interface OverlordClient
      filter - Must be non-null

    • markSegmentAsUnused

      public com.google.common.util.concurrent.ListenableFuture<SegmentUpdateResponse> markSegmentAsUnused(SegmentId segmentId)
      Description copied from interface: OverlordClient
      Marks the given segment as unused.

      API: DELETE /druid/indexer/v1/datasources/{dataSourceName}/segments/{segmentId}

      Specified by:
      markSegmentAsUnused in interface OverlordClient

    • isCompactionSupervisorEnabled

      public com.google.common.util.concurrent.ListenableFuture<Boolean> isCompactionSupervisorEnabled()
      Description copied from interface: OverlordClient
      Checks if compaction supervisors are enabled on the Overlord. When this returns true, the Coordinator does not run CompactSegments duty.

      API: GET /druid/indexer/v1/compaction/isSupervisorEnabled

      Specified by:
      isCompactionSupervisorEnabled in interface OverlordClient

    • getClusterCompactionConfig

      public com.google.common.util.concurrent.ListenableFuture<ClusterCompactionConfig> getClusterCompactionConfig()
      Description copied from interface: OverlordClient
      Gets the current cluster-level compaction config.

      API: GET /druid/indexer/v1/compaction/config/cluster

      Specified by:
      getClusterCompactionConfig in interface OverlordClient

    • updateClusterCompactionConfig

      public com.google.common.util.concurrent.ListenableFuture<UpdateResponse> updateClusterCompactionConfig(ClusterCompactionConfig config)
      Description copied from interface: OverlordClient
      Gets the current cluster-level compaction config.

      API: POST /druid/indexer/v1/compaction/config/cluster

      Specified by:
      updateClusterCompactionConfig in interface OverlordClient
      Returns:
      UpdateResponse containing the success status of this operation.

    • withRetryPolicy

      public OverlordClientImpl withRetryPolicy(ServiceRetryPolicy retryPolicy)
      Description copied from interface: OverlordClient
      Returns a copy of this client with a different retry policy.
      Specified by:
      withRetryPolicy in interface OverlordClient