Package org.apache.druid.rpc.indexing

Interface OverlordClient

All Known Implementing Classes:
NoopOverlordClient, OverlordClientImpl
public interface OverlordClient
High-level Overlord client. All methods return futures, enabling asynchronous logic. If you want a synchronous response, use FutureUtils.get or FutureUtils.getUnchecked. Futures resolve to exceptions in the manner described by ServiceClient.asyncRequest(org.apache.druid.rpc.RequestBuilder, org.apache.druid.java.util.http.client.response.HttpResponseHandler<IntermediateType, FinalType>). Typically acquired via Guice, where it is registered using ServiceClientModule.

  • Method Details

    • findCurrentLeader

      com.google.common.util.concurrent.ListenableFuture<URI> findCurrentLeader()
      Contact the Overlord that we believe to be the leader, and return the result of its /druid/indexer/v1/leader API. This may be a different Overlord server than the one we contacted, if a leader change happened since the last time we updated our sense of who the leader is.

    • runTask

      com.google.common.util.concurrent.ListenableFuture<Void> runTask(String taskId, Object taskObject)
      Run a task with the provided ID and payload. The payload must be convertible by an ObjectMapper into a Task object. This method does not take Task objects directly, because Task is in the indexing-service package.
      Parameters:
      taskId - task ID
      taskObject - task payload

    • runKillTask

      default com.google.common.util.concurrent.ListenableFuture<String> runKillTask(String idPrefix, String dataSource, org.joda.time.Interval interval, @Nullable List<String> versions, @Nullable Integer maxSegmentsToKill, @Nullable org.joda.time.DateTime maxUsedStatusLastUpdatedTime)
      Run a "kill" task for a particular datasource and interval. Shortcut to runTask(String, Object). The kill task deletes all unused segment records from deep storage and the metadata store. The task runs asynchronously after the API call returns. The resolved future is the ID of the task, which can be used to monitor its progress through the taskStatus(String) API.
      Parameters:
      idPrefix - Descriptive prefix to include at the start of task IDs
      dataSource - Datasource to kill
      interval - Umbrella interval to be considered by the kill task. Note that unused segments falling in this widened umbrella interval may have different used_status_last_updated time, so the kill task should also filter by maxUsedStatusLastUpdatedTime
      versions - An optional list of segment versions to kill in the given interval. If unspecified, all versions of segments in the interval must be killed.
      maxSegmentsToKill - The maximum number of segments to kill
      maxUsedStatusLastUpdatedTime - The maximum used_status_last_updated time. Any unused segment in interval with used_status_last_updated no later than this time will be included in the kill task. Segments without used_status_last_updated time (due to an upgrade from legacy Druid) will have maxUsedStatusLastUpdatedTime ignored
      Returns:
      future with task ID

    • cancelTask

      com.google.common.util.concurrent.ListenableFuture<Void> cancelTask(String taskId)
      Cancel a task.
      Parameters:
      taskId - task ID

    • taskStatuses

      com.google.common.util.concurrent.ListenableFuture<CloseableIterator<TaskStatusPlus>> taskStatuses(@Nullable String state, @Nullable String dataSource, @Nullable Integer maxCompletedTasks)
      Return TaskStatusPlus for all tasks matching a set of optional search parameters. Complete tasks are returned in descending order by creation timestamp. Active tasks are returned in no particular order.
      Parameters:
      state - task state: may be "pending", "waiting", "running", or "complete"
      dataSource - datasource
      maxCompletedTasks - maximum number of completed tasks to return. If zero, no complete tasks are returned. If null, all complete tasks within druid.indexer.storage.recentlyFinishedThreshold are returned. This parameter does not affect the number of active tasks returned.
      Returns:
      list of tasks that match the search parameters

    • taskStatuses

      com.google.common.util.concurrent.ListenableFuture<Map<String,TaskStatus>> taskStatuses(Set<String> taskIds)
      Return TaskStatus for a set of task IDs.
      Parameters:
      taskIds - task IDs
      Returns:
      map of task ID to status for known tasks. Unknown tasks are not included in the returned map.

    • taskStatus

      com.google.common.util.concurrent.ListenableFuture<TaskStatusResponse> taskStatus(String taskId)
      Returns TaskStatusResponse for a particular task ID. This includes somewhat more information than the TaskStatus returned by taskStatuses(Set).

    • taskReportAsMap

      com.google.common.util.concurrent.ListenableFuture<TaskReport.ReportMap> taskReportAsMap(String taskId)
      Returns the report object for a task as a map. Certain task types offer live reports; for these task types, this method may return a task report while the task is running. Certain task types only write reports upon successful completion. Certain other task types do not write reports at all. Returns a HttpResponseException with code Response.Status.NOT_FOUND if there is no report available for some reason.

    • taskPayload

      com.google.common.util.concurrent.ListenableFuture<TaskPayloadResponse> taskPayload(String taskId)
      Returns the payload for a task as an instance of ClientTaskQuery. This method only works for tasks that have a ClientTaskQuery model or are subclasses of ClientTaskQuery.

    • postSupervisor

      com.google.common.util.concurrent.ListenableFuture<Map<String,String>> postSupervisor(SupervisorSpec supervisor)
      Submits a supervisor spec to the Overlord.

      API: /druid/indexer/v1/supervisor

      Returns:
      Map containing a single entry "id"

    • terminateSupervisor

      com.google.common.util.concurrent.ListenableFuture<Map<String,String>> terminateSupervisor(String supervisorId)
      Shuts down a supervisor.

      API: /druid/indexer/v1/supervisor/<id>/terminate

      Returns:
      Map containing a single entry "id"

    • supervisorStatuses

      com.google.common.util.concurrent.ListenableFuture<CloseableIterator<SupervisorStatus>> supervisorStatuses()
      Returns all current supervisor statuses.

    • findLockedIntervals

      com.google.common.util.concurrent.ListenableFuture<Map<String,List<org.joda.time.Interval>>> findLockedIntervals(List<LockFilterPolicy> lockFilterPolicies)
      Returns a list of intervals locked by higher priority conflicting lock types
      Parameters:
      lockFilterPolicies - List of all filters for different datasources
      Returns:
      Map from datasource name to list of intervals locked by tasks that have a conflicting lock type with priority greater than or equal to the minTaskPriority for that datasource.

    • killPendingSegments

      com.google.common.util.concurrent.ListenableFuture<Integer> killPendingSegments(String dataSource, org.joda.time.Interval interval)
      Deletes pending segment records from the metadata store for a particular datasource. Records with created_date within the provided interval are deleted; other records are left alone. Deletion is done synchronously with the API call. When the future resolves, the deletion is complete.
      Parameters:
      dataSource - datasource name
      interval - created time interval
      Returns:
      number of pending segments deleted

    • getWorkers

      com.google.common.util.concurrent.ListenableFuture<List<IndexingWorkerInfo>> getWorkers()
      Returns information about workers.

    • getTotalWorkerCapacity

      com.google.common.util.concurrent.ListenableFuture<IndexingTotalWorkerCapacityInfo> getTotalWorkerCapacity()
      Returns total worker capacity details.

    • isCompactionSupervisorEnabled

      com.google.common.util.concurrent.ListenableFuture<Boolean> isCompactionSupervisorEnabled()
      Checks if compaction supervisors are enabled on the Overlord. When this returns true, the Coordinator does not run CompactSegments duty.

      API: GET /druid/indexer/v1/compaction/isSupervisorEnabled

    • getClusterCompactionConfig

      com.google.common.util.concurrent.ListenableFuture<ClusterCompactionConfig> getClusterCompactionConfig()
      Gets the current cluster-level compaction config.

      API: GET /druid/indexer/v1/compaction/config/cluster

    • updateClusterCompactionConfig

      com.google.common.util.concurrent.ListenableFuture<UpdateResponse> updateClusterCompactionConfig(ClusterCompactionConfig config)
      Gets the current cluster-level compaction config.

      API: POST /druid/indexer/v1/compaction/config/cluster

      Returns:
      UpdateResponse containing the success status of this operation.

    • markNonOvershadowedSegmentsAsUsed

      com.google.common.util.concurrent.ListenableFuture<SegmentUpdateResponse> markNonOvershadowedSegmentsAsUsed(String dataSource)
      Marks all non-overshadowed segments of the datasource as used.

      API: POST /druid/indexer/v1/datasources/{dataSourceName}

    • markNonOvershadowedSegmentsAsUsed

      com.google.common.util.concurrent.ListenableFuture<SegmentUpdateResponse> markNonOvershadowedSegmentsAsUsed(String dataSource, SegmentsToUpdateFilter filter)
      Marks non-overshadowed segments that satisfy the given filter as used.

      API: POST /druid/indexer/v1/datasources/{dataSourceName}/markUsed

      Parameters:
      filter - Must be non-null

    • markSegmentAsUsed

      com.google.common.util.concurrent.ListenableFuture<SegmentUpdateResponse> markSegmentAsUsed(SegmentId segmentId)
      Marks the given segment as used.

      API: POST /druid/indexer/v1/datasources/{dataSourceName}/segments/{segmentId}

    • markSegmentsAsUnused

      com.google.common.util.concurrent.ListenableFuture<SegmentUpdateResponse> markSegmentsAsUnused(String dataSource)
      Marks all non-overshadowed segments of the datasource as unused.

      API: DELETE /druid/indexer/v1/datasources/{dataSourceName}

    • markSegmentsAsUnused

      com.google.common.util.concurrent.ListenableFuture<SegmentUpdateResponse> markSegmentsAsUnused(String dataSource, SegmentsToUpdateFilter filter)
      Marks non-overshadowed segments that satisfy the given filter as unused.

      API: POST /druid/indexer/v1/datasources/{dataSourceName}/markUnused

      Parameters:
      filter - Must be non-null

    • markSegmentAsUnused

      com.google.common.util.concurrent.ListenableFuture<SegmentUpdateResponse> markSegmentAsUnused(SegmentId segmentId)
      Marks the given segment as unused.

      API: DELETE /druid/indexer/v1/datasources/{dataSourceName}/segments/{segmentId}

    • withRetryPolicy

      OverlordClient withRetryPolicy(ServiceRetryPolicy retryPolicy)
      Returns a copy of this client with a different retry policy.