Package org.apache.druid.metadata.segment

Class SqlSegmentsMetadataManagerV2

java.lang.Object

org.apache.druid.metadata.segment.SqlSegmentsMetadataManagerV2

All Implemented Interfaces:

SegmentsMetadataManager

public class SqlSegmentsMetadataManagerV2
extends Object
implements SegmentsMetadataManager

Implementation V2 of SegmentsMetadataManager, that can use the segments cached in SegmentMetadataCache to build a DataSourcesSnapshot.

This class acts as a wrapper over SqlSegmentsMetadataManager and the SegmentMetadataCache. If the cache is enabled, an additional poll is not done and the segments already present in the cache are used to build the snapshot. If the SegmentMetadataCache is disabled, the polling is delegated to the legacy implementation in SqlSegmentsMetadataManager.

The Coordinator always uses the snapshot to perform various segment management duties such as loading, balancing, etc. The Overlord uses the snapshot only when compaction supervisors are enabled. Thus, when running the Overlord as a standalone service (i.e. not combined with the Coordinator), startPollingDatabasePeriodically() and stopPollingDatabasePeriodically() are called based on the current state of DruidCompactionConfig.isUseSupervisors().

Constructor Summary

Constructors

Constructor

Description

SqlSegmentsMetadataManagerV2(SegmentMetadataCache segmentMetadataCache, SegmentSchemaCache segmentSchemaCache, SQLMetadataConnector connector, com.google.common.base.Supplier<SegmentsMetadataManagerConfig> managerConfig, com.google.common.base.Supplier<MetadataStorageTablesConfig> tablesConfig, com.google.common.base.Supplier<CentralizedDatasourceSchemaConfig> centralizedDatasourceSchemaConfig, ServiceEmitter serviceEmitter, com.fasterxml.jackson.databind.ObjectMapper jsonMapper)

Method Summary

Modifier and Type	Method	Description
DataSourcesSnapshot	forceUpdateDataSourcesSnapshot()	Forces the manager to poll the metadata store and update its snapshot.
DataSourcesSnapshot	getRecentDataSourcesSnapshot()	Returns the latest snapshot containing all used segments currently cached on the manager.
boolean	isPollingDatabasePeriodically()
void	populateUsedFlagLastUpdatedAsync()	Populates used_status_last_updated column in the segments table iteratively until there are no segments with a NULL value for that column.
void	start()	Initializes the manager when the service is being started.
void	startPollingDatabasePeriodically()	Starts polling segments from the metadata store upon becoming leader.
void	stop()	Cleans up resources when the service is being shut down.
void	stopAsyncUsedFlagLastUpdatedUpdate()
void	stopPollingDatabasePeriodically()	Stops polling segments from the metadata store when leadership is lost.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

SqlSegmentsMetadataManagerV2

public SqlSegmentsMetadataManagerV2(SegmentMetadataCache segmentMetadataCache,
 SegmentSchemaCache segmentSchemaCache,
 SQLMetadataConnector connector,
 com.google.common.base.Supplier<SegmentsMetadataManagerConfig> managerConfig,
 com.google.common.base.Supplier<MetadataStorageTablesConfig> tablesConfig,
 com.google.common.base.Supplier<CentralizedDatasourceSchemaConfig> centralizedDatasourceSchemaConfig,
 ServiceEmitter serviceEmitter,
 com.fasterxml.jackson.databind.ObjectMapper jsonMapper)

Method Details

start

public void start()

Description copied from interface: SegmentsMetadataManager

Initializes the manager when the service is being started.

Specified by:

start in interface SegmentsMetadataManager

stop

public void stop()

Description copied from interface: SegmentsMetadataManager

Cleans up resources when the service is being shut down.

Specified by:

stop in interface SegmentsMetadataManager

startPollingDatabasePeriodically

public void startPollingDatabasePeriodically()

Description copied from interface: SegmentsMetadataManager

Starts polling segments from the metadata store upon becoming leader.

Specified by:

startPollingDatabasePeriodically in interface SegmentsMetadataManager

stopPollingDatabasePeriodically

public void stopPollingDatabasePeriodically()

Description copied from interface: SegmentsMetadataManager

Stops polling segments from the metadata store when leadership is lost.

Specified by:

stopPollingDatabasePeriodically in interface SegmentsMetadataManager

isPollingDatabasePeriodically

public boolean isPollingDatabasePeriodically()

Specified by:

isPollingDatabasePeriodically in interface SegmentsMetadataManager

Returns:

true if currently the leader and polling the metadata store.

getRecentDataSourcesSnapshot

public DataSourcesSnapshot getRecentDataSourcesSnapshot()

Description copied from interface: SegmentsMetadataManager

Returns the latest snapshot containing all used segments currently cached on the manager. This method returns immediately if the poll period has not elapsed since the latest snapshot was built. Otherwise, it blocks until the snapshot has been refreshed.

Specified by:

getRecentDataSourcesSnapshot in interface SegmentsMetadataManager

forceUpdateDataSourcesSnapshot

public DataSourcesSnapshot forceUpdateDataSourcesSnapshot()

Description copied from interface: SegmentsMetadataManager

Forces the manager to poll the metadata store and update its snapshot. If a poll is already in progress, a new poll is not started. This method blocks until the poll finishes and the snapshot has been refreshed.

Specified by:

forceUpdateDataSourcesSnapshot in interface SegmentsMetadataManager

Returns:

The updated snapshot.

populateUsedFlagLastUpdatedAsync

public void populateUsedFlagLastUpdatedAsync()

Description copied from interface: SegmentsMetadataManager

Populates used_status_last_updated column in the segments table iteratively until there are no segments with a NULL value for that column.

Specified by:

populateUsedFlagLastUpdatedAsync in interface SegmentsMetadataManager

stopAsyncUsedFlagLastUpdatedUpdate

public void stopAsyncUsedFlagLastUpdatedUpdate()

Specified by:

stopAsyncUsedFlagLastUpdatedUpdate in interface SegmentsMetadataManager