Package org.apache.druid.metadata

Class SQLMetadataConnector

java.lang.Object
org.apache.druid.metadata.SQLMetadataConnector
All Implemented Interfaces:
MetadataStorageConnector
Direct Known Subclasses:
DerbyConnector
public abstract class SQLMetadataConnector extends Object implements MetadataStorageConnector

  • Constructor Details

  • Method Details

    • getPayloadType

      public String getPayloadType()
      SQL type to use for payload data (e.g. JSON blobs). Must be a binary type, which values can be accessed using ResultSet.getBytes()

      The resulting string will be interpolated into the table creation statement, e.g. CREATE TABLE druid_table ( payload NOT NULL, ... )

      Returns:
      String representing the SQL type

    • getCollation

      public String getCollation()
      The character set and collation for case-sensitive nonbinary string comparison
      Returns:
      the collation for the character set

    • getSerialType

      public abstract String getSerialType()
      Auto-incrementing integer SQL type to use for IDs. The returned string is interpolated into the table creation statement as follows: 
       CREATE TABLE druid_table (
   id <serial-type> NOT NULL,
   col_2 VARCHAR(255) NOT NULL,
   col_3 VARCHAR(255) NOT NULL
   ...
 )
      Returns:
      String representing auto-incrementing SQL integer type to use for IDs.

    • getStreamingFetchSize

      public abstract int getStreamingFetchSize()
      Returns the value that should be passed to statement.setFetchSize to ensure results are streamed back from the database instead of fetching the entire result set in memory.
      Returns:
      optimal fetch size to stream results back

    • getQuoteString

      public abstract String getQuoteString()
      Returns:
      the string that should be used to quote string fields

    • getValidationQuery

      public String getValidationQuery()

    • tableExists

      public abstract boolean tableExists(org.skife.jdbi.v2.Handle handle, String tableName)

    • limitClause

      public abstract String limitClause(int limit)

    • retryWithHandle

      public <T> T retryWithHandle(org.skife.jdbi.v2.tweak.HandleCallback<T> callback, com.google.common.base.Predicate<Throwable> myShouldRetry)

    • retryWithHandle

      public <T> T retryWithHandle(org.skife.jdbi.v2.tweak.HandleCallback<T> callback)

    • retryTransaction

      public <T> T retryTransaction(org.skife.jdbi.v2.TransactionCallback<T> callback, int quietTries, int maxTries)

    • isTransientException

      public final boolean isTransientException(Throwable e)

    • connectorIsTransientException

      protected boolean connectorIsTransientException(Throwable e)
      Vendor specific errors that are not covered by isTransientException(Throwable)

    • isRootCausePacketTooBigException

      protected boolean isRootCausePacketTooBigException(Throwable t)
      Checks if the root cause of the given exception is a PacketTooBigException.
      Returns:
      false by default. Specific implementations should override this method to correctly classify their packet exceptions.

    • createTable

      public void createTable(String tableName, Iterable<String> sql)
      Creates the given table and indexes if the table doesn't already exist.

    • createPendingSegmentsTable

      public void createPendingSegmentsTable(String tableName)

    • createDataSourceTable

      public void createDataSourceTable(String tableName)
      The dataSource column stores the supervisor ID. It has not been renamed to retain backwards compatibility. Supervisors created without an explicit supervisor id default to using the datasource name.

    • createSegmentTable

      public void createSegmentTable(String tableName)

    • createRulesTable

      public void createRulesTable(String tableName)

    • createConfigTable

      public void createConfigTable(String tableName)

    • prepareTaskEntryTable

      public void prepareTaskEntryTable(String tableName)

    • createEntryTable

      public void createEntryTable(String tableName)

    • createLockTable

      public void createLockTable(String tableName, String entryTypeName)

    • createSupervisorsTable

      public void createSupervisorsTable(String tableName)

    • alterSegmentTable

      protected void alterSegmentTable()
      Adds new columns (used_status_last_updated) to the "segments" table. Conditionally, add schema_fingerprint, num_rows columns.

    • insertOrUpdate

      public Void insertOrUpdate(String tableName, String keyColumn, String valueColumn, String key, byte[] value)
      Specified by:
      insertOrUpdate in interface MetadataStorageConnector

    • compareAndSwap

      public boolean compareAndSwap(List<MetadataCASUpdate> updates)
      Specified by:
      compareAndSwap in interface MetadataStorageConnector

    • getDBI

      public abstract org.skife.jdbi.v2.DBI getDBI()

    • createDataSourceTable

      public void createDataSourceTable()
      Specified by:
      createDataSourceTable in interface MetadataStorageConnector

    • createPendingSegmentsTable

      public void createPendingSegmentsTable()
      Specified by:
      createPendingSegmentsTable in interface MetadataStorageConnector

    • createSegmentTable

      public void createSegmentTable()
      Specified by:
      createSegmentTable in interface MetadataStorageConnector

    • createUpgradeSegmentsTable

      public void createUpgradeSegmentsTable()
      Specified by:
      createUpgradeSegmentsTable in interface MetadataStorageConnector

    • createRulesTable

      public void createRulesTable()
      Specified by:
      createRulesTable in interface MetadataStorageConnector

    • createConfigTable

      public void createConfigTable()
      Specified by:
      createConfigTable in interface MetadataStorageConnector

    • createTaskTables

      public void createTaskTables()
      Specified by:
      createTaskTables in interface MetadataStorageConnector

    • createSupervisorsTable

      public void createSupervisorsTable()
      Specified by:
      createSupervisorsTable in interface MetadataStorageConnector

    • lookup

      @Nullable public byte[] lookup(String tableName, String keyColumn, String valueColumn, String key)
      Specified by:
      lookup in interface MetadataStorageConnector

    • lookupWithHandle

      @Nullable public byte[] lookupWithHandle(org.skife.jdbi.v2.Handle handle, String tableName, String keyColumn, String valueColumn, String key)

    • getConfig

      public MetadataStorageConnectorConfig getConfig()

    • makeDatasource

      protected static org.apache.commons.dbcp2.BasicDataSource makeDatasource(MetadataStorageConnectorConfig connectorConfig, String validationQuery)

    • getDatasource

      protected org.apache.commons.dbcp2.BasicDataSource getDatasource()

    • retryReadOnlyTransaction

      public final <T> T retryReadOnlyTransaction(org.skife.jdbi.v2.TransactionCallback<T> callback, int quietTries, int maxTries)

    • inReadOnlyTransaction

      public final <T> T inReadOnlyTransaction(org.skife.jdbi.v2.TransactionCallback<T> callback)

    • createAuditTable

      public void createAuditTable()
      Specified by:
      createAuditTable in interface MetadataStorageConnector

    • deleteAllRecords

      public void deleteAllRecords(String tableName)
      Specified by:
      deleteAllRecords in interface MetadataStorageConnector

    • createSegmentSchemaTable

      public void createSegmentSchemaTable(String tableName)

    • createSegmentSchemasTable

      public void createSegmentSchemasTable()
      Specified by:
      createSegmentSchemasTable in interface MetadataStorageConnector

    • getIndexOnTable

      public Set<String> getIndexOnTable(String tableName)
      Get the Set of the index on given table
      Parameters:
      tableName - name of the table to fetch the index map
      Returns:
      Set of the uppercase index names, returns empty set if table does not exist

    • getIndexInfo

      public ResultSet getIndexInfo(DatabaseMetaData databaseMetaData, String tableName) throws SQLException
      Get the ResultSet for indexInfo for given table
      Parameters:
      databaseMetaData - DatabaseMetaData
      tableName - Name of table
      Returns:
      ResultSet with index info
      Throws:
      SQLException

    • createIndex

      public void createIndex(String tableName, String indexName, List<String> indexCols, Set<String> createdIndexSet)
      create index on the table with retry if not already exist, to be called after createTable
      Parameters:
      tableName - Name of the table to create index on
      indexName - case-insensitive string index name, it helps to check the existing index on table
      indexCols - List of columns to be indexed on
      createdIndexSet -

    • tableHasColumn

      protected boolean tableHasColumn(String tableName, String columnName)
      Checks table metadata to determine if the given column exists in the table.
      Returns:
      true if the column exists in the table, false otherwise