org.apache.hadoop.hive.metastore

Class DatabaseProduct

java.lang.Object

org.apache.hadoop.hive.metastore.DatabaseProduct

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

public class DatabaseProduct
extends Object
implements org.apache.hadoop.conf.Configurable

Database product inferred via JDBC. Encapsulates all SQL logic associated with the database product. This class is a singleton, which is instantiated the first time method determineDatabaseProduct is invoked. Tests that need to create multiple instances can use the reset method

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DatabaseProduct.DbType

Field Summary

Fields

Modifier and Type	Field and Description
static DatabaseProduct.DbType	dbType
static String	DERBY_NAME
static String	MARIADB_NAME
static String	MYSQL_NAME
static String	ORACLE_NAME
static String	POSTGRESQL_NAME
static String	SQL_SERVER_NAME
static String	UNDEFINED_NAME

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	DatabaseProduct() Protected constructor for singleton class

Method Summary

Modifier and Type	Method and Description
String	addEscapeCharacters(String s)
String	addForUpdateClause(String selectStatement)
String	addLimitClause(int numRows, String noSelectsqlQuery) Add a limit clause to a given query
List<String>	createInsertValuesStmt(String tblColumns, List<String> rows, List<Integer> rowsCountInStmts, org.apache.hadoop.conf.Configuration conf) Generates "Insert into T(a,b,c) values(1,2,'f'),(3,4,'c')" for appropriate DB.
static DatabaseProduct	determineDatabaseProduct(String productName, org.apache.hadoop.conf.Configuration conf) Determine the database product type
String	getBatchInsertQuery(String tableName, String columns, String rowFormat, int batchCount) Gets the multiple row insert query for the given table with specified columns and row format
Object	getBoolean(boolean val) Gets the boolean value specific to database for the given input
org.apache.hadoop.conf.Configuration	getConf()
Map<String,String>	getDataSourceProperties() Get datasource properties for connection pools
String	getDBTime() Returns the query to fetch the current timestamp in milliseconds in the database
String	getHiveSchemaPostfix()
int	getMaxRows(int batch, int paramSize) Get the max rows in a query with paramSize.
String	getMillisAfterEpochFn() Get database specific function which returns the milliseconds value after the epoch.
String	getPrepareTxnStmt() Returns db-specific logic to be executed at the beginning of a transaction.
List<String>	getResetTxnSequenceStmts()
String	getTruncateStatement(String name)
String	getTxnSeedFn(long seedTxnId)
protected boolean	hasJoinOperationOrderBug() Whether the RDBMS has a bug in join and filter operation order described in DERBY-6358.
boolean	isCUSTOM()
boolean	isDeadlock(SQLException e)
boolean	isDERBY()
boolean	isDuplicateKeyError(Throwable t)
boolean	isMYSQL()
boolean	isORACLE()
boolean	isPOSTGRES()
static boolean	isRecoverableException(Throwable t)
boolean	isSQLSERVER()
boolean	isTableNotExistsError(Throwable t) Is the given exception a table not found exception
boolean	isUNDEFINED()
String	isWithinCheckInterval(String expr, long intervalInSeconds) Given an expr (such as a column name) returns a database specific SQL function string which when executed returns if the value of the expr is within the given interval of the current timestamp in the database
void	lockInternal() lockInternal() and unlockInternal() are used to serialize those operations that require Select ...
String	lockTable(String txnLockTable, boolean shared) Returns the SQL query to lock the given table name in either shared/exclusive mode
protected boolean	needsInBatching() Whether the RDBMS has restrictions on IN list size (explicit, or poor perf-based).
static void	reset()
void	setConf(org.apache.hadoop.conf.Configuration c)
boolean	supportsGetGeneratedKeys() Checks if the dbms supports the getGeneratedKeys for multiline insert statements.
protected String	toDate(String tableValue)
protected String	toTimestamp(String tableValue)
void	unlockInternal()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

dbType

public static DatabaseProduct.DbType dbType

DERBY_NAME

public static final String DERBY_NAME

See Also:

Constant Field Values

SQL_SERVER_NAME

public static final String SQL_SERVER_NAME

See Also:

Constant Field Values

MYSQL_NAME

public static final String MYSQL_NAME

See Also:

Constant Field Values

MARIADB_NAME

public static final String MARIADB_NAME

See Also:

Constant Field Values

POSTGRESQL_NAME

public static final String POSTGRESQL_NAME

See Also:

Constant Field Values

ORACLE_NAME

public static final String ORACLE_NAME

See Also:

Constant Field Values

UNDEFINED_NAME

public static final String UNDEFINED_NAME

See Also:

Constant Field Values

Constructor Detail

DatabaseProduct

protected DatabaseProduct()

Protected constructor for singleton class

Method Detail

determineDatabaseProduct

public static DatabaseProduct determineDatabaseProduct(String productName,
                                                       org.apache.hadoop.conf.Configuration conf)

Determine the database product type

Parameters:

productName - string to defer database connection

Returns:

database product type

isRecoverableException

public static boolean isRecoverableException(Throwable t)

isDERBY

public final boolean isDERBY()

isMYSQL

public final boolean isMYSQL()

isORACLE

public final boolean isORACLE()

isSQLSERVER

public final boolean isSQLSERVER()

isPOSTGRES

public final boolean isPOSTGRES()

isCUSTOM

public final boolean isCUSTOM()

isUNDEFINED

public final boolean isUNDEFINED()

isDeadlock

public boolean isDeadlock(SQLException e)

isTableNotExistsError

public boolean isTableNotExistsError(Throwable t)

Is the given exception a table not found exception

Parameters:

t - Exception

Returns:

needsInBatching

protected boolean needsInBatching()

Whether the RDBMS has restrictions on IN list size (explicit, or poor perf-based).

hasJoinOperationOrderBug

protected boolean hasJoinOperationOrderBug()

Whether the RDBMS has a bug in join and filter operation order described in DERBY-6358.

getHiveSchemaPostfix

public String getHiveSchemaPostfix()

reset

public static void reset()

toDate

protected String toDate(String tableValue)

toTimestamp

protected String toTimestamp(String tableValue)

getPrepareTxnStmt

public String getPrepareTxnStmt()

Returns db-specific logic to be executed at the beginning of a transaction. Used in pooled connections.

getTxnSeedFn

public String getTxnSeedFn(long seedTxnId)

getMillisAfterEpochFn

public String getMillisAfterEpochFn()
                             throws MetaException

Get database specific function which returns the milliseconds value after the epoch.

Throws:

MetaException - For unknown database type.

getDBTime

public String getDBTime()
                 throws MetaException

Returns the query to fetch the current timestamp in milliseconds in the database

Throws:

MetaException - when the dbType is unknown.

isWithinCheckInterval

public String isWithinCheckInterval(String expr,
                                    long intervalInSeconds)
                             throws MetaException

Given an expr (such as a column name) returns a database specific SQL function string which when executed returns if the value of the expr is within the given interval of the current timestamp in the database

Parameters:

expr -

intervalInSeconds -

Returns:

Throws:

MetaException

addForUpdateClause

public String addForUpdateClause(String selectStatement)
                          throws MetaException

Throws:

MetaException

addLimitClause

public String addLimitClause(int numRows,
                             String noSelectsqlQuery)
                      throws MetaException

Add a limit clause to a given query

Parameters:

numRows -

noSelectsqlQuery -

Returns:

Throws:

MetaException

lockTable

public String lockTable(String txnLockTable,
                        boolean shared)
                 throws MetaException

Returns the SQL query to lock the given table name in either shared/exclusive mode

Parameters:

txnLockTable -

shared -

Returns:

Throws:

MetaException

getResetTxnSequenceStmts

public List<String> getResetTxnSequenceStmts()

getTruncateStatement

public String getTruncateStatement(String name)

supportsGetGeneratedKeys

public boolean supportsGetGeneratedKeys()
                                 throws MetaException

Checks if the dbms supports the getGeneratedKeys for multiline insert statements.

Returns:

true if supports

Throws:

MetaException

isDuplicateKeyError

public boolean isDuplicateKeyError(Throwable t)

createInsertValuesStmt

public List<String> createInsertValuesStmt(String tblColumns,
                                           List<String> rows,
                                           List<Integer> rowsCountInStmts,
                                           org.apache.hadoop.conf.Configuration conf)

Generates "Insert into T(a,b,c) values(1,2,'f'),(3,4,'c')" for appropriate DB.

Parameters:

tblColumns - e.g. "T(a,b,c)"

rows - e.g. list of Strings like 3,4,'d'

rowsCountInStmts - Output the number of rows in each insert statement returned.

conf -

Returns:

fully formed INSERT INTO ... statements

addEscapeCharacters

public String addEscapeCharacters(String s)

getDataSourceProperties

public Map<String,String> getDataSourceProperties()

Get datasource properties for connection pools

getBatchInsertQuery

public String getBatchInsertQuery(String tableName,
                                  String columns,
                                  String rowFormat,
                                  int batchCount)

Gets the multiple row insert query for the given table with specified columns and row format

Parameters:

tableName - table name to be used in query

columns - comma separated column names string

rowFormat - values format string used in the insert query. Format is like (?,?...?) and the number of question marks in the format is equal to number of column names in the columns argument

batchCount - number of rows in the query

Returns:

database specific multiple row insert query

getBoolean

public Object getBoolean(boolean val)

Gets the boolean value specific to database for the given input

Parameters:

val - boolean value

Returns:

database specific value

getMaxRows

public int getMaxRows(int batch,
                      int paramSize)

Get the max rows in a query with paramSize.

Parameters:

batch - the configured batch size

paramSize - the parameter size in a query statement

Returns:

the max allowed rows in a query

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

setConf

public void setConf(org.apache.hadoop.conf.Configuration c)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

lockInternal

public void lockInternal()

lockInternal() and unlockInternal() are used to serialize those operations that require Select ... For Update to sequence operations properly. In practice that means when running with Derby database. See more notes at class level.

unlockInternal

public void unlockInternal()