org.apache.nifi.dbcp.hive

Class HiveConnectionPool

java.lang.Object

org.apache.nifi.components.AbstractConfigurableComponent

org.apache.nifi.controller.AbstractControllerService

org.apache.nifi.dbcp.hive.HiveConnectionPool

All Implemented Interfaces:

ConfigurableComponent, ControllerService, DBCPService, HiveDBCPService

@RequiresInstanceClassLoading
 @Tags(value={"hive","dbcp","jdbc","database","connection","pooling","store"})
 @CapabilityDescription(value="Provides Database Connection Pooling Service for Apache Hive. Connections can be asked from pool and returned after usage.")
 @DeprecationNotice(classNames="org.apache.nifi.dbcp.hive.Hive3ConnectionPool")
public class HiveConnectionPool
extends AbstractControllerService
implements HiveDBCPService

Implementation for Database Connection Pooling Service used for Apache Hive connections. Apache DBCP is used for connection pooling functionality.

Field Summary

Fields

Modifier and Type	Field and Description
private static String	ALLOW_EXPLICIT_KEYTAB
private String	connectionUrl
static PropertyDescriptor	DATABASE_URL
private org.apache.commons.dbcp2.BasicDataSource	dataSource
static PropertyDescriptor	DB_PASSWORD
static PropertyDescriptor	DB_USER
private static String	DEFAULT_MAX_CONN_LIFETIME
static PropertyDescriptor	HIVE_CONFIGURATION_RESOURCES
private HiveConfigurator	hiveConfigurator
(package private) static PropertyDescriptor	KERBEROS_CREDENTIALS_SERVICE
private File	kerberosConfigFile
private KerberosProperties	kerberosProperties
private AtomicReference<KerberosUser>	kerberosUserReference
static PropertyDescriptor	MAX_CONN_LIFETIME
static PropertyDescriptor	MAX_TOTAL_CONNECTIONS
static PropertyDescriptor	MAX_WAIT_TIME
private List<PropertyDescriptor>	properties
private org.apache.hadoop.security.UserGroupInformation	ugi
static PropertyDescriptor	VALIDATION_QUERY
private AtomicReference<ValidationResources>	validationResourceHolder

Constructor Summary

Constructors

Constructor and Description
HiveConnectionPool()

Method Summary

Modifier and Type	Method and Description
protected Collection<ValidationResult>	customValidate(ValidationContext validationContext)
private long	extractMillisWithInfinite(PropertyValue prop)
Connection	getConnection()
String	getConnectionURL()
protected List<PropertyDescriptor>	getSupportedPropertyDescriptors()
protected void	init(ControllerServiceInitializationContext context)
(package private) boolean	isAllowExplicitKeytab()
void	onConfigured(ConfigurationContext context) Configures connection pool by creating an instance of the BasicDataSource based on configuration provided with ConfigurationContext.
void	shutdown() Shutdown pool, close all open connections.
String	toString()

Methods inherited from class org.apache.nifi.controller.AbstractControllerService

abstractClearConfigContext, abstractStoreConfigContext, disabled, enabled, getConfigurationContext, getControllerServiceLookup, getIdentifier, getLogger, getNodeTypeProvider, getProperty, getStateManager, initialize, isEnabled

Methods inherited from class org.apache.nifi.components.AbstractConfigurableComponent

equals, getPropertyDescriptor, getPropertyDescriptors, getSupportedDynamicPropertyDescriptor, hashCode, onPropertyModified, validate

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.nifi.dbcp.DBCPService

getConnection, getFlowFileFilter, getFlowFileFilter

Methods inherited from interface org.apache.nifi.controller.ControllerService

initialize, isStateful

Methods inherited from interface org.apache.nifi.components.ConfigurableComponent

getIdentifier, getPropertyDescriptor, getPropertyDescriptors, onPropertyModified, validate

Field Detail

ALLOW_EXPLICIT_KEYTAB

private static final String ALLOW_EXPLICIT_KEYTAB

See Also:

Constant Field Values

DEFAULT_MAX_CONN_LIFETIME

private static final String DEFAULT_MAX_CONN_LIFETIME

See Also:

Constant Field Values

DATABASE_URL

public static final PropertyDescriptor DATABASE_URL

HIVE_CONFIGURATION_RESOURCES

public static final PropertyDescriptor HIVE_CONFIGURATION_RESOURCES

DB_USER

public static final PropertyDescriptor DB_USER

DB_PASSWORD

public static final PropertyDescriptor DB_PASSWORD

MAX_WAIT_TIME

public static final PropertyDescriptor MAX_WAIT_TIME

MAX_TOTAL_CONNECTIONS

public static final PropertyDescriptor MAX_TOTAL_CONNECTIONS

MAX_CONN_LIFETIME

public static final PropertyDescriptor MAX_CONN_LIFETIME

VALIDATION_QUERY

public static final PropertyDescriptor VALIDATION_QUERY

KERBEROS_CREDENTIALS_SERVICE

static final PropertyDescriptor KERBEROS_CREDENTIALS_SERVICE

properties

private List<PropertyDescriptor> properties

connectionUrl

private String connectionUrl

validationResourceHolder

private final AtomicReference<ValidationResources> validationResourceHolder

dataSource

private volatile org.apache.commons.dbcp2.BasicDataSource dataSource

hiveConfigurator

private volatile HiveConfigurator hiveConfigurator

ugi

private volatile org.apache.hadoop.security.UserGroupInformation ugi

kerberosUserReference

private final AtomicReference<KerberosUser> kerberosUserReference

kerberosConfigFile

private volatile File kerberosConfigFile

kerberosProperties

private volatile KerberosProperties kerberosProperties

Constructor Detail

HiveConnectionPool

public HiveConnectionPool()

Method Detail

init

protected void init(ControllerServiceInitializationContext context)

Overrides:

init in class AbstractControllerService

getSupportedPropertyDescriptors

protected List<PropertyDescriptor> getSupportedPropertyDescriptors()

Overrides:

getSupportedPropertyDescriptors in class AbstractConfigurableComponent

customValidate

protected Collection<ValidationResult> customValidate(ValidationContext validationContext)

Overrides:

customValidate in class AbstractConfigurableComponent

onConfigured

@OnEnabled
public void onConfigured(ConfigurationContext context)
                             throws InitializationException

Configures connection pool by creating an instance of the BasicDataSource based on configuration provided with ConfigurationContext.

This operation makes no guarantees that the actual connection could be made since the underlying system may still go off-line during normal operation of the connection pool.

As of Apache NiFi 1.5.0, due to changes made to SecurityUtil.loginKerberos(Configuration, String, String), which is used by this class invoking HiveConfigurator.authenticate(Configuration, String, String) to authenticate a principal with Kerberos, Hive controller services no longer use a separate thread to relogin, and instead call UserGroupInformation.checkTGTAndReloginFromKeytab() from getConnection(). The relogin request is performed in a synchronized block to prevent threads from requesting concurrent relogins. For more information, please read the documentation for SecurityUtil.loginKerberos(Configuration, String, String).

In previous versions of NiFi, a KerberosTicketRenewer was started by HiveConfigurator.authenticate(Configuration, String, String, long) when the Hive controller service was enabled. The use of a separate thread to explicitly relogin could cause race conditions with the implicit relogin attempts made by hadoop/Hive code on a thread that references the same UserGroupInformation instance. One of these threads could leave the Subject in UserGroupInformation to be cleared or in an unexpected state while the other thread is attempting to use the Subject, resulting in failed authentication attempts that would leave the Hive controller service in an unrecoverable state.

Parameters:

context - the configuration context

Throws:

InitializationException - if unable to create a database connection

See Also:

SecurityUtil.loginKerberos(Configuration, String, String), HiveConfigurator.authenticate(Configuration, String, String), HiveConfigurator.authenticate(Configuration, String, String, long)

shutdown

@OnDisabled
public void shutdown()

Shutdown pool, close all open connections.

getConnection

public Connection getConnection()
                         throws ProcessException

Specified by:

getConnection in interface DBCPService

Throws:

ProcessException

toString

public String toString()

Overrides:

toString in class AbstractConfigurableComponent

getConnectionURL

public String getConnectionURL()

Specified by:

getConnectionURL in interface HiveDBCPService

isAllowExplicitKeytab

boolean isAllowExplicitKeytab()

extractMillisWithInfinite

private long extractMillisWithInfinite(PropertyValue prop)