org.apache.hadoop.hive.metastore

Class PersistenceManagerProvider

java.lang.Object

org.apache.hadoop.hive.metastore.PersistenceManagerProvider

public class PersistenceManagerProvider
extends Object

This class is a wrapper class around PersistenceManagerFactory and its properties These objects are static and need to be carefully modified together such that there are no race-conditions when updating them. Additionally, this class provides thread-safe methods to get PersistenceManager instances from the current PersistenceManagerFactory. The most common usage of this class is to create a PersistenceManager from existing PersistenceManagerFactory PersistenceManagerFactory properties are modified less often and hence the update pmf properties can make use of read/write locks such that it is only blocking when current properties change.

Method Summary

Modifier and Type	Method and Description
static void	clearOutPmfClassLoaderCache() Removed cached classloaders from DataNucleus DataNucleus caches classloaders in NucleusContext.
static void	closePmfIfNeeded(boolean forCompactor, boolean shouldClose) Close the corresponding PersistenceManagerFactory
static void	closePmfInternal(javax.jdo.PersistenceManagerFactory pmf) When closing PersistenceManagerFactory, the connection factory should be closed as well, otherwise the underlying dangling connections would be accumulated resulting to connection leaks.
static javax.jdo.PersistenceManager	getPersistenceManager() creates a PersistenceManager instance for the current PersistenceManagerFactory.
static javax.jdo.PersistenceManager	getPersistenceManager(boolean forCompactor)
static String	getProperty(String key)
static void	setTwoMetastoreTesting(boolean twoMetastoreTesting) To make possible to run multiple metastore in unit test
static void	updatePmfProperties(org.apache.hadoop.conf.Configuration conf) This method updates the PersistenceManagerFactory and its properties if the given configuration is different from its current set of properties.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

closePmfInternal

public static void closePmfInternal(javax.jdo.PersistenceManagerFactory pmf)

When closing PersistenceManagerFactory, the connection factory should be closed as well, otherwise the underlying dangling connections would be accumulated resulting to connection leaks. See TestDataSourceProviderFactory#testClosePersistenceManagerProvider for details.

Parameters:

pmf - the PersistenceManagerFactory tended to be closed

closePmfIfNeeded

public static void closePmfIfNeeded(boolean forCompactor,
                                    boolean shouldClose)

Close the corresponding PersistenceManagerFactory

Parameters:

forCompactor - true for compactorPmf, otherwise pmf

shouldClose - close the PersistenceManagerFactory if true

updatePmfProperties

public static void updatePmfProperties(org.apache.hadoop.conf.Configuration conf)

This method updates the PersistenceManagerFactory and its properties if the given configuration is different from its current set of properties. Most common case is that the persistenceManagerFactory properties do not change, and hence this method is optimized to be non-blocking in such cases. However, if the properties are different, this method blocks other threads until the properties are updated, current pmf is closed and a new pmf is re-initialized. Note that when a PersistenceManagerFactory is re-initialized all the PersistenceManagers which are instantiated using old factory become invalid and will throw JDOUserException. Hence it is recommended that this method is called in the setup/init phase of the Metastore service when there are no other active threads serving clients.

Parameters:

conf - Configuration which provides the datanucleus/datasource properties for comparison

clearOutPmfClassLoaderCache

public static void clearOutPmfClassLoaderCache()

Removed cached classloaders from DataNucleus DataNucleus caches classloaders in NucleusContext. In UDFs, this can result in classloaders not getting GCed resulting in PermGen leaks. This is particularly an issue when using embedded metastore with HiveServer2, since the current classloader gets modified with each new add jar, becoming the classloader for downstream classes, which DataNucleus ends up using. The NucleusContext cache gets freed up only on calling a close on it. We're not closing NucleusContext since it does a bunch of other things which we don't want. We're not clearing the cache HashMap by calling HashMap#clear to avoid concurrency issues.

getPersistenceManager

public static javax.jdo.PersistenceManager getPersistenceManager()

creates a PersistenceManager instance for the current PersistenceManagerFactory. Note that this acquires a read-lock on PersistenceManagerFactory so that this method will block if any other thread is actively, (re-)initializing PersistenceManagerFactory when this method is called Note that this method throws a RuntimeException, if PersistenceManagerFactory is not yet initialized.

Returns:

PersistenceManager from the current PersistenceManagerFactory instance

getPersistenceManager

public static javax.jdo.PersistenceManager getPersistenceManager(boolean forCompactor)

setTwoMetastoreTesting

public static void setTwoMetastoreTesting(boolean twoMetastoreTesting)

To make possible to run multiple metastore in unit test

Parameters:

twoMetastoreTesting - if we are using multiple metastore in unit test

getProperty

public static String getProperty(String key)