org.apache.flink.runtime.execution.librarycache

Class BlobLibraryCacheManager

java.lang.Object

java.util.TimerTask

org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager

All Implemented Interfaces:

Runnable, LibraryCacheManager

public final class BlobLibraryCacheManager
extends TimerTask
implements LibraryCacheManager

For each job graph that is submitted to the system the library cache manager maintains a set of libraries (typically JAR files) which the job requires to run. The library cache manager caches library files in order to avoid unnecessary retransmission of data. It is based on a singleton programming pattern, so there exists at most one library manager at a time.

Constructor Summary

Constructors

Constructor and Description
BlobLibraryCacheManager(BlobService blobService, long cleanupInterval)

Method Summary

Modifier and Type	Method and Description
int	getBlobServerPort()
ClassLoader	getClassLoader(org.apache.flink.api.common.JobID id) Returns the user code class loader associated with id.
File	getFile(BlobKey blobKey) Returns a file handle to the file identified by the blob key.
int	getNumberOfReferenceHolders(org.apache.flink.api.common.JobID jobId)
void	registerJob(org.apache.flink.api.common.JobID id, Collection<BlobKey> requiredJarFiles, Collection<URL> requiredClasspaths) Registers a job with its required jar files and classpaths.
void	registerTask(org.apache.flink.api.common.JobID jobId, ExecutionAttemptID task, Collection<BlobKey> requiredJarFiles, Collection<URL> requiredClasspaths) Registers a job task execution with its required jar files and classpaths.
void	run() Cleans up blobs which are not referenced anymore
void	shutdown() Shutdown method
void	unregisterJob(org.apache.flink.api.common.JobID id) Unregisters a job from the library cache manager.
void	unregisterTask(org.apache.flink.api.common.JobID jobId, ExecutionAttemptID task) Unregisters a job from the library cache manager.

Methods inherited from class java.util.TimerTask

cancel, scheduledExecutionTime

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BlobLibraryCacheManager

public BlobLibraryCacheManager(BlobService blobService,
                               long cleanupInterval)

Method Detail

registerJob

public void registerJob(org.apache.flink.api.common.JobID id,
                        Collection<BlobKey> requiredJarFiles,
                        Collection<URL> requiredClasspaths)
                 throws IOException

Description copied from interface: LibraryCacheManager

Registers a job with its required jar files and classpaths. The jar files are identified by their blob keys.

Specified by:

registerJob in interface LibraryCacheManager

Parameters:

id - job ID

requiredJarFiles - collection of blob keys identifying the required jar files

requiredClasspaths - collection of classpaths that are added to the user code class loader

Throws:

IOException

registerTask

public void registerTask(org.apache.flink.api.common.JobID jobId,
                         ExecutionAttemptID task,
                         Collection<BlobKey> requiredJarFiles,
                         Collection<URL> requiredClasspaths)
                  throws IOException

Description copied from interface: LibraryCacheManager

Registers a job task execution with its required jar files and classpaths. The jar files are identified by their blob keys.

Specified by:

registerTask in interface LibraryCacheManager

Parameters:

jobId - job ID

requiredJarFiles - collection of blob keys identifying the required jar files

requiredClasspaths - collection of classpaths that are added to the user code class loader

Throws:

IOException

unregisterJob

public void unregisterJob(org.apache.flink.api.common.JobID id)

Description copied from interface: LibraryCacheManager

Unregisters a job from the library cache manager.

Specified by:

unregisterJob in interface LibraryCacheManager

Parameters:

id - job ID

unregisterTask

public void unregisterTask(org.apache.flink.api.common.JobID jobId,
                           ExecutionAttemptID task)

Description copied from interface: LibraryCacheManager

Unregisters a job from the library cache manager.

Specified by:

unregisterTask in interface LibraryCacheManager

Parameters:

jobId - job ID

getClassLoader

public ClassLoader getClassLoader(org.apache.flink.api.common.JobID id)

Description copied from interface: LibraryCacheManager

Returns the user code class loader associated with id.

Specified by:

getClassLoader in interface LibraryCacheManager

Parameters:

id - identifying the job

Returns:

ClassLoader which can load the user code

getFile

public File getFile(BlobKey blobKey)
             throws IOException

Description copied from interface: LibraryCacheManager

Returns a file handle to the file identified by the blob key.

Specified by:

getFile in interface LibraryCacheManager

Parameters:

blobKey - identifying the requested file

Returns:

File handle

Throws:

IOException

getBlobServerPort

public int getBlobServerPort()

shutdown

public void shutdown()
              throws IOException

Description copied from interface: LibraryCacheManager

Shutdown method

Specified by:

shutdown in interface LibraryCacheManager

Throws:

IOException

run

public void run()

Cleans up blobs which are not referenced anymore

Specified by:

run in interface Runnable

Specified by:

run in class TimerTask

getNumberOfReferenceHolders

public int getNumberOfReferenceHolders(org.apache.flink.api.common.JobID jobId)