org.apache.flink.runtime.execution.librarycache

Class BlobLibraryCacheManager

java.lang.Object

org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager

All Implemented Interfaces:

LibraryCacheManager

public class BlobLibraryCacheManager
extends Object
implements LibraryCacheManager

Provides facilities to download a set of libraries (typically JAR files) for a job from a PermanentBlobService and create a class loader with references to them.

Constructor Summary

Constructors

Constructor and Description
BlobLibraryCacheManager(PermanentBlobService blobService, FlinkUserCodeClassLoaders.ResolveOrder classLoaderResolveOrder, String[] alwaysParentFirstPatterns)

Method Summary

Modifier and Type	Method and Description
ClassLoader	getClassLoader(org.apache.flink.api.common.JobID jobId) Returns the user code class loader associated with id.
boolean	hasClassLoader(org.apache.flink.api.common.JobID jobId) True if the LibraryCacheManager has a user code class loader registered for the given job id.
void	registerJob(org.apache.flink.api.common.JobID id, Collection<PermanentBlobKey> requiredJarFiles, Collection<URL> requiredClasspaths) Registers a job with its required jar files and classpaths.
void	registerTask(org.apache.flink.api.common.JobID jobId, ExecutionAttemptID task, Collection<PermanentBlobKey> requiredJarFiles, Collection<URL> requiredClasspaths) Registers a job task execution with its required jar files and classpaths.
void	shutdown() Shutdown method which may release created class loaders.
void	unregisterJob(org.apache.flink.api.common.JobID id) Unregisters a job from the library cache manager.
void	unregisterTask(org.apache.flink.api.common.JobID jobId, ExecutionAttemptID task) Unregisters a job task execution from the library cache manager.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BlobLibraryCacheManager

public BlobLibraryCacheManager(PermanentBlobService blobService,
                               FlinkUserCodeClassLoaders.ResolveOrder classLoaderResolveOrder,
                               String[] alwaysParentFirstPatterns)

Method Detail

registerJob

public void registerJob(org.apache.flink.api.common.JobID id,
                        Collection<PermanentBlobKey> requiredJarFiles,
                        Collection<URL> requiredClasspaths)
                 throws IOException

Description copied from interface: LibraryCacheManager

Registers a job with its required jar files and classpaths. The jar files are identified by their blob keys and downloaded for use by a ClassLoader.

Specified by:

registerJob in interface LibraryCacheManager

Parameters:

id - job ID

requiredJarFiles - collection of blob keys identifying the required jar files

requiredClasspaths - collection of classpaths that are added to the user code class loader

Throws:

IOException - if any error occurs when retrieving the required jar files

See Also:

counterpart of this method

registerTask

public void registerTask(org.apache.flink.api.common.JobID jobId,
                         ExecutionAttemptID task,
                         @Nullable
                         Collection<PermanentBlobKey> requiredJarFiles,
                         @Nullable
                         Collection<URL> requiredClasspaths)
                  throws IOException

Description copied from interface: LibraryCacheManager

Registers a job task execution with its required jar files and classpaths. The jar files are identified by their blob keys and downloaded for use by a ClassLoader.

Specified by:

registerTask in interface LibraryCacheManager

Parameters:

jobId - job ID

requiredJarFiles - collection of blob keys identifying the required jar files

requiredClasspaths - collection of classpaths that are added to the user code class loader

Throws:

IOException - if any error occurs when retrieving the required jar files

See Also:

counterpart of this method

unregisterJob

public void unregisterJob(org.apache.flink.api.common.JobID id)

Description copied from interface: LibraryCacheManager

Unregisters a job from the library cache manager.

Note: this is the counterpart of LibraryCacheManager.registerJob(JobID, Collection, Collection) and it will not remove any job task execution added via LibraryCacheManager.registerTask(JobID, ExecutionAttemptID, Collection, Collection)!

Specified by:

unregisterJob in interface LibraryCacheManager

Parameters:

id - job ID

See Also:

counterpart of this method

unregisterTask

public void unregisterTask(org.apache.flink.api.common.JobID jobId,
                           ExecutionAttemptID task)

Description copied from interface: LibraryCacheManager

Unregisters a job task execution from the library cache manager.

Note: this is the counterpart of LibraryCacheManager.registerTask(JobID, ExecutionAttemptID, Collection, Collection) and it will not remove any job added via LibraryCacheManager.registerJob(JobID, Collection, Collection)!

Specified by:

unregisterTask in interface LibraryCacheManager

Parameters:

jobId - job ID

See Also:

counterpart of this method

getClassLoader

public ClassLoader getClassLoader(org.apache.flink.api.common.JobID jobId)

Description copied from interface: LibraryCacheManager

Returns the user code class loader associated with id.

Specified by:

getClassLoader in interface LibraryCacheManager

Parameters:

jobId - identifying the job

Returns:

ClassLoader which can load the user code

shutdown

public void shutdown()

Description copied from interface: LibraryCacheManager

Shutdown method which may release created class loaders.

Specified by:

shutdown in interface LibraryCacheManager

hasClassLoader

public boolean hasClassLoader(@Nonnull
                              org.apache.flink.api.common.JobID jobId)

Description copied from interface: LibraryCacheManager

True if the LibraryCacheManager has a user code class loader registered for the given job id.

Specified by:

hasClassLoader in interface LibraryCacheManager

Parameters:

jobId - identifying the job for which to check the class loader

Returns:

true if the user code class loader for the given job has been registered. Otherwise false.