Class AbstractPythonProcessService
- java.lang.Object
-
- de.iip_ecosphere.platform.services.environment.AbstractService
-
- de.iip_ecosphere.platform.services.environment.AbstractRunnablesService
-
- de.iip_ecosphere.platform.services.environment.AbstractPythonProcessService
-
- All Implemented Interfaces:
GenericMultiTypeService,ParameterConfigurerProvider,Service,ServiceBase
- Direct Known Subclasses:
PythonAsyncProcessService,PythonSyncProcessService
public abstract class AbstractPythonProcessService extends AbstractRunnablesService implements GenericMultiTypeService
Generic command-line-based Python integration for multiple data types.- Author:
- Holger Eichelberger, SSE
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description protected static classAbstractPythonProcessService.AbstractTypeInfo<T>Represents an input or output type.private static classAbstractPythonProcessService.ByteArrayReceptionCallbackSelf-registering abstract byte array reception callback.protected static interfaceAbstractPythonProcessService.InputHandlerDefines a function that handles "parsed" input, split into type and serialized data.protected classAbstractPythonProcessService.InTypeInfo<T>Represents an input type.protected classAbstractPythonProcessService.OutTypeInfo<T>Represents an output type.protected static classAbstractPythonProcessService.SyncDataIngestor<D>Ingestor implementation for synchronous processing.
-
Field Summary
Fields Modifier and Type Field Description private java.lang.StringaverageResponseTimeprivate java.util.Map<java.lang.String,de.iip_ecosphere.platform.transport.connectors.ReceptionCallback<?>>callbacksprivate java.io.Filehomeprivate java.util.Map<java.lang.String,AbstractPythonProcessService.InTypeInfo<?>>inTypeInfosprivate java.lang.StringlocationKeyprivate java.util.Map<java.lang.String,AbstractPythonProcessService.OutTypeInfo<?>>outTypeInfosprivate java.util.Map<java.lang.String,ParameterConfigurer<?>>paramConfigurersprivate java.util.List<java.lang.String>pythonArgsprivate java.lang.StringtransportChannelstatic charTYPE_SEPARATOR_CHAR-
Fields inherited from interface de.iip_ecosphere.platform.services.environment.switching.ServiceBase
APPLICATION_SEPARATOR, DEFAULT_APPLICATION_INSTANCE_ID
-
-
Constructor Summary
Constructors Constructor Description AbstractPythonProcessService(YamlService yaml)Creates a service from YAML information.AbstractPythonProcessService(java.lang.String serviceId, java.io.InputStream ymlFile)Creates a service from a service id and a YAML artifact.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description voidaddParameterConfigurer(java.util.function.Consumer<java.util.Map<java.lang.String,ParameterConfigurer<?>>> paramConsumer)Adds parameter configurers via a consumer.<O> voidattachIngestor(java.lang.Class<O> outCls, java.lang.String outTypeName, DataIngestor<O> ingestor)Attaches an asynchronous result data ingestor.protected static java.lang.Stringcompose(java.lang.String typeName, java.lang.String data)Composes a symbolic type name with the string-serialized data.protected java.lang.ProcesscreateAndCustomizeProcess(java.lang.String data, java.util.Map<java.lang.String,java.lang.String> reconfValues)Creates and customizes the process.protected java.lang.ThreadcreateScanInputThread(java.lang.Process proc, AbstractPythonProcessService.InputHandler handler)Creates a thread callingscanInputStream(Process, InputHandler).private voidestablishClientListener(java.lang.String typeName, java.lang.String serverChannel)Establishes a client listener for internal server-client communication via Transport.private voidestablishServerListener(java.lang.String typeName, java.lang.String serverChannel)Establishes a server listener for internal server-client communication via Transport.longgetAvgResponseTime()Returns the average response time for the execution in Python (without transport).protected java.io.FilegetHome()Returns the home directory.AbstractPythonProcessService.InTypeInfo<?>getInTypeInfo(java.lang.String inTypeName)Returns the input type information object for the given symbolic type name.protected java.lang.StringgetLocationKey()Returns the location key for access intoInstalledDependenciesSetup.protected static org.slf4j.LoggergetLogger()Returns the logger.AbstractPythonProcessService.OutTypeInfo<?>getOutTypeInfo(java.lang.String outTypeName)Returns the output type information object for the given symbolic type name.ParameterConfigurer<?>getParameterConfigurer(java.lang.String paramName)Returns the parameter configurer for a given parameter.java.util.Set<java.lang.String>getParameterNames()The set of parameter names.protected java.io.FilegetPythonExecutable()Returns the Python executable, either viaInstalledDependenciesSetupandgetLocationKey()or as fallback viaPythonUtils.getPythonExecutable().private static java.lang.StringgetPythonModule(java.lang.String module, YamlService yaml, java.io.File homePath)Returns the name of the Python module.protected voidhandleErrorStream(java.io.InputStream err)Handles the error stream upon process creation.protected <O> voidhandleResult(java.lang.Class<O> cls, java.lang.String data, java.lang.String typeName)Handles a received processing result and ingests it back asynchronously.protected voidinitializeFrom(YamlService yaml)Does further setup of this instance from the given YAML information.private <I> AbstractPythonProcessService.InTypeInfo<I>obtainInTypeInfo(java.lang.Class<I> cls, java.lang.String typeName)Obtains an input type information object.private <O> AbstractPythonProcessService.OutTypeInfo<O>obtainOutTypeInfo(java.lang.Class<O> cls, java.lang.String typeName)Obtains an output type information object.<I> voidregisterInputTypeTranslator(java.lang.Class<I> inCls, java.lang.String inTypeName, de.iip_ecosphere.platform.transport.serialization.TypeTranslator<I,java.lang.String> inTrans)Adds an input type translator.<O> voidregisterOutputTypeTranslator(java.lang.Class<O> outCls, java.lang.String outTypeName, de.iip_ecosphere.platform.transport.serialization.TypeTranslator<java.lang.String,O> outTrans)Adds an output type translator.protected java.lang.StringscanInputStream(java.lang.Process proc, AbstractPythonProcessService.InputHandler handler)Scans the input stream of the given process for return data.protected ServiceStatestart()Starts the service and the background process.protected booleanstartExecutableByName()Returns whether the Python executable shall be started by name or by full path.protected ServiceStatestop()Stops the service and the background process.protected java.lang.StringtoJson(java.util.Map<java.lang.String,java.lang.String> reconfValues)Turns a map of string values into JSON.-
Methods inherited from class de.iip_ecosphere.platform.services.environment.AbstractRunnablesService
register
-
Methods inherited from class de.iip_ecosphere.platform.services.environment.AbstractService
activate, addConfigurer, addConfigurer, addConfigurer, createInstance, createInstance, createInstance, getDescription, getId, getKind, getName, getNetMgtKeyAddress, getResourceAsStream, getState, getVersion, isDeployable, isTopLevel, notifyReconfigured, passivate, reconf, reconfigure, reconfigure, rollbackReconfigurationOnFailure, setLibJars, setState
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface de.iip_ecosphere.platform.services.environment.GenericMultiTypeService
process, processQuiet, processSync, processSyncQuiet
-
Methods inherited from interface de.iip_ecosphere.platform.services.environment.Service
activate, getDescription, getKind, getName, getNetMgtKeyAddress, getVersion, isDeployable, isTopLevel, migrate, passivate, reconfigure, switchTo, update
-
Methods inherited from interface de.iip_ecosphere.platform.services.environment.switching.ServiceBase
getId, getState, setState
-
-
-
-
Field Detail
-
TYPE_SEPARATOR_CHAR
public static final char TYPE_SEPARATOR_CHAR
- See Also:
- Constant Field Values
-
home
private java.io.File home
-
pythonArgs
private java.util.List<java.lang.String> pythonArgs
-
locationKey
private java.lang.String locationKey
-
transportChannel
private java.lang.String transportChannel
-
outTypeInfos
private java.util.Map<java.lang.String,AbstractPythonProcessService.OutTypeInfo<?>> outTypeInfos
-
inTypeInfos
private java.util.Map<java.lang.String,AbstractPythonProcessService.InTypeInfo<?>> inTypeInfos
-
paramConfigurers
private java.util.Map<java.lang.String,ParameterConfigurer<?>> paramConfigurers
-
callbacks
private java.util.Map<java.lang.String,de.iip_ecosphere.platform.transport.connectors.ReceptionCallback<?>> callbacks
-
averageResponseTime
private java.lang.String averageResponseTime
-
-
Constructor Detail
-
AbstractPythonProcessService
public AbstractPythonProcessService(java.lang.String serviceId, java.io.InputStream ymlFile)Creates a service from a service id and a YAML artifact.- Parameters:
serviceId- the service idymlFile- the YML file containing the YAML artifact with the service descriptor
-
AbstractPythonProcessService
public AbstractPythonProcessService(YamlService yaml)
Creates a service from YAML information.- Parameters:
yaml- the service information as read from YAML. By default, the Python executable is "ServiceEnvironment.py", which can be overridden byYamlProcess.getExecutable().YamlProcess.getHomePath()is set to the home path where the executable was extracted to. Further,YamlProcess.getCmdArg()are taken over if given.
-
-
Method Detail
-
initializeFrom
protected void initializeFrom(YamlService yaml)
Does further setup of this instance from the given YAML information.- Overrides:
initializeFromin classAbstractService- Parameters:
yaml- the service information as read from YAML
-
getHome
protected java.io.File getHome()
Returns the home directory.- Returns:
- the home directory
-
getPythonModule
private static java.lang.String getPythonModule(java.lang.String module, YamlService yaml, java.io.File homePath)Returns the name of the Python module. The default (if not explicitly given) is "ServiceEnvironment.py".- Parameters:
module- the module name, may be empty or nullyaml- the YAML service deployment informationhomePath- optional home path to check for the module first if not given- Returns:
- the Python module name
-
startExecutableByName
protected boolean startExecutableByName()
Returns whether the Python executable shall be started by name or by full path.- Returns:
falsefor starting by full path returned bygetPythonExecutable()
-
scanInputStream
protected java.lang.String scanInputStream(java.lang.Process proc, AbstractPythonProcessService.InputHandler handler)Scans the input stream of the given process for return data.- Parameters:
proc- the Python processhandler- the input handler- Returns:
- the first line if
returnFirst, else null - See Also:
handleResult(Class, String, String)
-
createScanInputThread
protected java.lang.Thread createScanInputThread(java.lang.Process proc, AbstractPythonProcessService.InputHandler handler)Creates a thread callingscanInputStream(Process, InputHandler).- Parameters:
proc- the Python processhandler- the input handler- Returns:
- the thread
-
handleResult
protected <O> void handleResult(java.lang.Class<O> cls, java.lang.String data, java.lang.String typeName)Handles a received processing result and ingests it back asynchronously.- Type Parameters:
O- the data type- Parameters:
cls- the data type classdata- the serialized datatypeName- the data type name as specified in the configuration model
-
toJson
protected java.lang.String toJson(java.util.Map<java.lang.String,java.lang.String> reconfValues)
Turns a map of string values into JSON.- Parameters:
reconfValues- the values to turn into JSON- Returns:
- the JSON representation
-
createAndCustomizeProcess
protected java.lang.Process createAndCustomizeProcess(java.lang.String data, java.util.Map<java.lang.String,java.lang.String> reconfValues) throws java.util.concurrent.ExecutionExceptionCreates and customizes the process.- Parameters:
data- the data to be directly handed to the process via command linereconfValues- values to (re)configure the service environment, may be null for none- Returns:
- the process
- Throws:
java.util.concurrent.ExecutionException- if the process cannot be created
-
getLocationKey
protected java.lang.String getLocationKey()
Returns the location key for access intoInstalledDependenciesSetup.- Returns:
- the key, may be null for fallback
-
getPythonExecutable
protected java.io.File getPythonExecutable()
Returns the Python executable, either viaInstalledDependenciesSetupandgetLocationKey()or as fallback viaPythonUtils.getPythonExecutable().- Returns:
- the Python executable
-
handleErrorStream
protected void handleErrorStream(java.io.InputStream err)
Handles the error stream upon process creation.- Parameters:
err- the process error stream
-
getLogger
protected static org.slf4j.Logger getLogger()
Returns the logger.- Returns:
- the logger
-
registerInputTypeTranslator
public <I> void registerInputTypeTranslator(java.lang.Class<I> inCls, java.lang.String inTypeName, de.iip_ecosphere.platform.transport.serialization.TypeTranslator<I,java.lang.String> inTrans)Adds an input type translator.- Specified by:
registerInputTypeTranslatorin interfaceGenericMultiTypeService- Type Parameters:
I- the input data type- Parameters:
inCls- the class representing the input typeinTypeName- symbolic name ofinCls, e.g. from configuration modelinTrans- the input data type translator- See Also:
registerOutputTypeTranslator(Class, String, TypeTranslator)
-
getInTypeInfo
public AbstractPythonProcessService.InTypeInfo<?> getInTypeInfo(java.lang.String inTypeName)
Returns the input type information object for the given symbolic type name.- Parameters:
inTypeName- the symbolic type name- Returns:
- the information object or null if none was registered
-
obtainInTypeInfo
private <I> AbstractPythonProcessService.InTypeInfo<I> obtainInTypeInfo(java.lang.Class<I> cls, java.lang.String typeName)
Obtains an input type information object.- Type Parameters:
I- the input type- Parameters:
cls- the class representing the typetypeName- the associated symbolic type name- Returns:
- the input type information object, may be retrieved or new
-
getOutTypeInfo
public AbstractPythonProcessService.OutTypeInfo<?> getOutTypeInfo(java.lang.String outTypeName)
Returns the output type information object for the given symbolic type name.- Parameters:
outTypeName- the symbolic type name- Returns:
- the information object or null if none was registered
-
obtainOutTypeInfo
private <O> AbstractPythonProcessService.OutTypeInfo<O> obtainOutTypeInfo(java.lang.Class<O> cls, java.lang.String typeName)
Obtains an output type information object.- Type Parameters:
O- the output type- Parameters:
cls- the class representing the typetypeName- the associated symbolic type name- Returns:
- the output type information object, may be retrieved or new
-
registerOutputTypeTranslator
public <O> void registerOutputTypeTranslator(java.lang.Class<O> outCls, java.lang.String outTypeName, de.iip_ecosphere.platform.transport.serialization.TypeTranslator<java.lang.String,O> outTrans)Adds an output type translator.- Specified by:
registerOutputTypeTranslatorin interfaceGenericMultiTypeService- Type Parameters:
O- the output data type- Parameters:
outCls- the class representing the input typeoutTypeName- symbolic name ofoutCls, e.g. from configuration modeloutTrans- the output data type translator- See Also:
registerInputTypeTranslator(Class, String, TypeTranslator)
-
attachIngestor
public <O> void attachIngestor(java.lang.Class<O> outCls, java.lang.String outTypeName, DataIngestor<O> ingestor)Description copied from interface:GenericMultiTypeServiceAttaches an asynchronous result data ingestor.- Specified by:
attachIngestorin interfaceGenericMultiTypeService- Type Parameters:
O- the output data type- Parameters:
outCls- the class representing the typeoutTypeName- symbolic name ofoutCls, e.g. from configuration modelingestor- the ingestor instance
-
compose
protected static java.lang.String compose(java.lang.String typeName, java.lang.String data)Composes a symbolic type name with the string-serialized data.- Parameters:
typeName- the type namedata- the data- Returns:
- the composed String
-
getParameterConfigurer
public ParameterConfigurer<?> getParameterConfigurer(java.lang.String paramName)
Description copied from interface:ParameterConfigurerProviderReturns the parameter configurer for a given parameter. We need this generic approach to ease code generation and initial setting of parameter values.- Specified by:
getParameterConfigurerin interfaceParameterConfigurerProvider- Specified by:
getParameterConfigurerin interfaceService- Parameters:
paramName- the name of the parameter- Returns:
- the associated parameter configurer or null if there is none
-
getParameterNames
public java.util.Set<java.lang.String> getParameterNames()
Description copied from interface:ParameterConfigurerProviderThe set of parameter names.- Specified by:
getParameterNamesin interfaceParameterConfigurerProvider- Specified by:
getParameterNamesin interfaceService- Returns:
- the parameter names, may be null for none
-
addParameterConfigurer
public void addParameterConfigurer(java.util.function.Consumer<java.util.Map<java.lang.String,ParameterConfigurer<?>>> paramConsumer)
Adds parameter configurers via a consumer.- Parameters:
paramConsumer- the consumer, userAbstractService.addConfigurer(Map, String, Class, TypeTranslator, ValueConfigurer)and related methods in there
-
start
protected ServiceState start() throws java.util.concurrent.ExecutionException
Description copied from class:AbstractServiceStarts the service and the background process.- Overrides:
startin classAbstractService- Returns:
- the state to transition to, may be null for none
- Throws:
java.util.concurrent.ExecutionException- if starting the process fails
-
stop
protected ServiceState stop()
Description copied from class:AbstractServiceStops the service and the background process.- Overrides:
stopin classAbstractRunnablesService- Returns:
- the state to transition to, may be null for none
-
establishServerListener
private void establishServerListener(java.lang.String typeName, java.lang.String serverChannel) throws java.io.IOExceptionEstablishes a server listener for internal server-client communication via Transport.- Parameters:
typeName- the name of the type for Java-Python communicationserverChannel- the name of the server channel to send initial request to- Throws:
java.io.IOException- if sending/receiving messages fails
-
establishClientListener
private void establishClientListener(java.lang.String typeName, java.lang.String serverChannel) throws java.io.IOExceptionEstablishes a client listener for internal server-client communication via Transport.- Parameters:
typeName- the name of the type for Java-Python communicationserverChannel- the name of the server channel to send initial request to- Throws:
java.io.IOException- if sending/receiving messages fails
-
getAvgResponseTime
public long getAvgResponseTime()
Returns the average response time for the execution in Python (without transport).- Returns:
- the average response time
-
-