de.gerdiproject.harvest.etls.utils

Class EtlUtils

java.lang.Object

de.gerdiproject.harvest.etls.utils.EtlUtils

public class EtlUtils
extends Object

This class contains static helper functions for processing multiple AbstractETLs.

Author:

Robin Weiss

Method Summary

Modifier and Type	Method and Description
static long	estimateRemainingHarvestTime(long harvestStartTimestamp, ETLState etlStatus, int harvestedDocuments, int maxDocuments) Estimates the remaining harvesting duration in milliseconds.
static String	formatHarvestTime(long milliseconds) Creates a duration string out of a specified number of seconds
static String	getCombinedHashes(Collection<AbstractETL<?,?>> etls) Generates a combined hash over the concatenated hashes of a specified Collection of enabled AbstractETLs.
static ETLHealth	getCombinedHealth(Collection<AbstractETL<?,?>> etls) Retrieves the combined health status of a specified Collection of AbstractETLs.
static void	processETLs(Collection<AbstractETL<?,?>> etls, Consumer<AbstractETL<?,?>> consumer) Sequentially processes a Collection of AbstractETLs and executes a specified Consumer on each one.
static <T> List<T>	processETLsAsList(Collection<AbstractETL<?,?>> etls, Function<AbstractETL<?,?>,T> function) Processes a Collection of AbstractETLs sequentially and stores the return values of a specified Function in a List.
static int	sumUpETLValues(Collection<AbstractETL<?,?>> etls, Function<AbstractETL<?,?>,Integer> intFunction) Iterates a Collection of AbstractETLs sequentially and sums up the return values of a specified Function.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getCombinedHashes

public static String getCombinedHashes(Collection<AbstractETL<?,?>> etls)

Generates a combined hash over the concatenated hashes of a specified Collection of enabled AbstractETLs.

Parameters:

etls - the AbstractETLs of which the hashes are combined

Returns:

a combined hash over the concatenated hashes of specified AbstractETLs

getCombinedHealth

public static ETLHealth getCombinedHealth(Collection<AbstractETL<?,?>> etls)

Retrieves the combined health status of a specified Collection of AbstractETLs.

Parameters:

etls - the AbstractETLs of which the health statusses are combined

Returns:

a health status representing the combined health of all specified ETLs

estimateRemainingHarvestTime

public static long estimateRemainingHarvestTime(long harvestStartTimestamp,
                                                ETLState etlStatus,
                                                int harvestedDocuments,
                                                int maxDocuments)

Estimates the remaining harvesting duration in milliseconds.

Parameters:

harvestStartTimestamp - the unix timestamp that denotes when the harvest started

etlStatus - the current status of the AbstractETLs

harvestedDocuments - the number of documents that were harvested

maxDocuments - the total number of harvestable documents, or -1 if unknown

Returns:

the remaining harvesting duration in milliseconds, or -1 if it cannot be estimated

formatHarvestTime

public static String formatHarvestTime(long milliseconds)

Creates a duration string out of a specified number of seconds

Parameters:

milliseconds - the time span in milliseconds

Returns:

a formatted duration string, or "unknown" if the duration is negative

processETLsAsList

public static <T> List<T> processETLsAsList(Collection<AbstractETL<?,?>> etls,
                                            Function<AbstractETL<?,?>,T> function)

Processes a Collection of AbstractETLs sequentially and stores the return values of a specified Function in a List.

Type Parameters:

T - the type of the Function return value

Parameters:

etls - the AbstractETLs that are to be processed

function - a Function that is called for each ETL, using the ETL as a parameter

Returns:

a List containing the return values of all Functions

processETLs

public static void processETLs(Collection<AbstractETL<?,?>> etls,
                               Consumer<AbstractETL<?,?>> consumer)

Sequentially processes a Collection of AbstractETLs and executes a specified Consumer on each one.

Parameters:

etls - the AbstractETLs that are to be processed

consumer - a Consumer that is called for each ETL, using the ETL as a parameter

sumUpETLValues

public static int sumUpETLValues(Collection<AbstractETL<?,?>> etls,
                                 Function<AbstractETL<?,?>,Integer> intFunction)

Iterates a Collection of AbstractETLs sequentially and sums up the return values of a specified Function.

Parameters:

etls - the AbstractETLs that are to be processed

intFunction - a Function that is called for each ETL, using the ETL as a parameter and returning an integer value

Returns:

the sum of all return values of the Function