Class RetrieverEvaluationUtils
- java.lang.Object
-
- de.uni_trier.wi2.procake.utils.eval.RetrieverEvaluationUtils
-
public class RetrieverEvaluationUtils extends Object
-
-
Constructor Summary
Constructors Constructor Description RetrieverEvaluationUtils()
-
Method Summary
All Methods Static Methods Concrete Methods Modifier and Type Method Description static List<SimpleSimilarityResult>computeGroundTruthSimilarities(Retriever<NESTWorkflowObject,Query> groundTruthRetriever, WriteableObjectPool<NESTWorkflowObject> trainCaseBase, WriteableObjectPool<NESTWorkflowObject> testCaseBase, String pathGroundTruthSimilarities)Computes the ground-truth similarities with the given ground-truth retriever.static List<SimpleSimilarityResult>computeGroundTruthSimilarities(Retriever<NESTWorkflowObject,Query> groundTruthRetriever, TrainingObjectPool<NESTWorkflowObject> trainingObjectPool, String pathGroundTruthSimilarities)Computes the ground-truth similarities with the given ground-truth retriever.static List<SimpleSimilarityResult>loadGroundTruthSimilarities(String pathGroundTruthSimilarities)Loads the ground-truth similarities from file system at the specified path.static voidtestGroundTruthSimilarities(WriteableObjectPool<NESTWorkflowObject> trainCaseBase, WriteableObjectPool<NESTWorkflowObject> testCaseBase, List<SimpleSimilarityResult> groundTruthSimilarities)Checks the ground-truth similarities by searching for every pair of query and case in them.static voidtestGroundTruthSimilarities(TrainingObjectPool<NESTWorkflowObject> trainingObjectPool, SimpleSimilarityCache similarityCache)Checks the ground-truth similarities by searching for every pair of query and case in them.static voidtestGroundTruthSimilarities(TrainingObjectPool<NESTWorkflowObject> trainingObjectPool, List<SimpleSimilarityResult> groundTruthSimilarities)Checks the ground-truth similarities by searching for every pair of query and case in them.static voidtestRetrievers(WriteableObjectPool<NESTWorkflowObject> trainPool, WriteableObjectPool<NESTWorkflowObject> testPool, Map<String,Retriever<NESTWorkflowObject,Query>> retrieverMap)Simple method to test all retrievers in retrieverMap prior to the actual evaluation.static voidtestRetrievers(TrainingObjectPool<NESTWorkflowObject> trainingObjectPool, Map<String,Retriever<NESTWorkflowObject,Query>> retrieverMap)Simple method to test all retrievers in retrieverMap prior to the actual evaluation.
-
-
-
Method Detail
-
testRetrievers
public static void testRetrievers(TrainingObjectPool<NESTWorkflowObject> trainingObjectPool, Map<String,Retriever<NESTWorkflowObject,Query>> retrieverMap) throws RetrieverEvaluationException
Simple method to test all retrievers in retrieverMap prior to the actual evaluation.- Parameters:
trainingObjectPool- Pool containing the train pool and the test pool.retrieverMap- the map with completely initialized retrievers- Throws:
RetrieverEvaluationException- if something goes wrong
-
testRetrievers
public static void testRetrievers(WriteableObjectPool<NESTWorkflowObject> trainPool, WriteableObjectPool<NESTWorkflowObject> testPool, Map<String,Retriever<NESTWorkflowObject,Query>> retrieverMap) throws RetrieverEvaluationException
Simple method to test all retrievers in retrieverMap prior to the actual evaluation.- Parameters:
trainPool- the train pooltestPool- the test poolretrieverMap- the map with completely initialized retrievers- Throws:
RetrieverEvaluationException- if something goes wrong
-
loadGroundTruthSimilarities
public static List<SimpleSimilarityResult> loadGroundTruthSimilarities(String pathGroundTruthSimilarities)
Loads the ground-truth similarities from file system at the specified path. This file is a CSV file. The loaded values are afterwards stored in groundTruthSimilaritiesMap.- Parameters:
pathGroundTruthSimilarities- the path to the stored similarities (CSV file)- Returns:
- the loaded ground-truth similarities
-
computeGroundTruthSimilarities
public static List<SimpleSimilarityResult> computeGroundTruthSimilarities(Retriever<NESTWorkflowObject,Query> groundTruthRetriever, TrainingObjectPool<NESTWorkflowObject> trainingObjectPool, String pathGroundTruthSimilarities) throws IOException
Computes the ground-truth similarities with the given ground-truth retriever. The computed values are afterwards stored in groundTruthSimilaritiesMap. Optionally, the computed similarities can also be stored as a CSV file for future usage.- Parameters:
groundTruthRetriever- the ground-truth retrievertrainingObjectPool- Pool containing the training case base and the test case base.pathGroundTruthSimilarities- if this path is notnull, the similarities will- Returns:
- the computed ground-truth similarities
- Throws:
IOException
-
computeGroundTruthSimilarities
public static List<SimpleSimilarityResult> computeGroundTruthSimilarities(Retriever<NESTWorkflowObject,Query> groundTruthRetriever, WriteableObjectPool<NESTWorkflowObject> trainCaseBase, WriteableObjectPool<NESTWorkflowObject> testCaseBase, String pathGroundTruthSimilarities) throws IOException
Computes the ground-truth similarities with the given ground-truth retriever. The computed values are afterwards stored in groundTruthSimilaritiesMap. Optionally, the computed similarities can also be stored as a CSV file for future usage.- Parameters:
groundTruthRetriever- the ground-truth retrievertrainCaseBase- the training case basetestCaseBase- the test case basepathGroundTruthSimilarities- if this path is notnull, the similarities will- Returns:
- the computed ground-truth similarities
- Throws:
IOException
-
testGroundTruthSimilarities
public static void testGroundTruthSimilarities(TrainingObjectPool<NESTWorkflowObject> trainingObjectPool, List<SimpleSimilarityResult> groundTruthSimilarities) throws RetrieverEvaluationException
Checks the ground-truth similarities by searching for every pair of query and case in them.- Parameters:
trainingObjectPool- Pool containing the training case base and the test case base.groundTruthSimilarities- the list of ground-truth similarities- Throws:
RetrieverEvaluationException- if something goes wrong
-
testGroundTruthSimilarities
public static void testGroundTruthSimilarities(TrainingObjectPool<NESTWorkflowObject> trainingObjectPool, SimpleSimilarityCache similarityCache) throws RetrieverEvaluationException
Checks the ground-truth similarities by searching for every pair of query and case in them.- Parameters:
trainingObjectPool- Pool containing the training case base and the test case base.similarityCache- the similarity cache to check- Throws:
RetrieverEvaluationException- if something goes wrong
-
testGroundTruthSimilarities
public static void testGroundTruthSimilarities(WriteableObjectPool<NESTWorkflowObject> trainCaseBase, WriteableObjectPool<NESTWorkflowObject> testCaseBase, List<SimpleSimilarityResult> groundTruthSimilarities) throws RetrieverEvaluationException
Checks the ground-truth similarities by searching for every pair of query and case in them.- Parameters:
trainCaseBase- the training case basetestCaseBase- the test case basegroundTruthSimilarities- the list of ground-truth similarities- Throws:
RetrieverEvaluationException- if something goes wrong
-
-