Package de.uni_trier.wi2.procake.utils.eval

Class RetrieverEvaluationUtils

java.lang.Object

de.uni_trier.wi2.procake.utils.eval.RetrieverEvaluationUtils

public class RetrieverEvaluationUtils
extends Object

Constructor Summary

Constructors

Constructor	Description
RetrieverEvaluationUtils()

Method Summary

Modifier and Type	Method	Description
static List<SimpleSimilarityResult>	computeGroundTruthSimilarities(Retriever<NESTWorkflowObject,Query> groundTruthRetriever, WriteableObjectPool<NESTWorkflowObject> trainCaseBase, WriteableObjectPool<NESTWorkflowObject> testCaseBase, String pathGroundTruthSimilarities)	Computes the ground-truth similarities with the given ground-truth retriever.
static List<SimpleSimilarityResult>	computeGroundTruthSimilarities(Retriever<NESTWorkflowObject,Query> groundTruthRetriever, TrainingObjectPool<NESTWorkflowObject> trainingObjectPool, String pathGroundTruthSimilarities)	Computes the ground-truth similarities with the given ground-truth retriever.
static List<SimpleSimilarityResult>	loadGroundTruthSimilarities(String pathGroundTruthSimilarities)	Loads the ground-truth similarities from file system at the specified path.
static void	testGroundTruthSimilarities(WriteableObjectPool<NESTWorkflowObject> trainCaseBase, WriteableObjectPool<NESTWorkflowObject> testCaseBase, List<SimpleSimilarityResult> groundTruthSimilarities)	Checks the ground-truth similarities by searching for every pair of query and case in them.
static void	testGroundTruthSimilarities(TrainingObjectPool<NESTWorkflowObject> trainingObjectPool, SimpleSimilarityCache similarityCache)	Checks the ground-truth similarities by searching for every pair of query and case in them.
static void	testGroundTruthSimilarities(TrainingObjectPool<NESTWorkflowObject> trainingObjectPool, List<SimpleSimilarityResult> groundTruthSimilarities)	Checks the ground-truth similarities by searching for every pair of query and case in them.
static void	testRetrievers(WriteableObjectPool<NESTWorkflowObject> trainPool, WriteableObjectPool<NESTWorkflowObject> testPool, Map<String,Retriever<NESTWorkflowObject,Query>> retrieverMap)	Simple method to test all retrievers in retrieverMap prior to the actual evaluation.
static void	testRetrievers(TrainingObjectPool<NESTWorkflowObject> trainingObjectPool, Map<String,Retriever<NESTWorkflowObject,Query>> retrieverMap)	Simple method to test all retrievers in retrieverMap prior to the actual evaluation.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RetrieverEvaluationUtils

public RetrieverEvaluationUtils()

Method Detail

testRetrievers

public static void testRetrievers(TrainingObjectPool<NESTWorkflowObject> trainingObjectPool,
                                  Map<String,Retriever<NESTWorkflowObject,Query>> retrieverMap)
                           throws RetrieverEvaluationException

Simple method to test all retrievers in retrieverMap prior to the actual evaluation.

Parameters:

trainingObjectPool - Pool containing the train pool and the test pool.

retrieverMap - the map with completely initialized retrievers

Throws:

RetrieverEvaluationException - if something goes wrong

testRetrievers

public static void testRetrievers(WriteableObjectPool<NESTWorkflowObject> trainPool,
                                  WriteableObjectPool<NESTWorkflowObject> testPool,
                                  Map<String,Retriever<NESTWorkflowObject,Query>> retrieverMap)
                           throws RetrieverEvaluationException

Simple method to test all retrievers in retrieverMap prior to the actual evaluation.

Parameters:

trainPool - the train pool

testPool - the test pool

retrieverMap - the map with completely initialized retrievers

Throws:

RetrieverEvaluationException - if something goes wrong

loadGroundTruthSimilarities

public static List<SimpleSimilarityResult> loadGroundTruthSimilarities(String pathGroundTruthSimilarities)

Loads the ground-truth similarities from file system at the specified path. This file is a CSV file. The loaded values are afterwards stored in groundTruthSimilaritiesMap.

Parameters:

pathGroundTruthSimilarities - the path to the stored similarities (CSV file)

Returns:

the loaded ground-truth similarities

computeGroundTruthSimilarities

public static List<SimpleSimilarityResult> computeGroundTruthSimilarities(Retriever<NESTWorkflowObject,Query> groundTruthRetriever,
                                                                          TrainingObjectPool<NESTWorkflowObject> trainingObjectPool,
                                                                          String pathGroundTruthSimilarities)
                                                                   throws IOException

Computes the ground-truth similarities with the given ground-truth retriever. The computed values are afterwards stored in groundTruthSimilaritiesMap. Optionally, the computed similarities can also be stored as a CSV file for future usage.

Parameters:

groundTruthRetriever - the ground-truth retriever

trainingObjectPool - Pool containing the training case base and the test case base.

pathGroundTruthSimilarities - if this path is not null, the similarities will

Returns:

the computed ground-truth similarities

Throws:

IOException

computeGroundTruthSimilarities

public static List<SimpleSimilarityResult> computeGroundTruthSimilarities(Retriever<NESTWorkflowObject,Query> groundTruthRetriever,
                                                                          WriteableObjectPool<NESTWorkflowObject> trainCaseBase,
                                                                          WriteableObjectPool<NESTWorkflowObject> testCaseBase,
                                                                          String pathGroundTruthSimilarities)
                                                                   throws IOException

Computes the ground-truth similarities with the given ground-truth retriever. The computed values are afterwards stored in groundTruthSimilaritiesMap. Optionally, the computed similarities can also be stored as a CSV file for future usage.

Parameters:

groundTruthRetriever - the ground-truth retriever

trainCaseBase - the training case base

testCaseBase - the test case base

pathGroundTruthSimilarities - if this path is not null, the similarities will

Returns:

the computed ground-truth similarities

Throws:

IOException

testGroundTruthSimilarities

public static void testGroundTruthSimilarities(TrainingObjectPool<NESTWorkflowObject> trainingObjectPool,
                                               List<SimpleSimilarityResult> groundTruthSimilarities)
                                        throws RetrieverEvaluationException

Checks the ground-truth similarities by searching for every pair of query and case in them.

Parameters:

trainingObjectPool - Pool containing the training case base and the test case base.

groundTruthSimilarities - the list of ground-truth similarities

Throws:

RetrieverEvaluationException - if something goes wrong

testGroundTruthSimilarities

public static void testGroundTruthSimilarities(TrainingObjectPool<NESTWorkflowObject> trainingObjectPool,
                                               SimpleSimilarityCache similarityCache)
                                        throws RetrieverEvaluationException

Checks the ground-truth similarities by searching for every pair of query and case in them.

Parameters:

trainingObjectPool - Pool containing the training case base and the test case base.

similarityCache - the similarity cache to check

Throws:

RetrieverEvaluationException - if something goes wrong

testGroundTruthSimilarities

public static void testGroundTruthSimilarities(WriteableObjectPool<NESTWorkflowObject> trainCaseBase,
                                               WriteableObjectPool<NESTWorkflowObject> testCaseBase,
                                               List<SimpleSimilarityResult> groundTruthSimilarities)
                                        throws RetrieverEvaluationException

Checks the ground-truth similarities by searching for every pair of query and case in them.

Parameters:

trainCaseBase - the training case base

testCaseBase - the test case base

groundTruthSimilarities - the list of ground-truth similarities

Throws:

RetrieverEvaluationException - if something goes wrong