com.aliasi.classify

Class ConditionalClassifierEvaluator<E>

java.lang.Object

com.aliasi.classify.BaseClassifierEvaluator<E>

com.aliasi.classify.RankedClassifierEvaluator<E>

com.aliasi.classify.ScoredClassifierEvaluator<E>

com.aliasi.classify.ConditionalClassifierEvaluator<E>

Type Parameters:

E - The type of objects being classified by the evaluated classifier.

All Implemented Interfaces:

Handler, ObjectHandler<Classified<E>>

Direct Known Subclasses:

JointClassifierEvaluator

public class ConditionalClassifierEvaluator<E>
extends ScoredClassifierEvaluator<E>

A ConditionalClassifierEvaluator provides an evaluation harness for conditional probability-based n-best classifiers. It extends the scored classifier evaluator with conditional probability specific evaluation metrics.

Thread Safety

This class must be read-write synchronized externally for use in multiple threads.

Since:

LingPipe3.9.1

Version:

3.9.1

Author:

Bob Carpenter

Constructor Summary

Constructors

Constructor and Description
ConditionalClassifierEvaluator(ConditionalClassifier<E> classifier, String[] categories, boolean storeInputs) Construct a scored classifier evaluator with the specified classifier, categories and flag indicating whether or not to store inputs.

Method Summary

Methods

Modifier and Type	Method and Description
double	averageConditionalProbability(String refCategory, String responseCategory) Returns the average conditional probability of the specified response category for test cases with the specified reference category.
double	averageConditionalProbabilityReference() Returns the average over all test cases of the conditional probability of the response that matches the reference category.
ConditionalClassifier<E>	classifier() Return the classifier being evaluated.
ScoredPrecisionRecallEvaluation	conditionalOneVersusAll(String refCategory) Returns a scored precision-recall evaluation of the classifcation of the specified reference category versus all other categories using the conditional probability scores.
void	handle(Classified<E> classified) Add the specified classified object to this evaluator.
void	setClassifier(ConditionalClassifier<E> classifier) Set the classifier being evaluated to the specified value.

Methods inherited from class com.aliasi.classify.ScoredClassifierEvaluator

averageScore, averageScoreReference, scoredOneVersusAll, setClassifier

Methods inherited from class com.aliasi.classify.RankedClassifierEvaluator

averageRank, averageRankReference, meanReciprocalRank, rankCount, setClassifier

Methods inherited from class com.aliasi.classify.BaseClassifierEvaluator

addClassification, categories, confusionMatrix, falseNegatives, falsePositives, numCases, numCategories, oneVersusAll, setClassifier, toString, trueNegatives, truePositives

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

ConditionalClassifierEvaluator

public ConditionalClassifierEvaluator(ConditionalClassifier<E> classifier,
                              String[] categories,
                              boolean storeInputs)

Construct a scored classifier evaluator with the specified classifier, categories and flag indicating whether or not to store inputs.

Parameters:

classifier - Classifier to evaluate.

categories - Complete list of categories.

storeInputs - Set to true to store input objects.

Method Detail

setClassifier

public void setClassifier(ConditionalClassifier<E> classifier)

Set the classifier being evaluated to the specified value. This method is useful to evaluate multiple classifiers with the same evaluator, for instance for use in cross-validation.

Parameters:

classifier - New classifier for this evaluation.

Throws:

IllegalArgumentException - If called from an evaluator with a runtime type other than ConditionalClassifierEvaluator.

classifier

public ConditionalClassifier<E> classifier()

Return the classifier being evaluated.

Overrides:

classifier in class ScoredClassifierEvaluator<E>

Returns:

The classifier for this evaluator.

handle

public void handle(Classified<E> classified)

Description copied from class: BaseClassifierEvaluator

Add the specified classified object to this evaluator.

Specified by:

handle in interface ObjectHandler<Classified<E>>

Overrides:

handle in class ScoredClassifierEvaluator<E>

Parameters:

classified - Classified object to add to evaluation.

averageConditionalProbability

public double averageConditionalProbability(String refCategory,
                                   String responseCategory)

Returns the average conditional probability of the specified response category for test cases with the specified reference category. If there are no cases matching the reference category, the result is Double.NaN. If the conditional classifiers' results are properly normalized, the sum of the averages over all categories will be 1.0.

Better classifiers return high values when the reference and response categories are the same and lower values when they are different. The log value would be extremely volatile given the extremely low and high conditional estimates of the language model classifiers.

Parameters:

refCategory - Reference category.

responseCategory - Response category.

Returns:

Average conditional probability of response category in cases for specified reference category.

Throws:

IllegalArgumentException - If the either category is unknown.

averageConditionalProbabilityReference

public double averageConditionalProbabilityReference()

Returns the average over all test cases of the conditional probability of the response that matches the reference category. Better classifiers return higher values for this average.

As a normalized value, the average conditional probability always has a sensible interpretation across training instances.

Returns:

The average conditional probability of the reference category in the response.

conditionalOneVersusAll

public ScoredPrecisionRecallEvaluation conditionalOneVersusAll(String refCategory)

Returns a scored precision-recall evaluation of the classifcation of the specified reference category versus all other categories using the conditional probability scores. This method may only be called for evaluations that have scores.

Parameters:

refCategory - Reference category.

Returns:

The conditional one-versus-all precision-recall evaluatuion.

Throws:

IllegalArgumentException - If the specified category is unknown.