com.aliasi.classify

Class JointClassifierEvaluator<E>

java.lang.Object

com.aliasi.classify.BaseClassifierEvaluator<E>

com.aliasi.classify.RankedClassifierEvaluator<E>

com.aliasi.classify.ScoredClassifierEvaluator<E>

com.aliasi.classify.ConditionalClassifierEvaluator<E>

com.aliasi.classify.JointClassifierEvaluator<E>

Type Parameters:

E - The type of objects being classified by the evaluated classifier.

All Implemented Interfaces:

Handler, ObjectHandler<Classified<E>>

public class JointClassifierEvaluator<E>
extends ConditionalClassifierEvaluator<E>

A JointClassifierEvaluator provides an evaluation harness for joint probability-based n-best classifiers. It extends the conditional classifier evaluator with joint probability specific evaluation metrics.

Thread Safety

This class must be read-write synchronized externally for use in multiple threads.

Since:

LingPipe3.9.1

Version:

3.9.1

Author:

Bob Carpenter

Constructor Summary

Constructors

Constructor and Description
JointClassifierEvaluator(JointClassifier<E> classifier, String[] categories, boolean storeInputs) Construct a scored classifier evaluator with the specified classifier, categories and flag indicating whether or not to store inputs.

Method Summary

Modifier and Type	Method and Description
double	averageLog2JointProbability(String refCategory, String responseCategory) Returns the average log (base 2) joint probability of the response category for cases of the specified reference category.
double	averageLog2JointProbabilityReference() Returns the average over all test cases of the joint log (base 2) probability of the response that matches the reference category.
JointClassifier<E>	classifier() Return the classifier being evaluated.
double	corpusLog2JointProbability() Returns the joint log (base 2) probability of the entire evaluation corpus.
void	setClassifier(JointClassifier<E> classifier) Set the classifier being evaluated to the specified value.

Methods inherited from class com.aliasi.classify.ConditionalClassifierEvaluator

averageConditionalProbability, averageConditionalProbabilityReference, conditionalOneVersusAll, handle, setClassifier

Methods inherited from class com.aliasi.classify.ScoredClassifierEvaluator

averageScore, averageScoreReference, scoredOneVersusAll, setClassifier

Methods inherited from class com.aliasi.classify.RankedClassifierEvaluator

averageRank, averageRankReference, meanReciprocalRank, rankCount, setClassifier

Methods inherited from class com.aliasi.classify.BaseClassifierEvaluator

addClassification, categories, confusionMatrix, falseNegatives, falsePositives, numCases, numCategories, oneVersusAll, setClassifier, toString, trueNegatives, truePositives

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

JointClassifierEvaluator

public JointClassifierEvaluator(JointClassifier<E> classifier,
                                String[] categories,
                                boolean storeInputs)

Construct a scored classifier evaluator with the specified classifier, categories and flag indicating whether or not to store inputs.

Parameters:

classifier - Classifier to evaluate.

categories - Complete list of categories.

storeInputs - Set to true to store input objects.

Method Detail

setClassifier

public void setClassifier(JointClassifier<E> classifier)

Set the classifier being evaluated to the specified value. This method is useful to evaluate multiple classifiers with the same evaluator, for instance for use in cross-validation.

Parameters:

classifier - New classifier for this evaluation.

Throws:

IllegalArgumentException - If called from an evaluator with a runtime type other than JointClassifierEvaluator.

classifier

public JointClassifier<E> classifier()

Return the classifier being evaluated.

Overrides:

classifier in class ConditionalClassifierEvaluator<E>

Returns:

The classifier for this evaluator.

averageLog2JointProbability

public double averageLog2JointProbability(String refCategory,
                                          String responseCategory)

Returns the average log (base 2) joint probability of the response category for cases of the specified reference category. If there are no cases matching the reference category, the result is Double.NaN.

Better classifiers return high values when the reference and response categories are the same and lower values when they are different. Unlike the conditional probability values, joint probability averages are not particularly useful because they are not normalized by input length. For the language model classifiers, the scores are normalized by length, and provide a better cross-case view.

Parameters:

refCategory - Reference category.

responseCategory - Response category.

Returns:

Average log (base 2) conditional probability of response category in cases for specified reference category.

Throws:

IllegalArgumentException - If the either category is unknown.

ClassCastException - if the classifications are not joint classifications.

averageLog2JointProbabilityReference

public double averageLog2JointProbabilityReference()

Returns the average over all test cases of the joint log (base 2) probability of the response that matches the reference category. Better classifiers return higher values for this average.

Whether average scores make sense across training instances depends on the classifier. For the language-model based classifiers, the normalized score values are more reasonable averages.

Returns:

The average joint log probability of the reference category in the response.

corpusLog2JointProbability

public double corpusLog2JointProbability()

Returns the joint log (base 2) probability of the entire evaluation corpus. This is defined independently of the reference categories by summing over inputs x:

log2 p(corpus) = Σx in corpus log2 p(x)

where the probability p(x) for a single case with input x is defined in the usual way by summing over categories:

p(x) = Σc in cats p(c,x)

Returns:

The log probability of the set of inputs.

Throws:

ClassCastException - if the classifications are not joint classifications.