BaseClassifierEvaluator (LingPipe, Alias-i 4.1.0 API)

java.lang.Object
- com.aliasi.classify.BaseClassifierEvaluator<E>

Type Parameters:
E - The type of objects being classified by the evaluated classifier.

All Implemented Interfaces:

Handler, ObjectHandler<Classified<E>>

Direct Known Subclasses:

RankedClassifierEvaluator
```
public class BaseClassifierEvaluator<E>
extends Object
implements ObjectHandler<Classified<E>>
```
A BaseClassifierEvaluator provides an evaluation harness for first-best classifiers. An evaluator is constructed from a classifier and a complete list of the categories returned by the classifier.
Test cases are then added using the handle(Classified) which accepts a string-based category and object to classify. The evaluator will run the classifier over the input object and collect results over multiple cases.
There are subtypes of this classifier that evaluate richer classifiers.
An exhaustive set of evaluation metrics for first-best classification results is accessbile as a confusion matrix through the confusionMatrix() method. Confusion matrices provide dozens of statistics on classification which can be computed from first-best results; see ConfusionMatrix for more information.
Thread Safety

This class requires concurrent read and synchronous write synchronization. Reads are any of the statistics gathering methods and write is just adding new test cases.
Storing Cases
This class always stores the classification results and true category of an cases. There is a flag in the constructor that additionally allows the inputs for cases to be stored as part of an evaluation. If the flag is set to true, all input cases are stored. This enables the output of true positives, false positives, false negatives, and true negatives through the methods of the same names.

Since:

LingPipe3.9.1

Version:

3.9.1

Author:

Bob Carpenter

Constructor Summary

Constructors
Constructor and Description
`BaseClassifierEvaluator(BaseClassifier<E> classifier, String[] categories, boolean storeInputs)` Construct a classifier evaluator for the specified classifier that records results for the specified set of categories, storing cases or not based on the specified flag.

Method Summary

Methods
Modifier and Type	Method and Description
`void`	`addClassification(String referenceCategory, Classification classification, E input)` Adds the specified classification as a response for the specified reference category.
`String[]`	`categories()` Returns a copy of the the categories for which this evaluator stores results.
`BaseClassifier<E>`	`classifier()` Returns the classifier for this evaluator.
`ConfusionMatrix`	`confusionMatrix()` Returns the confusion matrix of first-best classification result statistics for this evaluator.
`List<Classified<E>>`	`falseNegatives(String category)` Returns the list of false negative cases along with their classifications for items of the specified category.
`List<Classified<E>>`	`falsePositives(String category)` Returns the list of false positive cases along with their classifications for items of the specified category.
`void`	`handle(Classified<E> classified)` Add the specified classified object to this evaluator.
`int`	`numCases()` Returns the number of test cases which have been provided to this evaluator.
`int`	`numCategories()` Returns the number of categories for the classifier being evaluated.
`PrecisionRecallEvaluation`	`oneVersusAll(String refCategory)` Returns the first-best one-versus-all precision-recall evaluation of the classification of the specified reference category versus all other categories.
`void`	`setClassifier(BaseClassifier<E> classifier)` Set the classfier for this evaluator to the specified value.
`String`	`toString()` Return a string-based representation of the results of the evaluation.
`List<Classified<E>>`	`trueNegatives(String category)` Returns the list of true negative cases along with their classifications for items of the specified category.
`List<Classified<E>>`	`truePositives(String category)` Returns the list of true positive cases along with their classifications for items of the specified category.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Constructor Detail
  - BaseClassifierEvaluator
```
public BaseClassifierEvaluator(BaseClassifier<E> classifier,
                       String[] categories,
                       boolean storeInputs)
```
    Construct a classifier evaluator for the specified classifier that records results for the specified set of categories, storing cases or not based on the specified flag.
    If the classifier evaluator is only going to be populated using the addClassification(String,Classification,Object) method, then the classifier may be null.
    
    Parameters:
    classifier - Classifier to evaluate.
    categories - Categories of the classifier.
    storeInputs - Flag indicating whether input cases should be stored.
- Method Detail
  - numCategories
```
public int numCategories()
```
    Returns the number of categories for the classifier being evaluated.
    
    Returns:
    Number of categories being evaluated.
  - categories
```
public String[] categories()
```
    Returns a copy of the the categories for which this evaluator stores results.
    
    Returns:
    The categories for which this evaluator stores results.
  - classifier
```
public BaseClassifier<E> classifier()
```
    Returns the classifier for this evaluator.
    
    Returns:
    The classifier for this evaluator.
  - setClassifier
```
public void setClassifier(BaseClassifier<E> classifier)
```
    Set the classfier for this evaluator to the specified value. This method allows the results of evaluating several different classifiers to be aggregated into a single evaluation. The primary use case is cross-validation, where a single evaluator may be used to evaluate classifiers trained on different folds.
    This method will throw an exception if called from an evaluator with a more specific runtime type.
    
    Parameters:
    classifier - New classifier for this evaluator.
    
    Throws:
    
    IllegalArgumentException - If called from a class with a runtime type other than BaseClassifierEvaluator.
  - truePositives
```
public List<Classified<E>> truePositives(String category)
```
    Returns the list of true positive cases along with their classifications for items of the specified category.
    The cases will be returned in decreasing order of conditional probability if applicable, decreasing order of score otherwise, and if not scored, in the order in which they were processed.
    A true positive case for the specified category has reference category equal to the specified category and first-best classification result equal to the specified category.
    
    Parameters:
    category - Category whose cases are returned.
    
    Returns:
    True positives for specified category.
    
    Throws:
    
    UnsupportedOperationException - If this class does not store its cases.
  - falsePositives
```
public List<Classified<E>> falsePositives(String category)
```
    Returns the list of false positive cases along with their classifications for items of the specified category.
    A false positive case for the specified category has reference category unequal to the specified category and first-best classification result equal to the specified category.
    
    Parameters:
    category - Category whose cases are returned.
    
    Returns:
    False positives for specified category.
    
    Throws:
    
    UnsupportedOperationException - If this class does not store its cases.
  - falseNegatives
```
public List<Classified<E>> falseNegatives(String category)
```
    Returns the list of false negative cases along with their classifications for items of the specified category.
    A false negative case for the specified category has reference category equal to the specified category and first-best classification result unequal to the specified category.
    
    Parameters:
    category - Category whose cases are returned.
    
    Returns:
    False negatives for specified category.
    
    Throws:
    
    UnsupportedOperationException - If this class does not store its cases.
  - trueNegatives
```
public List<Classified<E>> trueNegatives(String category)
```
    Returns the list of true negative cases along with their classifications for items of the specified category.
    A true negative case for the specified category has reference category unequal to the specified category and first-best classification result unequal to the specified category.
    
    Parameters:
    category - Category whose cases are returned.
    
    Returns:
    True positives for specified category.
    
    Throws:
    
    UnsupportedOperationException - If this class does not store its cases.
  - handle
```
public void handle(Classified<E> classified)
```
    Add the specified classified object to this evaluator.
    
    Specified by:
    
    handle in interface ObjectHandler<Classified<E>>
    
    Parameters:
    classified - Classified object to add to evaluation.
  - addClassification
```
public void addClassification(String referenceCategory,
                     Classification classification,
                     E input)
```
    Adds the specified classification as a response for the specified reference category. If this evaluator stores cases, the input will be stored.
    
    Parameters:
    referenceCategory - Reference category for case.
    classification - Response classification for case.
    input - Input for the specified classification.
  - numCases
```
public int numCases()
```
    Returns the number of test cases which have been provided to this evaluator.
    
    Returns:
    The number of test cases which have been provided to this evaluator.
  - confusionMatrix
```
public ConfusionMatrix confusionMatrix()
```
    Returns the confusion matrix of first-best classification result statistics for this evaluator. See ConfusionMatrix for details of the numerous available evaluation metrics provided by confusion matrices.
    
    Returns:
    The confusion matrix for the test cases evaluated so far.
  - oneVersusAll
```
public PrecisionRecallEvaluation oneVersusAll(String refCategory)
```
    Returns the first-best one-versus-all precision-recall evaluation of the classification of the specified reference category versus all other categories. This method may be called for any evaluation.
    
    Parameters:
    refCategory - Reference category.
    
    Returns:
    The first-best one-versus-all precision-recall evaluatuion.
    
    Throws:
    
    IllegalArgumentException - If the specified category is unknown.
  - toString
```
public String toString()
```
    Return a string-based representation of the results of the evaluation.
    
    Overrides:
    
    toString in class Object
    
    Returns:
    String-based representation of the results.

Class BaseClassifierEvaluator<E>

Thread Safety

Storing Cases

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

BaseClassifierEvaluator

Method Detail

numCategories

categories

classifier

setClassifier

truePositives

falsePositives

falseNegatives

trueNegatives

handle

addClassification

numCases

confusionMatrix

oneVersusAll

toString