ConfidenceEvaluator (jcore-mallet-0.4 1.0.0 API)

java.lang.Object
- edu.umass.cs.mallet.base.fst.confidence.ConfidenceEvaluator

public class ConfidenceEvaluator
extends Object

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static class`	`ConfidenceEvaluator.EntityConfidence` a simple class to store a confidence score and whether or not this labeling is correct

Constructor Summary

Constructors
Constructor and Description
`ConfidenceEvaluator(InstanceWithConfidence[] instances, boolean sorted)`
`ConfidenceEvaluator(PipedInstanceWithConfidence[] instances, boolean sorted)`
`ConfidenceEvaluator(Segment[] segments, boolean sorted)`
`ConfidenceEvaluator(Vector confidences)`
`ConfidenceEvaluator(Vector confidences, int nBins)`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`double`	`accuracyAtCoverage(double cov)`
`String`	`accuracyCoverageValuesToString()`
`String`	`accuracyRecallValuesToString(int totalTrue)`
`double`	`correlation()` Calculate pearson's R for the corellation between confidence and correct, where 1 = correct and -1 = incorrect
`double[]`	`getAccuracyCoverageValues()` get accuracy at coverage for each bin of values
`double[][]`	`getAccuracyRecallValues(int totalTrue)` get accuracy at recall for each bin of values
`double`	`getAverageAccuracy()`
`double`	`getAverageCorrectConfidence()` Average confidence score for the incorrect entities
`double`	`getAverageIncorrectConfidence()` Average confidence score for the incorrect entities
`double`	`getAveragePrecision()` IR Average precision measure.
`double`	`getConfidenceMean()`
`double`	`getConfidenceStandardDeviation()` Standard deviation of confidence scores
`double`	`getConfidenceSum()`
`double`	`getWorstAveragePrecision()` For comparison, rank segments as badly as possible (all "incorrect" before "correct").
`int`	`numCorrect()`
`int`	`numCorrectAtCoverage(double cov)`
`double`	`pointBiserialCorrelation()` Correlation when one variable (X) is binary: r = (bar(x1) - bar(x0)) * sqrt(p(1-p)) / sx , where bar(x1) = mean of X when Y is 1 bar(x0) = mean of X when Y is 0 sx = standard deviation of X p = proportion of values where Y=1
`int`	`size()`
`String`	`toString()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Constructor Detail
  - ConfidenceEvaluator
```
public ConfidenceEvaluator(Vector confidences,
                           int nBins)
```
  - ConfidenceEvaluator
```
public ConfidenceEvaluator(Vector confidences)
```
  - ConfidenceEvaluator
```
public ConfidenceEvaluator(Segment[] segments,
                           boolean sorted)
```
  - ConfidenceEvaluator
```
public ConfidenceEvaluator(InstanceWithConfidence[] instances,
                           boolean sorted)
```
  - ConfidenceEvaluator
```
public ConfidenceEvaluator(PipedInstanceWithConfidence[] instances,
                           boolean sorted)
```
- Method Detail
  - pointBiserialCorrelation
```
public double pointBiserialCorrelation()
```
    Correlation when one variable (X) is binary: r = (bar(x1) - bar(x0)) * sqrt(p(1-p)) / sx , where bar(x1) = mean of X when Y is 1 bar(x0) = mean of X when Y is 0 sx = standard deviation of X p = proportion of values where Y=1
  - getAveragePrecision
```
public double getAveragePrecision()
```
    IR Average precision measure. Analogous to ranking _correct_ documents by confidence score.
  - getWorstAveragePrecision
```
public double getWorstAveragePrecision()
```
    For comparison, rank segments as badly as possible (all "incorrect" before "correct").
  - getConfidenceSum
```
public double getConfidenceSum()
```
  - getConfidenceMean
```
public double getConfidenceMean()
```
  - getConfidenceStandardDeviation
```
public double getConfidenceStandardDeviation()
```
    Standard deviation of confidence scores
  - correlation
```
public double correlation()
```
    Calculate pearson's R for the corellation between confidence and correct, where 1 = correct and -1 = incorrect
  - getAccuracyCoverageValues
```
public double[] getAccuracyCoverageValues()
```
    get accuracy at coverage for each bin of values
  - accuracyCoverageValuesToString
```
public String accuracyCoverageValuesToString()
```
  - getAccuracyRecallValues
```
public double[][] getAccuracyRecallValues(int totalTrue)
```
    get accuracy at recall for each bin of values
    
    Parameters:
    
    totalTrue - total number of true Segments
    
    Returns:
    
    2-d array where values[i][0] is coverage and values[i][1] is accuracy at position i.
  - accuracyRecallValuesToString
```
public String accuracyRecallValuesToString(int totalTrue)
```
  - accuracyAtCoverage
```
public double accuracyAtCoverage(double cov)
```
  - numCorrectAtCoverage
```
public int numCorrectAtCoverage(double cov)
```
  - getAverageAccuracy
```
public double getAverageAccuracy()
```
  - numCorrect
```
public int numCorrect()
```
  - getAverageIncorrectConfidence
```
public double getAverageIncorrectConfidence()
```
    Average confidence score for the incorrect entities
  - getAverageCorrectConfidence
```
public double getAverageCorrectConfidence()
```
    Average confidence score for the incorrect entities
  - size
```
public int size()
```
  - toString
```
public String toString()
```
    Overrides:
    
    toString in class Object

Class ConfidenceEvaluator

Nested Class Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

ConfidenceEvaluator

ConfidenceEvaluator

ConfidenceEvaluator

ConfidenceEvaluator

ConfidenceEvaluator

Method Detail

pointBiserialCorrelation

getAveragePrecision

getWorstAveragePrecision

getConfidenceSum

getConfidenceMean

getConfidenceStandardDeviation

correlation

getAccuracyCoverageValues

accuracyCoverageValuesToString

getAccuracyRecallValues

accuracyRecallValuesToString

accuracyAtCoverage

numCorrectAtCoverage

getAverageAccuracy

numCorrect

getAverageIncorrectConfidence

getAverageCorrectConfidence

size

toString