de.datexis.sector.eval

Class SegmentationEvaluation

java.lang.Object

de.datexis.annotator.AnnotatorEvaluation

de.datexis.sector.eval.SegmentationEvaluation

public class SegmentationEvaluation
extends AnnotatorEvaluation

Nested Class Summary

Nested classes/interfaces inherited from class de.datexis.annotator.AnnotatorEvaluation

AnnotatorEvaluation.Measure

Field Summary

Fields

Modifier and Type	Field and Description
protected double	countExp
protected double	countPred
protected double	DEFAULT_SCORE
protected boolean	enableKPerDocument whether to recalculate the value of K for each document in the evaluation
protected boolean	enableMergeSections whether to merge adjacent sections with same label into one (in both GOLD and PRED)
protected double	pksum
protected double	wdsum

Fields inherited from class de.datexis.annotator.AnnotatorEvaluation

countAnnotations, countDocs, countExamples, countSentences, countTokens, expectedSource, experimentName, log, predictedSource

Constructor Summary

Constructors

Constructor and Description
SegmentationEvaluation(String experimentName)
SegmentationEvaluation(String experimentName, Annotation.Source expected, Annotation.Source predicted)

Method Summary

Modifier and Type	Method and Description
int	calculateK(Collection<Document> docs)
int	calculateK(Document doc)
double	calculatePk(Document doc, int k) Calculate Pk metric.
void	calculateScores(Collection<Document> docs)
void	calculateScoresFromAnnotations(Collection<Document> docs, Class<? extends Annotation> annotationClass)
double	calculateWD(Document doc, int k) Calculate WD metric.
protected void	clear()
double	getCountExpected()
double	getCountPredicted()
int[]	getMassesArray(Document doc, Annotation.Source source)
double	getMeanSegmentLength(Collection<Document> docs)
double	getPk()
int[]	getPositionsArray(Document doc, Annotation.Source source)
double	getScore()
double	getWD()
SegmentationEvaluation	withMergeEnabled(boolean enabled) Enable/disable merging adjacent sections with same label into one (in both GOLD and PRED).
SegmentationEvaluation	withRecalculateK(boolean enabled) Enable/disable recalculation of K for each document in the evaluation.

Methods inherited from class de.datexis.annotator.AnnotatorEvaluation

calculateScores, countAnnotations, countDocuments, countExamples, countSentences, countTokens, div, fDbl, fInt, fStr, printEvaluationStats

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_SCORE

protected final double DEFAULT_SCORE

See Also:

Constant Field Values

countExp

protected double countExp

countPred

protected double countPred

pksum

protected double pksum

wdsum

protected double wdsum

enableKPerDocument

protected boolean enableKPerDocument

whether to recalculate the value of K for each document in the evaluation

enableMergeSections

protected boolean enableMergeSections

whether to merge adjacent sections with same label into one (in both GOLD and PRED)

Constructor Detail

SegmentationEvaluation

public SegmentationEvaluation(String experimentName)

SegmentationEvaluation

public SegmentationEvaluation(String experimentName,
                              Annotation.Source expected,
                              Annotation.Source predicted)

Method Detail

withRecalculateK

public SegmentationEvaluation withRecalculateK(boolean enabled)

Enable/disable recalculation of K for each document in the evaluation. if FALSE, we use a fixed K for all documents.

withMergeEnabled

public SegmentationEvaluation withMergeEnabled(boolean enabled)

Enable/disable merging adjacent sections with same label into one (in both GOLD and PRED).

clear

protected void clear()

getScore

public double getScore()

Specified by:

getScore in class AnnotatorEvaluation

calculateScores

public void calculateScores(Collection<Document> docs)

Specified by:

calculateScores in class AnnotatorEvaluation

calculateScoresFromAnnotations

public void calculateScoresFromAnnotations(Collection<Document> docs,
                                           Class<? extends Annotation> annotationClass)

getWD

public double getWD()

getPk

public double getPk()

getCountExpected

public double getCountExpected()

getCountPredicted

public double getCountPredicted()

calculatePk

public double calculatePk(Document doc,
                          int k)

Calculate Pk metric. Adapted from https://github.com/cfournie/segmentation.evaluation

calculateWD

public double calculateWD(Document doc,
                          int k)

Calculate WD metric. Adapted from https://github.com/cfournie/segmentation.evaluation

calculateK

public int calculateK(Collection<Document> docs)

Returns:

preferred window size as half the mean segment length

calculateK

public int calculateK(Document doc)

getMeanSegmentLength

public double getMeanSegmentLength(Collection<Document> docs)

getMassesArray

public int[] getMassesArray(Document doc,
                            Annotation.Source source)

Returns:

a masses array from SectionAnnotations, e.g. [1,1,1,2,2,2,2,2,2,3,3,3,3,4,4,5,5,5,5] -> [3,6,3,2,4]

getPositionsArray

public int[] getPositionsArray(Document doc,
                               Annotation.Source source)

Returns:

a positions array from SectionAnnotations, e.g. [1,1,1,2,2,2,2,2,2,3,3,3,3,4,4,5,5,5,5]