public class ChunkingEvaluation extends Object
ChunkingEvaluation stores and reports the results of
evaluating response chunkings against reference chunkings. Cases
to evaluate are supplied in the form of a reference and response
chunking through the method addCase(Chunking,Chunking).
The sets of true positive,
false positive and false negative chunks are available through the
methods truePositiveSet(), falsePositiveSet(),
and falseNegativeSet(). True positives are chunks that
are in both the reference and response, false positives are chunks
in the response but not the reference, and false negatives are in
the reference, but not the response. There is no notion of true
negative in this task, a fact that is reflected in the results of
the precision-recall evaluation.
The main method of reporting is through an instance of ScoredPrecisionRecallEvaluation returned by
the method precisionRecallEvaluation(). The return result
provides an object capable of extensive reporting for scored
classification tasks such as chunking. The instances of true and
false positive and negatives are described above; their scores are
derived from response scores.
This evaluator works solely on the basis of chunk offset and exact match. There is no notion of alignment or mapping, as found, for example, in the MUC Scoring Software User's Manual, and its descendants such as the 2005 ACE Evaluation Plan. In this regard, we follow the model of CoNLL 2000 Chunking Task.
This evaluation is able to handle overlapping chunks with
results being reported in the same manner. In particular, the
labeled precision and recall components of the approach that later
became known as PARSEVAL can
be generated by using the ChunkingEvaluation class.
| Constructor and Description |
|---|
ChunkingEvaluation()
Construct a chunking evaluation.
|
| Modifier and Type | Method and Description |
|---|---|
void |
addCase(Chunking referenceChunking,
Chunking responseChunking)
Add an evaluation case consisting of a reference chunk
set and a response chunk set.
|
Set<Chunking[]> |
cases()
Return the set of cases consisting of pairs of reference and
response chunkings.
|
Set<ChunkAndCharSeq> |
falseNegativeSet()
Returns the set of false negatives.
|
Set<ChunkAndCharSeq> |
falsePositiveSet()
Returns the set of false positives.
|
ChunkingEvaluation |
perTypeEvaluation(String chunkType)
Returns a chunking evaluation which consists of the current
chunking evaluation restricted to the specified type.
|
PrecisionRecallEvaluation |
precisionRecallEvaluation()
Return the scored precision-recall evaluation for this chunker.
|
String |
toString()
Returns the precision-recall evaluation for this chunking
as a string.
|
Set<ChunkAndCharSeq> |
truePositiveSet()
Returns the set of true positives.
|
public Set<Chunking[]> cases()
Chunking[], with the first element being the
reference chunk and the second element being the response
chunk.
The set returned is an unmodifiable view of the underlying set of cases and will change as cases are added to this evaluation.
public ChunkingEvaluation perTypeEvaluation(String chunkType)
chunkType - Type of chunk to be evaluated.public void addCase(Chunking referenceChunking, Chunking responseChunking)
referenceChunking - Chunking of reference chunks.responseChunking - Chunking of response chunks.IllegalArgumentException - If the chunkings are not
over the same character sequence.public Set<ChunkAndCharSeq> truePositiveSet()
ChunkAndCharSeq,
which combine a chunk and a character sequence.
The set is unmodifiable, but tracks the changes in this evaluator.
public Set<ChunkAndCharSeq> falsePositiveSet()
ChunkAndCharSeq, which combine a
chunk and a character sequence.
The set is unmodifiable, but tracks the changes in this evaluator.
public Set<ChunkAndCharSeq> falseNegativeSet()
ChunkAndCharSeq, which
combine a chunk and a character sequence.
The set is unmodifiable, but tracks the changes in this evaluator.
public PrecisionRecallEvaluation precisionRecallEvaluation()
Copyright © 2016 Alias-i, Inc.. All rights reserved.