edu.umass.cs.mallet.base.fst

Class TransducerEvaluator

java.lang.Object

edu.umass.cs.mallet.base.fst.TransducerEvaluator

Direct Known Subclasses:

FieldF1Evaluator, IEEvaluator, InstanceAccuracyEvaluator, MultiSegmentationEvaluator, MultiSegmentationEvaluator, PerClassAccuracyEvaluator, SegmentationEvaluator, TokenAccuracyEvaluator

public abstract class TransducerEvaluator
extends Object

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	alwaysEvaluateWhenFinished
protected String	checkpointFilePrefix
protected int	checkpointIterationsToSkip
protected boolean	checkpointTransducer
protected int	numIterationsToSkip
protected int	numIterationsToWait
protected boolean	printModelAtEnd
protected boolean	viterbiOutput
protected String	viterbiOutputEncoding
protected String	viterbiOutputFilePrefix
protected int	viterbiOutputIterationsToSkip
protected int	viterbiOutputIterationsToWait

Constructor Summary

Constructors

Constructor and Description
TransducerEvaluator()

Method Summary

Modifier and Type	Method and Description
boolean	evaluate(Transducer crf, boolean finishedTraining, int iteration, boolean converged, double cost, InstanceList training, InstanceList validation, InstanceList testing) Evaluates a Tranducers on a given training, validation, and testing set.
String	getCheckpointFilePrefix()
int	getCheckpointIterationsToSkip()
boolean	getCheckpointTransducer()
int	getNumIterationsToSkip()
int	getNumIterationsToWait()
boolean	getViterbiOutput()
String	getViterbiOutputEncoding()
String	getViterbiOutputFilePrefix()
int	getViterbiOutputIterationsToSkip()
int	getViterbiOutputIterationsToWait()
boolean	isAlwaysEvaluateWhenFinished()
boolean	isPrintModelAtEnd()
void	setAlwaysEvaluateWhenFinished(boolean alwaysEvaluateWhenFinished)
void	setCheckpointFilePrefix(String p)
void	setCheckpointIterationsToSkip(int i)
void	setCheckpointTransducer(boolean c)
void	setNumIterationsToSkip(int numIterationsToSkip)
void	setNumIterationsToWait(int numIterationsToWait)
void	setPrintModelAtEnd(boolean printModelAtEnd)
void	setViterbiOutput(boolean vo)
void	setViterbiOutputEncoding(String o)
void	setViterbiOutputFilePrefix(String p)
void	setViterbiOutputIterationsToSkip(int i)
void	setViterbiOutputIterationsToWait(int i)
protected boolean	shouldDoEvaluate(int iteration, boolean finishedTraining) Returns whether an evaluator should run this iteration, based on the current values of numIterationsToWait, alwaysEvaluateWhenFinished, and numIteartionsToSkip
abstract void	test(Transducer transducer, InstanceList data, String description, PrintStream viterbiOutputStream)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

numIterationsToWait

protected int numIterationsToWait

numIterationsToSkip

protected int numIterationsToSkip

alwaysEvaluateWhenFinished

protected boolean alwaysEvaluateWhenFinished

printModelAtEnd

protected boolean printModelAtEnd

checkpointTransducer

protected boolean checkpointTransducer

checkpointFilePrefix

protected String checkpointFilePrefix

checkpointIterationsToSkip

protected int checkpointIterationsToSkip

viterbiOutput

protected boolean viterbiOutput

viterbiOutputFilePrefix

protected String viterbiOutputFilePrefix

viterbiOutputIterationsToWait

protected int viterbiOutputIterationsToWait

viterbiOutputIterationsToSkip

protected int viterbiOutputIterationsToSkip

viterbiOutputEncoding

protected String viterbiOutputEncoding

Constructor Detail

TransducerEvaluator

public TransducerEvaluator()

Method Detail

getNumIterationsToWait

public int getNumIterationsToWait()

setNumIterationsToWait

public void setNumIterationsToWait(int numIterationsToWait)

getNumIterationsToSkip

public int getNumIterationsToSkip()

setNumIterationsToSkip

public void setNumIterationsToSkip(int numIterationsToSkip)

isAlwaysEvaluateWhenFinished

public boolean isAlwaysEvaluateWhenFinished()

setAlwaysEvaluateWhenFinished

public void setAlwaysEvaluateWhenFinished(boolean alwaysEvaluateWhenFinished)

isPrintModelAtEnd

public boolean isPrintModelAtEnd()

setPrintModelAtEnd

public void setPrintModelAtEnd(boolean printModelAtEnd)

setViterbiOutput

public void setViterbiOutput(boolean vo)

getViterbiOutput

public boolean getViterbiOutput()

setViterbiOutputFilePrefix

public void setViterbiOutputFilePrefix(String p)

getViterbiOutputFilePrefix

public String getViterbiOutputFilePrefix()

setViterbiOutputIterationsToWait

public void setViterbiOutputIterationsToWait(int i)

getViterbiOutputIterationsToWait

public int getViterbiOutputIterationsToWait()

setViterbiOutputIterationsToSkip

public void setViterbiOutputIterationsToSkip(int i)

getViterbiOutputIterationsToSkip

public int getViterbiOutputIterationsToSkip()

setViterbiOutputEncoding

public void setViterbiOutputEncoding(String o)

getViterbiOutputEncoding

public String getViterbiOutputEncoding()

setCheckpointTransducer

public void setCheckpointTransducer(boolean c)

getCheckpointTransducer

public boolean getCheckpointTransducer()

setCheckpointFilePrefix

public void setCheckpointFilePrefix(String p)

getCheckpointFilePrefix

public String getCheckpointFilePrefix()

setCheckpointIterationsToSkip

public void setCheckpointIterationsToSkip(int i)

getCheckpointIterationsToSkip

public int getCheckpointIterationsToSkip()

shouldDoEvaluate

protected boolean shouldDoEvaluate(int iteration,
                                   boolean finishedTraining)

Returns whether an evaluator should run this iteration, based on the current values of numIterationsToWait, alwaysEvaluateWhenFinished, and numIteartionsToSkip

Parameters:

iteration - Training iteration

finishedTraining - Whether training has finished

Returns:

boolean Whether to evaluate

evaluate

public boolean evaluate(Transducer crf,
                        boolean finishedTraining,
                        int iteration,
                        boolean converged,
                        double cost,
                        InstanceList training,
                        InstanceList validation,
                        InstanceList testing)

Evaluates a Tranducers on a given training, validation, and testing set. .

The default implementation calls the evaluator's test on the training, validation, and testing sets.

Parameters:

crf - The transducer to evaluate.

finishedTraining - Whether training has finished. Evaluators can be set to always evaluate when training completes.

iteration - The current iteration

converged - Whether training has converged. Most evaluators ignore this.

cost - The current cost during training. Some evaluators may include this in the output.

training - The training set. If null, the argument is ignored.

validation - The validation set. If null, the argument is ignored.

testing - The testing set. If null, the argument is ignored.

Returns:

Whether training should continue.

test

public abstract void test(Transducer transducer,
                          InstanceList data,
                          String description,
                          PrintStream viterbiOutputStream)