de.julielab.jsbd

Class SentenceSplitter

java.lang.Object

de.julielab.jsbd.SentenceSplitter

public class SentenceSplitter
extends Object

Constructor Summary

Constructors

Constructor and Description
SentenceSplitter()

Method Summary

Methods

Modifier and Type	Method and Description
ArrayList<String>	getLabelsFromLabelSequence(cc.mallet.types.LabelSequence ls)
cc.mallet.fst.CRF	getModel()
cc.mallet.types.Instance	makePredictionData(ArrayList<String> lines, cc.mallet.pipe.Pipe myPipe) creates a single instance from the arraylist with lines provided and the given pipe
cc.mallet.types.InstanceList	makePredictionData(File[] predictFiles, cc.mallet.pipe.Pipe myPipe) creates a list of instances with the pipe provided from the given array of files
cc.mallet.types.Instance	makePredictionData(File predictFile, cc.mallet.pipe.Pipe myPipe) creates a single instance from the file provided and the given pipe
cc.mallet.types.InstanceList	makeTrainingData(File[] trainFiles, boolean useTokenOffset)
ArrayList<String>	postprocessingFilter(ArrayList<String> predLabels, ArrayList<Unit> units) a postprocessing filter (to be used after prediction) which can correct known errors
ArrayList<Unit>	predict(ArrayList<String> lines, boolean doPostprocessing) predict a couple of lines
ArrayList<Unit>	predict(cc.mallet.types.Instance inst, boolean doPostProcessing) predict a single Instance
ArrayList<String>	readFile(File myFile)
void	readModel(File file) load a previously trained FeatureSubsetModel (CRF4+Properties) which was stored as serialized object to disk.
void	readModel(InputStream is)
void	train(cc.mallet.types.InstanceList instList, cc.mallet.pipe.Pipe dataPipe)
void	writeModel(String filename) Save the model learned to disk.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SentenceSplitter

public SentenceSplitter()

Method Detail

makePredictionData

public cc.mallet.types.Instance makePredictionData(ArrayList<String> lines,
                                          cc.mallet.pipe.Pipe myPipe)

creates a single instance from the arraylist with lines provided and the given pipe

makePredictionData

public cc.mallet.types.Instance makePredictionData(File predictFile,
                                          cc.mallet.pipe.Pipe myPipe)

creates a single instance from the file provided and the given pipe

makePredictionData

public cc.mallet.types.InstanceList makePredictionData(File[] predictFiles,
                                              cc.mallet.pipe.Pipe myPipe)

creates a list of instances with the pipe provided from the given array of files

makeTrainingData

public cc.mallet.types.InstanceList makeTrainingData(File[] trainFiles,
                                            boolean useTokenOffset)

Parameters:

trainFiles -

useTokenOffset - if true the tokens offset and not is string representation is stored in the instance source

Returns:

InstanceList with training data

train

public void train(cc.mallet.types.InstanceList instList,
         cc.mallet.pipe.Pipe dataPipe)

predict

public ArrayList<Unit> predict(ArrayList<String> lines,
                      boolean doPostprocessing)

predict a couple of lines

Parameters:

lines -

doPostprocessing -

Returns:

ArrayList of Unit objects

predict

public ArrayList<Unit> predict(cc.mallet.types.Instance inst,
                      boolean doPostProcessing)

predict a single Instance

Parameters:

inst -

doPostProcessing -

Returns:

ArrayList of Unit objects

postprocessingFilter

public ArrayList<String> postprocessingFilter(ArrayList<String> predLabels,
                                     ArrayList<Unit> units)

a postprocessing filter (to be used after prediction) which can correct known errors

Parameters:

predLabels -

units -

abbrList -

Returns:

getLabelsFromLabelSequence

public ArrayList<String> getLabelsFromLabelSequence(cc.mallet.types.LabelSequence ls)

writeModel

public void writeModel(String filename)

Save the model learned to disk. THis is done via Java's object serialization.

Parameters:

filename - where to write it (full path!)

readModel

public void readModel(File file)
               throws IOException,
                      FileNotFoundException,
                      ClassNotFoundException

load a previously trained FeatureSubsetModel (CRF4+Properties) which was stored as serialized object to disk.

Parameters:

filename - where to find the serialized featureSubsetModel (full path!)

Throws:

IOException

FileNotFoundException

ClassNotFoundException

readModel

public void readModel(InputStream is)
               throws IOException,
                      ClassNotFoundException

Throws:

IOException

ClassNotFoundException

readFile

public ArrayList<String> readFile(File myFile)

getModel

public cc.mallet.fst.CRF getModel()