de.datexis.ner

Class MentionAnnotator

java.lang.Object

de.datexis.annotator.Annotator

de.datexis.ner.MentionAnnotator

Direct Known Subclasses:

GenericMentionAnnotator

public class MentionAnnotator
extends Annotator

A simple NER Annotator that returns a Dataset with Named Entity mentions

Author:

Sebastian Arnold

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MentionAnnotator.Builder

Field Summary

Fields

Modifier and Type	Field and Description
protected Resource	bestModel
protected static org.slf4j.Logger	log

Fields inherited from class de.datexis.annotator.Annotator

components, it, provenance, tagger

Constructor Summary

Constructors

Constructor and Description
MentionAnnotator()
MentionAnnotator(Tagger root)

Method Summary

Modifier and Type	Method and Description
void	annotate(Collection<Document> docs)
protected void	createAnnotations(Iterable<Document> docs, Annotation.Source expected)
protected void	createTags(Iterable<Document> docs, Annotation.Source expected)
MentionTagger	getTagger()
String	toString()
void	trainModel(Collection<Sentence> sentences, Annotation.Source tagSource, WordHelpers.Language lang) Trains the NER model using selected Sentences (BIO2Tags are required and will not be generated)
void	trainModel(Dataset train, Annotation.Source annotationSource, WordHelpers.Language lang)
void	trainModel(Dataset train, Annotation.Source annotationSource, WordHelpers.Language lang, int limitExamples, boolean incremental, boolean randomize) Trains the NER model using a complete Dataset with MentionAnnotations
void	trainModel(Dataset train, Dataset test, WordHelpers.Language lang)
void	trainModelEarlyStopping(Dataset train, Dataset validation, Annotation.Source annotationSource, WordHelpers.Language lang, int epochSize, int minEpochs, int maxEpochs, int maxEpochsWithNoImprovement)
void	writeBestModel(Resource path, String name) Writes .xml and binary models

Methods inherited from class de.datexis.annotator.Annotator

addComponent, annotate, annotate, annotate, createDataset, createDocument, getComponent, getProvenance, isModelAvailable, isModelAvailableInChildren, readModel, trainModel, writeComponents, writeHTML, writeModel, writeModel, writeTestLog, writeTrainLog

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

log

protected static final org.slf4j.Logger log

bestModel

protected Resource bestModel

Constructor Detail

MentionAnnotator

public MentionAnnotator()

MentionAnnotator

public MentionAnnotator(Tagger root)

Method Detail

getTagger

public MentionTagger getTagger()

Overrides:

getTagger in class Annotator

toString

public String toString()

Overrides:

toString in class Object

annotate

public void annotate(Collection<Document> docs)

Overrides:

annotate in class Annotator

trainModel

public void trainModel(Dataset train,
                       Dataset test,
                       WordHelpers.Language lang)

trainModel

public void trainModel(Dataset train,
                       Annotation.Source annotationSource,
                       WordHelpers.Language lang)

trainModel

public void trainModel(Dataset train,
                       Annotation.Source annotationSource,
                       WordHelpers.Language lang,
                       int limitExamples,
                       boolean incremental,
                       boolean randomize)

Trains the NER model using a complete Dataset with MentionAnnotations

Parameters:

train - - dataset do use for training

annotationSource - - use MentionAnnotations of given source to create tags

lang - - language used for preprocessing

limitExamples - - use only n examples for training (after randomization), -1 for all examples

incremental - - don't reset the model before training

randomize - - randomize examples after each epoch

trainModelEarlyStopping

public void trainModelEarlyStopping(Dataset train,
                                    Dataset validation,
                                    Annotation.Source annotationSource,
                                    WordHelpers.Language lang,
                                    int epochSize,
                                    int minEpochs,
                                    int maxEpochs,
                                    int maxEpochsWithNoImprovement)

writeBestModel

public void writeBestModel(Resource path,
                           String name)
                    throws IOException

Writes .xml and binary models

Parameters:

path - Directory to write to

Throws:

IOException

trainModel

public void trainModel(Collection<Sentence> sentences,
                       Annotation.Source tagSource,
                       WordHelpers.Language lang)

Trains the NER model using selected Sentences (BIO2Tags are required and will not be generated)

Parameters:

sentences -

tagSource -

lang -

createTags

protected void createTags(Iterable<Document> docs,
                          Annotation.Source expected)

createAnnotations

protected void createAnnotations(Iterable<Document> docs,
                                 Annotation.Source expected)