de.datexis.ner.reader

Class CoNLLDatasetReader

java.lang.Object

de.datexis.ner.reader.CoNLLDatasetReader

All Implemented Interfaces:

DatasetReader

public class CoNLLDatasetReader
extends Object
implements DatasetReader

Reads a Dataset from CoNLL formatted file.

Author:

sarnold

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CoNLLDatasetReader.Charset

Field Summary

Fields

Modifier and Type	Field and Description
protected Annotation.Source	annotationSource
protected String	name
protected int	tagIndex
protected String	type
protected boolean	useFirstSentenceAsTitle

Constructor Summary

Constructors

Constructor and Description
CoNLLDatasetReader()

Method Summary

Modifier and Type	Method and Description
protected BIO2Tag	createTag(String label, String prevType) Returns the NER Label based on current and previous tags
protected Token	createTokenFromLine(String line, int cursor, String prevType) Creates Token from the given line of CoNLL2003 data
Dataset	read(Resource path) Read a Dataset from CoNLL file
Dataset	read(Resource path, CoNLLDatasetReader.Charset charset) Read a Dataset from CoNLL file
static Dataset	readDataset(Resource path, String name, CoNLLDatasetReader.Charset charset) Read a Dataset from CoNLL file (static version with default reader)
protected Dataset	readLines(Iterator<String> lines) Create a Document from the given data
CoNLLDatasetReader	withAnnotationSource(Annotation.Source annotationSource)
CoNLLDatasetReader	withFirstSentenceAsTitle(boolean useFirstSentence) Use a copy of every first sentence as Document title.
CoNLLDatasetReader	withGenericType(String type)
CoNLLDatasetReader	withName(String name) Use a specific name for the Dataset.
CoNLLDatasetReader	withTagIndex(int tagIndex) Use a specific column as NER tag.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

useFirstSentenceAsTitle

protected boolean useFirstSentenceAsTitle

annotationSource

protected Annotation.Source annotationSource

tagIndex

protected int tagIndex

type

protected String type

name

protected String name

Constructor Detail

CoNLLDatasetReader

public CoNLLDatasetReader()

Method Detail

withName

public CoNLLDatasetReader withName(String name)

Use a specific name for the Dataset.

withTagIndex

public CoNLLDatasetReader withTagIndex(int tagIndex)

Use a specific column as NER tag.

Parameters:

tagIndex - Use this index, starting from 0. Default: last column.

withFirstSentenceAsTitle

public CoNLLDatasetReader withFirstSentenceAsTitle(boolean useFirstSentence)

Use a copy of every first sentence as Document title.

withAnnotationSource

public CoNLLDatasetReader withAnnotationSource(Annotation.Source annotationSource)

Parameters:

annotationSource - Assign this source to all Annotations read from the file.

withGenericType

public CoNLLDatasetReader withGenericType(String type)

Parameters:

type - Use this type instead of the given ones in the dataset,

read

public Dataset read(Resource path)
             throws IOException

Read a Dataset from CoNLL file

Specified by:

read in interface DatasetReader

Throws:

IOException

read

public Dataset read(Resource path,
                    CoNLLDatasetReader.Charset charset)
             throws IOException

Read a Dataset from CoNLL file

Throws:

IOException

readDataset

public static Dataset readDataset(Resource path,
                                  String name,
                                  CoNLLDatasetReader.Charset charset)
                           throws IOException

Read a Dataset from CoNLL file (static version with default reader)

Throws:

IOException

readLines

protected Dataset readLines(Iterator<String> lines)

Create a Document from the given data

Returns:

- Document created from data

createTokenFromLine

protected Token createTokenFromLine(String line,
                                    int cursor,
                                    String prevType)

Creates Token from the given line of CoNLL2003 data

Parameters:

line - - CoNLL2003 data to create Token

cursor - - character index in the whole document

Returns:

Token created from line

createTag

protected BIO2Tag createTag(String label,
                            String prevType)

Returns the NER Label based on current and previous tags