de.datexis.sector.reader

Class MultiSegReader

java.lang.Object

de.datexis.reader.DirectoryDatasetReader<RawTextDatasetReader>

de.datexis.reader.RawTextDatasetReader

de.datexis.sector.reader.MultiSegReader

All Implemented Interfaces:

DatasetReader

public class MultiSegReader
extends RawTextDatasetReader

Reader for MultiSeg dataset by Ivan Titov: "Multi-document topic segmentation". CIKM '10, Pages 1119-1128

Author:

Sebastian Arnold

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.slf4j.Logger	log

Fields inherited from class de.datexis.reader.RawTextDatasetReader

generateUIDs, isTokenized, useFirstSentenceAsTitle

Fields inherited from class de.datexis.reader.DirectoryDatasetReader

limit, randomizeDocuments

Constructor Summary

Constructors

Constructor and Description
MultiSegReader()

Method Summary

Modifier and Type	Method and Description
Dataset	read(Resource path)
Document	readDocumentFromFile(Resource file) Read a single Document from file.
protected TreeSet[]	readSectionsFromLabel(Resource file, int docNum) Return a set of lines where a new sections starts.

Methods inherited from class de.datexis.reader.RawTextDatasetReader

withFirstSentenceAsTitle, withGeneratedUIDs, withTokenizedInput

Methods inherited from class de.datexis.reader.DirectoryDatasetReader

readDatasetFromDirectory, readDatasetFromDirectory, stream, streamDocumentsFromDirectory, tryReadDocumentsFromFile, withLimitNumberOfDocuments, withRandomizedDocuments

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

protected static final org.slf4j.Logger log

Constructor Detail

MultiSegReader

public MultiSegReader()

Method Detail

read

public Dataset read(Resource path)
             throws IOException

Specified by:

read in interface DatasetReader

Overrides:

read in class DirectoryDatasetReader<RawTextDatasetReader>

Throws:

IOException

readDocumentFromFile

public Document readDocumentFromFile(Resource file)

Read a single Document from file.

Overrides:

readDocumentFromFile in class RawTextDatasetReader

readSectionsFromLabel

protected TreeSet[] readSectionsFromLabel(Resource file,
                                          int docNum)
                                   throws IOException

Return a set of lines where a new sections starts.

Throws:

IOException