de.datexis.sector.tagger

Class SectorTaggerIterator

java.lang.Object

de.datexis.tagger.AbstractMultiDataSetIterator

de.datexis.sector.tagger.DocumentSentenceIterator

de.datexis.sector.tagger.SectorTaggerIterator

All Implemented Interfaces:

Serializable, Iterator<org.nd4j.linalg.dataset.api.MultiDataSet>, org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator

public class SectorTaggerIterator
extends DocumentSentenceIterator

Iterates through a Dataset with Document-Level Batches of Sentences

Author:

Sebastian Arnold

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class de.datexis.sector.tagger.DocumentSentenceIterator

DocumentSentenceIterator.DocumentBatch

Nested classes/interfaces inherited from class de.datexis.tagger.AbstractMultiDataSetIterator

AbstractMultiDataSetIterator.Stage

Field Summary

Fields

Modifier and Type	Field and Description
protected EncoderSet	inputEncoders
protected boolean	requireSubsampling
protected SectorTagger	tagger
protected EncoderSet	targetEncoders

Fields inherited from class de.datexis.tagger.AbstractMultiDataSetIterator

batchSize, cursor, docIt, documents, log, maxTimeSeriesLength, numExamples, randomize, stage, startTime

Constructor Summary

Constructors

Constructor and Description
SectorTaggerIterator(AbstractMultiDataSetIterator.Stage stage, Collection<Document> docs, SectorTagger tagger, int batchSize, boolean randomize, boolean requireSubsampling)
SectorTaggerIterator(AbstractMultiDataSetIterator.Stage stage, Collection<Document> docs, SectorTagger tagger, int numExamples, int batchSize, boolean randomize, boolean requireSubsampling)
SectorTaggerIterator(AbstractMultiDataSetIterator.Stage stage, Collection<Document> docs, SectorTagger tagger, int numExamples, int maxTimeSeriesLength, int batchSize, boolean randomize, boolean requireSubsampling)
SectorTaggerIterator(AbstractMultiDataSetIterator.Stage stage, Dataset dataset, SectorTagger tagger, int batchSize, boolean randomize, boolean useMultiClassLabels)

Method Summary

Modifier and Type	Method and Description
boolean	asyncSupported()
org.nd4j.linalg.api.ndarray.INDArray	createMask(List<Document> input, int maxTimeSteps, Class<? extends Span> timeStepClass)
protected org.nd4j.linalg.api.ndarray.INDArray	encodeAnnotation(Encoder enc, SectionAnnotation ann)
org.nd4j.linalg.api.ndarray.INDArray	encodeTarget(List<Document> input, int maxTimeSteps, Class<? extends Span> timeStepClass)
org.nd4j.linalg.dataset.api.MultiDataSet	generateDataSet(DocumentSentenceIterator.DocumentBatch batch)

Methods inherited from class de.datexis.sector.tagger.DocumentSentenceIterator

hasNext, hasNextDocument, next, nextBatch, nextDocument, nextDocumentBatch, nextDocumentBatch, reset

Methods inherited from class de.datexis.tagger.AbstractMultiDataSetIterator

getNumExamples, getPreProcessor, next, reachedEnd, reportProgress, resetSupported, setPreProcessor

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining, remove

Field Detail

inputEncoders

protected EncoderSet inputEncoders

targetEncoders

protected EncoderSet targetEncoders

tagger

protected SectorTagger tagger

requireSubsampling

protected boolean requireSubsampling

Constructor Detail

SectorTaggerIterator

public SectorTaggerIterator(AbstractMultiDataSetIterator.Stage stage,
                            Dataset dataset,
                            SectorTagger tagger,
                            int batchSize,
                            boolean randomize,
                            boolean useMultiClassLabels)

SectorTaggerIterator

public SectorTaggerIterator(AbstractMultiDataSetIterator.Stage stage,
                            Collection<Document> docs,
                            SectorTagger tagger,
                            int batchSize,
                            boolean randomize,
                            boolean requireSubsampling)

SectorTaggerIterator

public SectorTaggerIterator(AbstractMultiDataSetIterator.Stage stage,
                            Collection<Document> docs,
                            SectorTagger tagger,
                            int numExamples,
                            int batchSize,
                            boolean randomize,
                            boolean requireSubsampling)

SectorTaggerIterator

public SectorTaggerIterator(AbstractMultiDataSetIterator.Stage stage,
                            Collection<Document> docs,
                            SectorTagger tagger,
                            int numExamples,
                            int maxTimeSeriesLength,
                            int batchSize,
                            boolean randomize,
                            boolean requireSubsampling)

Method Detail

asyncSupported

public boolean asyncSupported()

Specified by:

asyncSupported in interface org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator

Overrides:

asyncSupported in class AbstractMultiDataSetIterator

generateDataSet

public org.nd4j.linalg.dataset.api.MultiDataSet generateDataSet(DocumentSentenceIterator.DocumentBatch batch)

Specified by:

generateDataSet in class DocumentSentenceIterator

createMask

public org.nd4j.linalg.api.ndarray.INDArray createMask(List<Document> input,
                                                       int maxTimeSteps,
                                                       Class<? extends Span> timeStepClass)

encodeTarget

public org.nd4j.linalg.api.ndarray.INDArray encodeTarget(List<Document> input,
                                                         int maxTimeSteps,
                                                         Class<? extends Span> timeStepClass)

encodeAnnotation

protected org.nd4j.linalg.api.ndarray.INDArray encodeAnnotation(Encoder enc,
                                                                SectionAnnotation ann)