de.datexis.sector.tagger

Class DocumentSentenceIterator

java.lang.Object

de.datexis.tagger.AbstractMultiDataSetIterator

de.datexis.sector.tagger.DocumentSentenceIterator

All Implemented Interfaces:

Serializable, Iterator<org.nd4j.linalg.dataset.api.MultiDataSet>, org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator

Direct Known Subclasses:

SectorTaggerIterator

public abstract class DocumentSentenceIterator
extends AbstractMultiDataSetIterator

A MultiDatasetIterator that returns one Document per Example, with Sentences as time steps.

Author:

Sebastian Arnold

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	DocumentSentenceIterator.DocumentBatch

Nested classes/interfaces inherited from class de.datexis.tagger.AbstractMultiDataSetIterator

AbstractMultiDataSetIterator.Stage

Field Summary

Fields inherited from class de.datexis.tagger.AbstractMultiDataSetIterator

batchSize, cursor, docIt, documents, log, maxTimeSeriesLength, numExamples, randomize, stage, startTime

Constructor Summary

Constructors

Constructor and Description
DocumentSentenceIterator(AbstractMultiDataSetIterator.Stage stage, Collection<Document> docs, int batchSize, boolean randomize)
DocumentSentenceIterator(AbstractMultiDataSetIterator.Stage stage, Collection<Document> docs, int numExamples, int maxTimeSeriesLength, int batchSize, boolean randomize)
DocumentSentenceIterator(AbstractMultiDataSetIterator.Stage stage, Dataset dataset, int batchSize, boolean randomize)

Method Summary

Modifier and Type	Method and Description
abstract org.nd4j.linalg.dataset.api.MultiDataSet	generateDataSet(DocumentSentenceIterator.DocumentBatch batch)
boolean	hasNext()
protected boolean	hasNextDocument()
org.nd4j.linalg.dataset.api.MultiDataSet	next(int num)
protected DocumentSentenceIterator.DocumentBatch	nextBatch(int num) Returns the next batch of documents.
protected Document	nextDocument()
DocumentSentenceIterator.DocumentBatch	nextDocumentBatch()
DocumentSentenceIterator.DocumentBatch	nextDocumentBatch(int num)
void	reset()

Methods inherited from class de.datexis.tagger.AbstractMultiDataSetIterator

asyncSupported, getNumExamples, getPreProcessor, next, reachedEnd, reportProgress, resetSupported, setPreProcessor

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining, remove

Constructor Detail

DocumentSentenceIterator

public DocumentSentenceIterator(AbstractMultiDataSetIterator.Stage stage,
                                Dataset dataset,
                                int batchSize,
                                boolean randomize)

DocumentSentenceIterator

public DocumentSentenceIterator(AbstractMultiDataSetIterator.Stage stage,
                                Collection<Document> docs,
                                int batchSize,
                                boolean randomize)

DocumentSentenceIterator

public DocumentSentenceIterator(AbstractMultiDataSetIterator.Stage stage,
                                Collection<Document> docs,
                                int numExamples,
                                int maxTimeSeriesLength,
                                int batchSize,
                                boolean randomize)

Method Detail

reset

public void reset()

hasNextDocument

protected boolean hasNextDocument()

hasNext

public boolean hasNext()

nextDocument

protected Document nextDocument()

nextBatch

protected DocumentSentenceIterator.DocumentBatch nextBatch(int num)

Returns the next batch of documents.

Parameters:

num - - batch size

Returns:

List of Documents and the size of the longest document (in Sentences)

next

public org.nd4j.linalg.dataset.api.MultiDataSet next(int num)

nextDocumentBatch

public DocumentSentenceIterator.DocumentBatch nextDocumentBatch()

nextDocumentBatch

public DocumentSentenceIterator.DocumentBatch nextDocumentBatch(int num)

generateDataSet

public abstract org.nd4j.linalg.dataset.api.MultiDataSet generateDataSet(DocumentSentenceIterator.DocumentBatch batch)