de.datexis.sector.encoder

Class HeadingEncoder

java.lang.Object

de.datexis.annotator.AnnotatorComponent

de.datexis.encoder.Encoder

de.datexis.encoder.LookupCacheEncoder

de.datexis.encoder.impl.BagOfWordsEncoder

de.datexis.sector.encoder.HeadingEncoder

All Implemented Interfaces:

IComponent, IEncoder

public class HeadingEncoder
extends BagOfWordsEncoder

Wrapper for Bag-Of-Words Headings with OTHER class and minWordLength

Author:

Sebastian Arnold

Field Summary

Fields

Modifier and Type	Field and Description
static String	ID
protected static org.slf4j.Logger	log
static String	OTHER_CLASS

Fields inherited from class de.datexis.encoder.impl.BagOfWordsEncoder

language, preprocessor, wordHelpers

Fields inherited from class de.datexis.encoder.LookupCacheEncoder

totalWords, vocab

Fields inherited from class de.datexis.annotator.AnnotatorComponent

id, model, modelAvailable, name, timer

Constructor Summary

Constructors

Constructor and Description
HeadingEncoder()

Method Summary

Modifier and Type	Method and Description
org.nd4j.linalg.api.ndarray.INDArray	encode(Iterable<? extends Span> spans)
org.nd4j.linalg.api.ndarray.INDArray	encode(String phrase)
protected org.nd4j.linalg.api.ndarray.INDArray	encode(String[] words)
protected org.nd4j.linalg.api.ndarray.INDArray	encodeOtherClass()
org.nd4j.linalg.api.ndarray.INDArray	encodeSubsampled(String phrase)
Collection<String>	getNearestNeighbours(org.nd4j.linalg.api.ndarray.INDArray v, int maxN)
void	trainModel(List<String> headlines, int minWordFrequency, int minWordLength, WordHelpers.Language language)

Methods inherited from class de.datexis.encoder.impl.BagOfWordsEncoder

asString, encode, getConfidence, getFrequency, getIndex, getLanguage, getMaxConfidence, getName, getNearestNeighbour, getProbability, isUnknown, keepWord, samplingRate, samplingRate, setLanguage, subsample, subsampleWeights, trainModel, trainModel, trainModel

Methods inherited from class de.datexis.encoder.LookupCacheEncoder

getEmbeddingVectorSize, getNearestNeighbours, getTotalWords, getWord, getWords, loadModel, oneHot, saveModel, setTotalWords

Methods inherited from class de.datexis.encoder.Encoder

encodeEach, encodeEach, encodeEach, encodeMatrix, trainModel

Methods inherited from class de.datexis.annotator.AnnotatorComponent

appendTestLog, appendTestLog, appendTrainLog, appendTrainLog, clearTestLog, clearTrainLog, getConf, getId, getModel, getTestLog, getTrainLog, isModelAvailable, isModelAvailableInChildren, setConf, setId, setModel, setModelAvailable, setModelFilename, setName

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface de.datexis.annotator.IComponent

getEncoders, getId, isModelAvailable, setEncoders

Field Detail

log

protected static final org.slf4j.Logger log

ID

public static final String ID

See Also:

Constant Field Values

OTHER_CLASS

public static String OTHER_CLASS

Constructor Detail

HeadingEncoder

public HeadingEncoder()

Method Detail

trainModel

public void trainModel(List<String> headlines,
                       int minWordFrequency,
                       int minWordLength,
                       WordHelpers.Language language)

encode

public org.nd4j.linalg.api.ndarray.INDArray encode(String phrase)

Specified by:

encode in interface IEncoder

Overrides:

encode in class BagOfWordsEncoder

encode

public org.nd4j.linalg.api.ndarray.INDArray encode(Iterable<? extends Span> spans)

Specified by:

encode in interface IEncoder

Overrides:

encode in class BagOfWordsEncoder

encode

protected org.nd4j.linalg.api.ndarray.INDArray encode(String[] words)

Overrides:

encode in class BagOfWordsEncoder

encodeSubsampled

public org.nd4j.linalg.api.ndarray.INDArray encodeSubsampled(String phrase)

Overrides:

encodeSubsampled in class BagOfWordsEncoder

encodeOtherClass

protected org.nd4j.linalg.api.ndarray.INDArray encodeOtherClass()

getNearestNeighbours

public Collection<String> getNearestNeighbours(org.nd4j.linalg.api.ndarray.INDArray v,
                                               int maxN)

Overrides:

getNearestNeighbours in class BagOfWordsEncoder