com.sharethis.textrank

Class CoreNlpLanguageModel

java.lang.Object

com.sharethis.textrank.LanguageModel

com.sharethis.textrank.CoreNlpLanguageModel

public class CoreNlpLanguageModel
extends LanguageModel

Implementation of English-specific tools for natural language processing.

Field Summary

Fields inherited from class com.sharethis.textrank.LanguageModel

TOKEN_LENGTH_LIMIT

Constructor Summary

Constructors

Constructor and Description
CoreNlpLanguageModel(edu.stanford.nlp.pipeline.WordsToSentencesAnnotator splitter, edu.stanford.nlp.pipeline.TokenizerAnnotator tokenizer, edu.stanford.nlp.pipeline.POSTaggerAnnotator tagger, org.tartarus.snowball.SnowballProgram stemmer)

Method Summary

Modifier and Type	Method and Description
static LanguageModel	englishModel()
static LanguageModel	germanModel()
String	getNodeKey(String text, String pos) Prepare a stable key for a graph node (stemmed, lemmatized) from a token.
boolean	isAdjective(String pos) Determine whether the given PoS tag is an adjective.
boolean	isNoun(String pos) Determine whether the given PoS tag is a noun.
Sentence	parseSentence(String text) Tokenize and tag the sentence text.
String[]	splitParagraph(String text) Split sentences within the paragraph text.
String	stemToken(String token) Perform stemming on the given token.

Methods inherited from class com.sharethis.textrank.LanguageModel

buildLanguage, isRelevant, scrubToken

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CoreNlpLanguageModel

public CoreNlpLanguageModel(edu.stanford.nlp.pipeline.WordsToSentencesAnnotator splitter,
                            edu.stanford.nlp.pipeline.TokenizerAnnotator tokenizer,
                            edu.stanford.nlp.pipeline.POSTaggerAnnotator tagger,
                            org.tartarus.snowball.SnowballProgram stemmer)

Method Detail

englishModel

public static LanguageModel englishModel()

germanModel

public static LanguageModel germanModel()

splitParagraph

public String[] splitParagraph(String text)

Description copied from class: LanguageModel

Split sentences within the paragraph text.

Specified by:

splitParagraph in class LanguageModel

parseSentence

public Sentence parseSentence(String text)

Description copied from class: LanguageModel

Tokenize and tag the sentence text.

Specified by:

parseSentence in class LanguageModel

getNodeKey

public String getNodeKey(String text,
                         String pos)

Description copied from class: LanguageModel

Prepare a stable key for a graph node (stemmed, lemmatized) from a token.

Specified by:

getNodeKey in class LanguageModel

isNoun

public boolean isNoun(String pos)

Description copied from class: LanguageModel

Determine whether the given PoS tag is a noun.

Specified by:

isNoun in class LanguageModel

isAdjective

public boolean isAdjective(String pos)

Description copied from class: LanguageModel

Determine whether the given PoS tag is an adjective.

Specified by:

isAdjective in class LanguageModel

stemToken

public String stemToken(String token)

Description copied from class: LanguageModel

Perform stemming on the given token.

Specified by:

stemToken in class LanguageModel