smile.nlp.collocation

Class BigramCollocationFinder

java.lang.Object

smile.nlp.collocation.BigramCollocationFinder

public class BigramCollocationFinder
extends java.lang.Object

Tools to identify collocations (words that often appear consecutively) within corpora. They may also be used to find other associations between word occurrences.

Finding collocations requires first calculating the frequencies of words and their appearance in the context of other words. Often the collection of words will then requiring filtering to only retain useful content terms. Each ngram of words may then be scored according to some association measure, in order to determine the relative likelihood of each ngram being a collocation.

Constructor Summary

Constructors

Constructor and Description
BigramCollocationFinder(int minFreq) Constructor.

Method Summary

Modifier and Type	Method and Description
BigramCollocation[]	find(Corpus corpus, double p) Finds bigram collocations in the given corpus whose p-value is less than the given threshold.
BigramCollocation[]	find(Corpus corpus, int k) Finds top k bigram collocations in the given corpus.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BigramCollocationFinder

public BigramCollocationFinder(int minFreq)

Constructor.

Parameters:

minFreq - the minimum frequency of collocation.

Method Detail

find

public BigramCollocation[] find(Corpus corpus,
                                int k)

Finds top k bigram collocations in the given corpus.

Returns:

the array of significant bigram collocations in descending order of likelihood ratio.

find

public BigramCollocation[] find(Corpus corpus,
                                double p)

Finds bigram collocations in the given corpus whose p-value is less than the given threshold.

Parameters:

p - the p-value threshold

Returns:

the array of significant bigram collocations in descending order of likelihood ratio.