edu.umass.cs.mallet.base.extract

Class HierarchicalTokenizationFilter

java.lang.Object

edu.umass.cs.mallet.base.extract.HierarchicalTokenizationFilter

All Implemented Interfaces:

TokenizationFilter

public class HierarchicalTokenizationFilter
extends Object
implements TokenizationFilter

Tokenization filter that will create nested spans based on a hierarchical labeling of the data. The labels should be of the form LBL1[|LBLk]*. For example,

   A   A|B   A|B|C   A|B|C  A|B  A   A
   w1  w2    w3      w4     w5   w6  w7
 

will result in LabeledSpans like <A>w1 <B>w2 <C>w3 w4</C> w5</B> w6 w7</A> Also, labels of the form <B-field> will force a new instance of the field to begin, even if it is already active. And prefixes of I- are ignored so you can use BIO labeling. Created: Nov 12, 2004

Version:

$Id: HierarchicalTokenizationFilter.java,v 1.2 2005/03/31 22:33:42 casutton Exp $

Author:

Constructor Summary

Constructors

Constructor and Description
HierarchicalTokenizationFilter()
HierarchicalTokenizationFilter(Pattern ignorePattern)

Method Summary

Modifier and Type	Method and Description
LabeledSpans	constructLabeledSpans(LabelAlphabet dict, Object document, Label backgroundTag, Tokenization input, Sequence seq) Converts a the sequence of labels into a set of labeled spans.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HierarchicalTokenizationFilter

public HierarchicalTokenizationFilter()

HierarchicalTokenizationFilter

public HierarchicalTokenizationFilter(Pattern ignorePattern)

Method Detail

constructLabeledSpans

public LabeledSpans constructLabeledSpans(LabelAlphabet dict,
                                          Object document,
                                          Label backgroundTag,
                                          Tokenization input,
                                          Sequence seq)

Description copied from interface: TokenizationFilter

Converts a the sequence of labels into a set of labeled spans. Essentially, this converts the output of sequence labeling into an extraction output.

Specified by:

constructLabeledSpans in interface TokenizationFilter

Returns: