com.aliasi.tokenizer

Class ModifiedTokenizerFactory

java.lang.Object

com.aliasi.tokenizer.ModifiedTokenizerFactory

All Implemented Interfaces:

TokenizerFactory

Direct Known Subclasses:

ModifyTokenTokenizerFactory

public abstract class ModifiedTokenizerFactory
extends Object
implements TokenizerFactory

A ModifiedTokenizerFactory is an abstract tokenizer factory that modifies a tokenizer returned by a base tokenizer factory.

The abstract method modify(Tokenizer) implements the modification.

Serialization

This class implements Serializable. There are no serialization methods defined, so the default serialization is used. There is a single reference to the base tokenizer factory, so a subclass will be serializable if all of its member objects are serializable and the base tokenizer is serializable.

It is good practice for each subclass to take completecontrol over serialization using a serialization proxy implemented on top of the AbstractExternalizable base class.

Thread Safety

A modified tokenizer factory is thread safe if its modify() method is thread safe.

Since:

Lingpipe3.8

Version:

3.8

Author:

Bob Carpenter

Constructor Summary

Constructors

Constructor and Description
ModifiedTokenizerFactory(TokenizerFactory baseFactory) Construct a modified tokenizer factory with the specified base factory.

Method Summary

Methods

Modifier and Type	Method and Description
TokenizerFactory	baseTokenizerFactory() Return the base tokenizer factory.
protected abstract Tokenizer	modify(Tokenizer tokenizer) Return a modified form of the specified tokenizer.
Tokenizer	tokenizer(char[] cs, int start, int length) Return the tokenizer for the specified character array slice, which is generated by the base tokenizer and modified with the modify method.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

ModifiedTokenizerFactory

public ModifiedTokenizerFactory(TokenizerFactory baseFactory)

Construct a modified tokenizer factory with the specified base factory.

Parameters:

baseFactory - Underlying tokenizer factory whose tokenizers are modified.

Method Detail

baseTokenizerFactory

public TokenizerFactory baseTokenizerFactory()

Return the base tokenizer factory.

Returns:

The base tokenizer factory.

tokenizer

public Tokenizer tokenizer(char[] cs,
                  int start,
                  int length)

Return the tokenizer for the specified character array slice, which is generated by the base tokenizer and modified with the modify method.

Specified by:

tokenizer in interface TokenizerFactory

Parameters:

cs - Characters to tokenize.

start - Index of first character to tokenize.

length - Number of characters to tokenize.

toString

public String toString()

Overrides:

toString in class Object

modify

protected abstract Tokenizer modify(Tokenizer tokenizer)

Return a modified form of the specified tokenizer. This method is used to modify the tokenizer produced by the base tokenizer in a call to tokenizer(char[],int,int).

Parameters:

tokenizer - Tokenizer to modify.

Returns:

The modified tokenizer.