net.sf.okapi.steps.wordcount.common

Class TokenCountStep

java.lang.Object

net.sf.okapi.common.pipeline.BasePipelineStep

net.sf.okapi.steps.wordcount.common.BaseCountStep

net.sf.okapi.steps.wordcount.common.TokenCountStep

All Implemented Interfaces:

IPipelineStep

Direct Known Subclasses:

GMXAlphanumericOnlyTextUnitWordCountStep, GMXMeasurementOnlyTextUnitWordCountStep, GMXNumericOnlyTextUnitWordCountStep, TokenCharacterCountStep, WordCountStep

public abstract class TokenCountStep
extends BaseCountStep

Constructor Summary

Constructors

Constructor and Description
TokenCountStep()

Method Summary

Modifier and Type	Method and Description
protected long	count(Segment segment, LocaleId locale)
protected long	count(TextContainer textContainer, LocaleId locale)
protected long	countInTextUnit(ITextUnit textUnit)
protected Tokens	filterTokens(Tokens allTokens)
protected abstract String[]	getTokenNames()
protected Tokens	getTokens(Segment segment, LocaleId locale)
protected Tokens	getTokens(TextContainer textContainer, LocaleId locale)

Methods inherited from class net.sf.okapi.steps.wordcount.common.BaseCountStep

countOnlyTranslatable, getBatchCount, getBatchItemCount, getDescription, getDocumentCount, getGroupCount, getMetric, getName, getSource, getSubDocumentCount, handleEndBatch, handleEndBatchItem, handleEndDocument, handleEndGroup, handleEndSubDocument, handleStartBatch, handleStartBatchItem, handleStartDocument, handleStartGroup, handleStartSubDocument, handleTextUnit, removeFromMetrics, removeFromMetrics, removeFromMetrics, saveCount, saveToMetrics, saveToMetrics, saveToMetrics

Methods inherited from class net.sf.okapi.common.pipeline.BasePipelineStep

cancel, destroy, getHelpLocation, getParameters, getSourceLocale, getTargetLocale, handleCustom, handleDocumentPart, handleEndSubfilter, handleEvent, handleMultiEvent, handlePipelineParameters, handleRawDocument, handleStartSubfilter, isDone, isLastOutputStep, setLastOutputStep, setParameters, setSourceLocale, setTargetLocale

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TokenCountStep

public TokenCountStep()

Method Detail

getTokenNames

protected abstract String[] getTokenNames()

filterTokens

protected Tokens filterTokens(Tokens allTokens)

getTokens

protected Tokens getTokens(TextContainer textContainer,
                           LocaleId locale)

count

protected long count(TextContainer textContainer,
                     LocaleId locale)

Specified by:

count in class BaseCountStep

getTokens

protected Tokens getTokens(Segment segment,
                           LocaleId locale)

count

protected long count(Segment segment,
                     LocaleId locale)

Specified by:

count in class BaseCountStep

countInTextUnit

protected long countInTextUnit(ITextUnit textUnit)

Specified by:

countInTextUnit in class BaseCountStep