net.sf.okapi.lib.segmentation

Class SRXSegmenter

java.lang.Object

net.sf.okapi.lib.segmentation.SRXSegmenter

All Implemented Interfaces:

ISegmenter

public class SRXSegmenter
extends Object
implements ISegmenter

Implements the ISegmenter interface for SRX rules.

Constructor Summary

Constructors

Constructor and Description
SRXSegmenter() Creates a new SRXSegmenter object.

Method Summary

Modifier and Type	Method and Description
protected void	addRule(net.sf.okapi.lib.segmentation.CompiledRule compiledRule) Adds a compiled rule to this segmenter.
boolean	cascade() Indicates if cascading must be applied when selecting the rules for a given language pattern.
int	computeSegments(String text)
int	computeSegments(TextContainer container)
LocaleId	getLanguage()
Range	getNextSegmentRange(TextContainer container)
List<Range>	getRanges()
List<Integer>	getSplitPositions()
boolean	includeEndCodes()
boolean	includeIsolatedCodes()
boolean	includeStartCodes()
boolean	oneSegmentIncludesAll()
void	reset()
boolean	segmentSubFlows()
protected void	setCascade(boolean value) Sets the flag indicating if cascading must be applied when selecting the rules for a given language pattern.
void	setIncludeEndCodes(boolean includeEndCodes)
void	setIncludeIsolatedCodes(boolean includeIsolatedCodes)
void	setIncludeStartCodes(boolean includeStartCodes)
void	setLanguage(LocaleId languageCode)
protected void	setMaskRule(String pattern) Sets the pattern for the mask rule.
void	setOneSegmentIncludesAll(boolean oneSegmentIncludesAll)
void	setOptions(boolean segmentSubFlows, boolean includeStartCodes, boolean includeEndCodes, boolean includeIsolatedCodes, boolean oneSegmentIncludesAll, boolean trimLeadingWS, boolean trimTrailingWS)
void	setOptions(boolean segmentSubFlows, boolean includeStartCodes, boolean includeEndCodes, boolean includeIsolatedCodes, boolean oneSegmentIncludesAll, boolean trimLeadingWS, boolean trimTrailingWS, boolean useJavaRegex, boolean useIcu4JBreakRules, boolean treatIsolatedCodesAsWhitespace) Sets the options for this segmenter.
void	setSegmentSubFlows(boolean segmentSubFlows)
void	setTreatIsolatedCodesAsWhitespace(boolean treatIsolatedCodesAsWhitespace)
void	setTrimCodes(boolean trimCodes)
void	setTrimLeadingWS(boolean trimLeadingWS)
void	setTrimTrailingWS(boolean trimTrailingWS)
void	setUseJavaRegex(boolean useJavaRegex) Sets the indicator that tells if this document has rules that are defined for the Java regular expression engine (vs ICU).
boolean	treatIsolatedCodesAsWhitespace()
boolean	trimLeadingWhitespaces()
boolean	trimTrailingWhitespaces()
boolean	useJavaRegex() Indicates if this document has rules that are defined for the Java regular expression engine (vs ICU).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SRXSegmenter

public SRXSegmenter()

Creates a new SRXSegmenter object.

Method Detail

reset

public void reset()

Specified by:

reset in interface ISegmenter

setOptions

public void setOptions(boolean segmentSubFlows,
                       boolean includeStartCodes,
                       boolean includeEndCodes,
                       boolean includeIsolatedCodes,
                       boolean oneSegmentIncludesAll,
                       boolean trimLeadingWS,
                       boolean trimTrailingWS,
                       boolean useJavaRegex,
                       boolean useIcu4JBreakRules,
                       boolean treatIsolatedCodesAsWhitespace)

Sets the options for this segmenter.

Parameters:

segmentSubFlows - true to segment sub-flows, false to no segment them.

includeStartCodes - true to include start codes just before a break in the 'left' segment, false to put them in the next segment.

includeEndCodes - true to include end codes just before a break in the 'left' segment, false to put them in the next segment.

includeIsolatedCodes - true to include isolated codes just before a break in the 'left' segment, false to put them in the next segment.

oneSegmentIncludesAll - true to include everything in segments that are alone.

trimLeadingWS - true to trim leading white-spaces from the segments, false to keep them.

trimTrailingWS - true to trim trailing white-spaces from the segments, false to keep them.

useJavaRegex - true if the rules are for the Java regular expression engine, false if they are for ICU.

treatIsolatedCodesAsWhitespace - if true then the isolated code markers in codedText get converted to spaces, so that they don't get in the way of the rules. If false, the codes are simply removed.

setOptions

public void setOptions(boolean segmentSubFlows,
                       boolean includeStartCodes,
                       boolean includeEndCodes,
                       boolean includeIsolatedCodes,
                       boolean oneSegmentIncludesAll,
                       boolean trimLeadingWS,
                       boolean trimTrailingWS)

Specified by:

setOptions in interface ISegmenter

oneSegmentIncludesAll

public boolean oneSegmentIncludesAll()

Specified by:

oneSegmentIncludesAll in interface ISegmenter

segmentSubFlows

public boolean segmentSubFlows()

Specified by:

segmentSubFlows in interface ISegmenter

cascade

public boolean cascade()

Indicates if cascading must be applied when selecting the rules for a given language pattern.

Returns:

true if cascading must be applied, false otherwise.

trimLeadingWhitespaces

public boolean trimLeadingWhitespaces()

Specified by:

trimLeadingWhitespaces in interface ISegmenter

trimTrailingWhitespaces

public boolean trimTrailingWhitespaces()

Specified by:

trimTrailingWhitespaces in interface ISegmenter

useJavaRegex

public boolean useJavaRegex()

Indicates if this document has rules that are defined for the Java regular expression engine (vs ICU).

Returns:

true if the rules are for the Java regular expression engine, false if they are for ICU.

treatIsolatedCodesAsWhitespace

public boolean treatIsolatedCodesAsWhitespace()

Specified by:

treatIsolatedCodesAsWhitespace in interface ISegmenter

setUseJavaRegex

public void setUseJavaRegex(boolean useJavaRegex)

Sets the indicator that tells if this document has rules that are defined for the Java regular expression engine (vs ICU).

Parameters:

useJavaRegex - true if the rules should be treated as Java regular expression, false for ICU.

includeStartCodes

public boolean includeStartCodes()

Specified by:

includeStartCodes in interface ISegmenter

includeEndCodes

public boolean includeEndCodes()

Specified by:

includeEndCodes in interface ISegmenter

includeIsolatedCodes

public boolean includeIsolatedCodes()

Specified by:

includeIsolatedCodes in interface ISegmenter

computeSegments

public int computeSegments(String text)

Specified by:

computeSegments in interface ISegmenter

computeSegments

public int computeSegments(TextContainer container)

Specified by:

computeSegments in interface ISegmenter

getNextSegmentRange

public Range getNextSegmentRange(TextContainer container)

Specified by:

getNextSegmentRange in interface ISegmenter

getSplitPositions

public List<Integer> getSplitPositions()

Specified by:

getSplitPositions in interface ISegmenter

getRanges

public List<Range> getRanges()

Specified by:

getRanges in interface ISegmenter

getLanguage

public LocaleId getLanguage()

Specified by:

getLanguage in interface ISegmenter

setLanguage

public void setLanguage(LocaleId languageCode)

Specified by:

setLanguage in interface ISegmenter

setCascade

protected void setCascade(boolean value)

Sets the flag indicating if cascading must be applied when selecting the rules for a given language pattern.

Parameters:

value - true if cascading must be applied, false otherwise.

addRule

protected void addRule(net.sf.okapi.lib.segmentation.CompiledRule compiledRule)

Adds a compiled rule to this segmenter.

Parameters:

compiledRule - the compiled rule to add.

setMaskRule

protected void setMaskRule(String pattern)

Sets the pattern for the mask rule.

Parameters:

pattern - the new pattern to use for the mask rule.

setSegmentSubFlows

public void setSegmentSubFlows(boolean segmentSubFlows)

Specified by:

setSegmentSubFlows in interface ISegmenter

setIncludeStartCodes

public void setIncludeStartCodes(boolean includeStartCodes)

Specified by:

setIncludeStartCodes in interface ISegmenter

setIncludeEndCodes

public void setIncludeEndCodes(boolean includeEndCodes)

Specified by:

setIncludeEndCodes in interface ISegmenter

setIncludeIsolatedCodes

public void setIncludeIsolatedCodes(boolean includeIsolatedCodes)

Specified by:

setIncludeIsolatedCodes in interface ISegmenter

setOneSegmentIncludesAll

public void setOneSegmentIncludesAll(boolean oneSegmentIncludesAll)

Specified by:

setOneSegmentIncludesAll in interface ISegmenter

setTrimLeadingWS

public void setTrimLeadingWS(boolean trimLeadingWS)

Specified by:

setTrimLeadingWS in interface ISegmenter

setTrimTrailingWS

public void setTrimTrailingWS(boolean trimTrailingWS)

Specified by:

setTrimTrailingWS in interface ISegmenter

setTrimCodes

public void setTrimCodes(boolean trimCodes)

Specified by:

setTrimCodes in interface ISegmenter

setTreatIsolatedCodesAsWhitespace

public void setTreatIsolatedCodesAsWhitespace(boolean treatIsolatedCodesAsWhitespace)

Specified by:

setTreatIsolatedCodesAsWhitespace in interface ISegmenter