A C D E F G H I K L M N O P R S T V W
All Classes All Packages

A

ABBREVIATION - net.sf.okapi.steps.tokenization.ITokenizer.TokenType

add(int, TokenConfigs.TokenConfig) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

add(TokenConfigs.TokenConfig) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

addAll(int, Collection<? extends TokenConfigs.TokenConfig>) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

addAll(Collection<? extends TokenConfigs.TokenConfig>) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

addTokens(Tokens) - Method in class net.sf.okapi.steps.tokenization.TokensAnnotation

apostrophe(Token, LocaleId) - Method in class net.sf.okapi.steps.tokenization.TokenizationStep

Break French and Italian words with apostrophe into three tokens WORD, PUNCTUATION, WORD

C

clear() - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

contains(Object) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

containsAll(Collection<?>) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

createEditorDescription(ParametersDescription) - Method in class net.sf.okapi.steps.tokenization.Parameters

CURRENCY - net.sf.okapi.steps.tokenization.ITokenizer.TokenType

D

DATE - net.sf.okapi.steps.tokenization.ITokenizer.TokenType

E

EMAIL - net.sf.okapi.steps.tokenization.ITokenizer.TokenType

EMOJI - net.sf.okapi.steps.tokenization.ITokenizer.TokenType

EMOTICON - net.sf.okapi.steps.tokenization.ITokenizer.TokenType

F

fixRanges(List<Integer>) - Method in class net.sf.okapi.steps.tokenization.Tokens

G

get(int) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

getDescription() - Method in enum net.sf.okapi.steps.tokenization.ITokenizer.TokenType

getDescription() - Method in class net.sf.okapi.steps.tokenization.Token

getDescription() - Method in class net.sf.okapi.steps.tokenization.TokenConfigs.TokenConfig

getDescription() - Method in class net.sf.okapi.steps.tokenization.TokenizationStep

getExcludedFilteredList(String...) - Method in class net.sf.okapi.steps.tokenization.Tokens

Return a list of TokenType objects.

getFilteredList(String...) - Method in class net.sf.okapi.steps.tokenization.Tokens

Return a list of TokenType objects.

getFilteredList(String...) - Method in class net.sf.okapi.steps.tokenization.TokensAnnotation

getFilteredLocales() - Method in class net.sf.okapi.steps.tokenization.Parameters

getId() - Method in class net.sf.okapi.steps.tokenization.Token

getId() - Method in class net.sf.okapi.steps.tokenization.TokenConfigs.TokenConfig

getIncludedTokenNames() - Method in class net.sf.okapi.steps.tokenization.Parameters

TokenType that will be filtered from the results of tokenization

getName() - Method in class net.sf.okapi.steps.tokenization.Token

getName() - Method in class net.sf.okapi.steps.tokenization.TokenConfigs.TokenConfig

getName() - Method in class net.sf.okapi.steps.tokenization.TokenizationStep

getParametersDescription() - Method in class net.sf.okapi.steps.tokenization.Parameters

getRange() - Method in class net.sf.okapi.steps.tokenization.Token

getRule(int) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

Gets a rule by its tokenId

getRules() - Static method in class net.sf.okapi.steps.tokenization.Tokens

getSample() - Method in class net.sf.okapi.steps.tokenization.TokenConfigs.TokenConfig

getSourceLocale() - Method in class net.sf.okapi.steps.tokenization.TokenizationStep

getTargetLocale() - Method in class net.sf.okapi.steps.tokenization.TokenizationStep

getTokenDescription(int) - Static method in class net.sf.okapi.steps.tokenization.Tokens

getTokenDescription(String) - Static method in class net.sf.okapi.steps.tokenization.Tokens

getTokenId(String) - Static method in class net.sf.okapi.steps.tokenization.Tokens

getTokenIDs() - Static method in class net.sf.okapi.steps.tokenization.Tokens

Gets list of IDs of all tokens.

getTokenIDs(List<String>) - Static method in class net.sf.okapi.steps.tokenization.Tokens

Gets list of IDs of the given tokens.

getTokenIndex(String) - Static method in class net.sf.okapi.steps.tokenization.Tokens

getTokenName(int) - Static method in class net.sf.okapi.steps.tokenization.Tokens

getTokenNames() - Static method in class net.sf.okapi.steps.tokenization.Tokens

Gets list of names of all tokens.

getTokenNamesStr() - Static method in class net.sf.okapi.steps.tokenization.Tokens

getTokens() - Method in class net.sf.okapi.steps.tokenization.TokensAnnotation

getValue() - Method in class net.sf.okapi.steps.tokenization.Token

H

handleStartDocument(Event) - Method in class net.sf.okapi.steps.tokenization.TokenizationStep

handleTextUnit(Event) - Method in class net.sf.okapi.steps.tokenization.TokenizationStep

hasNext() - Method in interface net.sf.okapi.steps.tokenization.ITokenizer

hasNext() - Method in class net.sf.okapi.steps.tokenization.RbbiTokenizer

HYPHENATED_WORD - net.sf.okapi.steps.tokenization.ITokenizer.TokenType

I

id - Variable in class net.sf.okapi.steps.tokenization.Token

IDEOGRAM - net.sf.okapi.steps.tokenization.ITokenizer.TokenType

indexOf(Object) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

init(String, LocaleId) - Method in interface net.sf.okapi.steps.tokenization.ITokenizer

init(String, LocaleId) - Method in class net.sf.okapi.steps.tokenization.RbbiTokenizer

INTERNET - net.sf.okapi.steps.tokenization.ITokenizer.TokenType

isABBREVIATION() - Method in class net.sf.okapi.steps.tokenization.Parameters

isCURRENCY() - Method in class net.sf.okapi.steps.tokenization.Parameters

isDATE() - Method in class net.sf.okapi.steps.tokenization.Parameters

isEMAIL() - Method in class net.sf.okapi.steps.tokenization.Parameters

isEMOJI() - Method in class net.sf.okapi.steps.tokenization.Parameters

isEMOTICON() - Method in class net.sf.okapi.steps.tokenization.Parameters

isEmpty() - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

isHYPHENATED_WORD() - Method in class net.sf.okapi.steps.tokenization.Parameters

isIDEOGRAM() - Method in class net.sf.okapi.steps.tokenization.Parameters

isINTERNET() - Method in class net.sf.okapi.steps.tokenization.Parameters

isKANA() - Method in class net.sf.okapi.steps.tokenization.Parameters

isMARKUP() - Method in class net.sf.okapi.steps.tokenization.Parameters

isNUMBER() - Method in class net.sf.okapi.steps.tokenization.Parameters

isOTHER_SYMBOL() - Method in class net.sf.okapi.steps.tokenization.Parameters

isPUNCTUATION() - Method in class net.sf.okapi.steps.tokenization.Parameters

isTIME() - Method in class net.sf.okapi.steps.tokenization.Parameters

isTokenizeSource() - Method in class net.sf.okapi.steps.tokenization.Parameters

isTokenizeTargets() - Method in class net.sf.okapi.steps.tokenization.Parameters

isWHITESPACE() - Method in class net.sf.okapi.steps.tokenization.Parameters

isWORD() - Method in class net.sf.okapi.steps.tokenization.Parameters

iterator() - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

ITokenizer - Interface in net.sf.okapi.steps.tokenization

ITokenizer.TokenType - Enum in net.sf.okapi.steps.tokenization

K

KANA - net.sf.okapi.steps.tokenization.ITokenizer.TokenType

L

lastIndexOf(Object) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

listIterator() - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

listIterator(int) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

load(ParametersString) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

load(ParametersString) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs.TokenConfig

M

MARKUP - net.sf.okapi.steps.tokenization.ITokenizer.TokenType

N

net.sf.okapi.steps.tokenization - package net.sf.okapi.steps.tokenization

next() - Method in interface net.sf.okapi.steps.tokenization.ITokenizer

next() - Method in class net.sf.okapi.steps.tokenization.RbbiTokenizer

NUMBER - net.sf.okapi.steps.tokenization.ITokenizer.TokenType

O

OTHER_SYMBOL - net.sf.okapi.steps.tokenization.ITokenizer.TokenType

P

Parameters - Class in net.sf.okapi.steps.tokenization

Tokenization step parameters

Parameters() - Constructor for class net.sf.okapi.steps.tokenization.Parameters

postProcess(Token, LocaleId) - Method in class net.sf.okapi.steps.tokenization.TokenizationStep

Various rules to make corrections to RbbiTokenizer

PUNCTUATION - net.sf.okapi.steps.tokenization.ITokenizer.TokenType

R

RbbiTokenizer - Class in net.sf.okapi.steps.tokenization

RbbiTokenizer() - Constructor for class net.sf.okapi.steps.tokenization.RbbiTokenizer

remove(int) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

remove(Object) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

removeAll(Collection<?>) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

reset() - Method in class net.sf.okapi.steps.tokenization.Parameters

reset() - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

reset() - Method in class net.sf.okapi.steps.tokenization.TokenConfigs.TokenConfig

retainAll(Collection<?>) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

S

save(ParametersString) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

save(ParametersString) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs.TokenConfig

set(int, TokenConfigs.TokenConfig) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

setABBREVIATION(boolean) - Method in class net.sf.okapi.steps.tokenization.Parameters

setCURRENCY(boolean) - Method in class net.sf.okapi.steps.tokenization.Parameters

setDATE(boolean) - Method in class net.sf.okapi.steps.tokenization.Parameters

setDescription(String) - Method in enum net.sf.okapi.steps.tokenization.ITokenizer.TokenType

setDescription(String) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs.TokenConfig

setEMAIL(boolean) - Method in class net.sf.okapi.steps.tokenization.Parameters

setEMOJI(boolean) - Method in class net.sf.okapi.steps.tokenization.Parameters

setEMOTICON(boolean) - Method in class net.sf.okapi.steps.tokenization.Parameters

setFilteredLocales(String) - Method in class net.sf.okapi.steps.tokenization.Parameters

setHYPHENATED_WORD(boolean) - Method in class net.sf.okapi.steps.tokenization.Parameters

setId(int) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs.TokenConfig

setIDEOGRAM(boolean) - Method in class net.sf.okapi.steps.tokenization.Parameters

setIncludedTokenNames(String...) - Method in class net.sf.okapi.steps.tokenization.Parameters

setINTERNET(boolean) - Method in class net.sf.okapi.steps.tokenization.Parameters

setKANA(boolean) - Method in class net.sf.okapi.steps.tokenization.Parameters

setMARKUP(boolean) - Method in class net.sf.okapi.steps.tokenization.Parameters

setName(String) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs.TokenConfig

setNUMBER(boolean) - Method in class net.sf.okapi.steps.tokenization.Parameters

setOTHER_SYMBOL(boolean) - Method in class net.sf.okapi.steps.tokenization.Parameters

setPUNCTUATION(boolean) - Method in class net.sf.okapi.steps.tokenization.Parameters

setSample(String) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs.TokenConfig

setSourceLocale(LocaleId) - Method in class net.sf.okapi.steps.tokenization.TokenizationStep

setTargetLocale(LocaleId) - Method in class net.sf.okapi.steps.tokenization.TokenizationStep

setTIME(boolean) - Method in class net.sf.okapi.steps.tokenization.Parameters

setTokenizeSource(boolean) - Method in class net.sf.okapi.steps.tokenization.Parameters

setTokenizeTargets(boolean) - Method in class net.sf.okapi.steps.tokenization.Parameters

setTokens(Tokens) - Method in class net.sf.okapi.steps.tokenization.TokensAnnotation

setWHITESPACE(boolean) - Method in class net.sf.okapi.steps.tokenization.Parameters

setWORD(boolean) - Method in class net.sf.okapi.steps.tokenization.Parameters

size() - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

subList(int, int) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

T

TIME - net.sf.okapi.steps.tokenization.ITokenizer.TokenType

toArray() - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

toArray(T[]) - Method in class net.sf.okapi.steps.tokenization.TokenConfigs

Token - Class in net.sf.okapi.steps.tokenization

TokenType generated by the RbbiTokenizer

Token(int, String, String, String, int, int) - Constructor for class net.sf.okapi.steps.tokenization.Token

TokenConfig() - Constructor for class net.sf.okapi.steps.tokenization.TokenConfigs.TokenConfig

TokenConfigs - Class in net.sf.okapi.steps.tokenization

TokenType Definitions

TokenConfigs() - Constructor for class net.sf.okapi.steps.tokenization.TokenConfigs

TokenConfigs.TokenConfig - Class in net.sf.okapi.steps.tokenization

TokenizationStep - Class in net.sf.okapi.steps.tokenization

TokenizationStep() - Constructor for class net.sf.okapi.steps.tokenization.TokenizationStep

tokenize(String, LocaleId, String...) - Static method in class net.sf.okapi.steps.tokenization.Tokenizer

tokenize(ITextUnit, LocaleId, String...) - Static method in class net.sf.okapi.steps.tokenization.Tokenizer

tokenize(TextContainer, LocaleId, String...) - Static method in class net.sf.okapi.steps.tokenization.Tokenizer

tokenize(TextFragment, LocaleId, String...) - Static method in class net.sf.okapi.steps.tokenization.Tokenizer

Tokenizer - Class in net.sf.okapi.steps.tokenization

Tokenizer() - Constructor for class net.sf.okapi.steps.tokenization.Tokenizer

tokenizeString(String, LocaleId, String...) - Static method in class net.sf.okapi.steps.tokenization.Tokenizer

Extracts tokens from the given text.

Tokens - Class in net.sf.okapi.steps.tokenization

List of Token and configuration for each token type.

Tokens() - Constructor for class net.sf.okapi.steps.tokenization.Tokens

TokensAnnotation - Class in net.sf.okapi.steps.tokenization

Holds a list of Token

TokensAnnotation(Tokens) - Constructor for class net.sf.okapi.steps.tokenization.TokensAnnotation

toString() - Method in class net.sf.okapi.steps.tokenization.Token

toString() - Method in class net.sf.okapi.steps.tokenization.Tokens

toString() - Method in class net.sf.okapi.steps.tokenization.TokensAnnotation

ts - Static variable in class net.sf.okapi.steps.tokenization.Tokenizer

V

valueOf(String) - Static method in enum net.sf.okapi.steps.tokenization.ITokenizer.TokenType

Returns the enum constant of this type with the specified name.

values() - Static method in enum net.sf.okapi.steps.tokenization.ITokenizer.TokenType

Returns an array containing the constants of this enum type, in the order they are declared.

W

WHITESPACE - net.sf.okapi.steps.tokenization.ITokenizer.TokenType

WORD - net.sf.okapi.steps.tokenization.ITokenizer.TokenType

A C D E F G H I K L M N O P R S T V W
All Classes All Packages