Package org.apache.tika.pipes.pipesiterator.kafka

Class KafkaPipesIterator

java.lang.Object

org.apache.tika.config.ConfigBase

org.apache.tika.pipes.pipesiterator.PipesIterator

org.apache.tika.pipes.pipesiterator.kafka.KafkaPipesIterator

All Implemented Interfaces:

Iterable<org.apache.tika.pipes.FetchEmitTuple>, Callable<Integer>, org.apache.tika.config.Initializable

public class KafkaPipesIterator
extends org.apache.tika.pipes.pipesiterator.PipesIterator
implements org.apache.tika.config.Initializable

Field Summary

Fields inherited from class org.apache.tika.pipes.pipesiterator.PipesIterator

COMPLETED_SEMAPHORE, DEFAULT_MAX_WAIT_MS, DEFAULT_QUEUE_SIZE

Constructor Summary

Constructors

Constructor	Description
KafkaPipesIterator()

Method Summary

Modifier and Type	Method	Description
void	checkInitialization(org.apache.tika.config.InitializableProblemHandler problemHandler)
protected void	enqueue()
void	initialize(Map<String,org.apache.tika.config.Param> params)
void	setAutoOffsetReset(String autoOffsetReset)
void	setBootstrapServers(String bootstrapServers)
void	setEmitMax(int emitMax)	If the kafka pipe iterator will keep polling for more documents until it returns an empty result.
void	setGroupId(String groupId)
void	setGroupInitialRebalanceDelayMs(int groupInitialRebalanceDelayMs)
void	setKeySerializer(String keySerializer)
void	setPollDelayMs(int pollDelayMs)
void	setTopic(String topic)
void	setValueSerializer(String valueSerializer)

Methods inherited from class org.apache.tika.pipes.pipesiterator.PipesIterator

build, call, getEmitterName, getFetcherName, getHandlerConfig, getOnParseException, iterator, setEmitterName, setFetcherName, setHandlerType, setMaxEmbeddedResources, setMaxWaitMs, setOnParseException, setOnParseException, setParseMode, setParseMode, setQueueSize, setThrowOnWriteLimitReached, setWriteLimit, tryToAdd

Methods inherited from class org.apache.tika.config.ConfigBase

buildComposite, buildComposite, buildSingle, buildSingle, configure, handleSettings

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Constructor Detail

KafkaPipesIterator

public KafkaPipesIterator()

Method Detail

setTopic

@Field
public void setTopic(String topic)

setGroupId

@Field
public void setGroupId(String groupId)

setBootstrapServers

@Field
public void setBootstrapServers(String bootstrapServers)

setKeySerializer

@Field
public void setKeySerializer(String keySerializer)

setAutoOffsetReset

@Field
public void setAutoOffsetReset(String autoOffsetReset)

setValueSerializer

@Field
public void setValueSerializer(String valueSerializer)

setPollDelayMs

@Field
public void setPollDelayMs(int pollDelayMs)

setGroupInitialRebalanceDelayMs

@Field
public void setGroupInitialRebalanceDelayMs(int groupInitialRebalanceDelayMs)

setEmitMax

@Field
public void setEmitMax(int emitMax)

If the kafka pipe iterator will keep polling for more documents until it returns an empty result. If you set emitMax is set to > 0, it will stop polling if the number of documents you have emitted so far > emitMax.

initialize

public void initialize(Map<String,org.apache.tika.config.Param> params)

Specified by:

initialize in interface org.apache.tika.config.Initializable

Overrides:

initialize in class org.apache.tika.pipes.pipesiterator.PipesIterator

checkInitialization

public void checkInitialization(org.apache.tika.config.InitializableProblemHandler problemHandler)
                         throws org.apache.tika.exception.TikaConfigException

Specified by:

checkInitialization in interface org.apache.tika.config.Initializable

Overrides:

checkInitialization in class org.apache.tika.pipes.pipesiterator.PipesIterator

Throws:

org.apache.tika.exception.TikaConfigException

enqueue

protected void enqueue()
                throws InterruptedException,
                       TimeoutException

Specified by:

enqueue in class org.apache.tika.pipes.pipesiterator.PipesIterator

Throws:

InterruptedException

TimeoutException