org.apache.flink.storm.wrappers

Class SpoutWrapper<OUT>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.streaming.api.functions.source.RichParallelSourceFunction<OUT>

org.apache.flink.storm.wrappers.SpoutWrapper<OUT>

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction, org.apache.flink.api.common.functions.StoppableFunction, ParallelSourceFunction<OUT>, SourceFunction<OUT>

public final class SpoutWrapper<OUT>
extends RichParallelSourceFunction<OUT>
implements org.apache.flink.api.common.functions.StoppableFunction

A SpoutWrapper wraps an IRichSpout in order to execute it within a Flink Streaming program. It takes the spout's output tuples and transforms them into Flink tuples of type OUT (see SpoutCollector for supported types).

Per default, SpoutWrapper calls the wrapped spout's nextTuple() method in an infinite loop.
Alternatively, SpoutWrapper can call nextTuple() for a finite number of times and terminate automatically afterwards (for finite input streams). The number of nextTuple() calls can be specified as a certain number of invocations or can be undefined. In the undefined case, SpoutWrapper terminates if no record was emitted to the output collector for the first time during a call to nextTuple().
If the given spout implements FiniteSpout interface and numberOfInvocations is not provided or is null, SpoutWrapper calls nextTuple() method until FiniteSpout.reachedEnd() returns true.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.streaming.api.functions.source.SourceFunction

SourceFunction.SourceContext<T>

Constructor Summary

Constructors

Constructor and Description
SpoutWrapper(backtype.storm.topology.IRichSpout spout) Instantiates a new SpoutWrapper that calls the nextTuple() method of the given spout in an infinite loop.
SpoutWrapper(backtype.storm.topology.IRichSpout spout, Collection<String> rawOutputs) Instantiates a new SpoutWrapper that calls the nextTuple() method of the given spout in an infinite loop.
SpoutWrapper(backtype.storm.topology.IRichSpout spout, Collection<String> rawOutputs, Integer numberOfInvocations) Instantiates a new SpoutWrapper that calls the nextTuple() method of the given spout a finite number of times.
SpoutWrapper(backtype.storm.topology.IRichSpout spout, Integer numberOfInvocations) Instantiates a new SpoutWrapper that calls the nextTuple() method of the given spout a finite number of times.
SpoutWrapper(backtype.storm.topology.IRichSpout spout, String[] rawOutputs) Instantiates a new SpoutWrapper that calls the nextTuple() method of the given spout in an infinite loop.
SpoutWrapper(backtype.storm.topology.IRichSpout spout, String[] rawOutputs, Integer numberOfInvocations) Instantiates a new SpoutWrapper that calls the nextTuple() method of the given spout a finite number of times.
SpoutWrapper(backtype.storm.topology.IRichSpout spout, String name, Collection<String> rawOutputs, Integer numberOfInvocations) Instantiates a new SpoutWrapper that calls the nextTuple() method of the given spout a finite number of times.

Method Summary

Methods

Modifier and Type	Method and Description
void	cancel()
void	close()
void	run(SourceFunction.SourceContext<OUT> ctx)
void	setStormTopology(backtype.storm.generated.StormTopology stormTopology) Sets the original Storm topology.
void	stop()

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

getIterationRuntimeContext, getRuntimeContext, open, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SpoutWrapper

public SpoutWrapper(backtype.storm.topology.IRichSpout spout)
             throws IllegalArgumentException

Instantiates a new SpoutWrapper that calls the nextTuple() method of the given spout in an infinite loop. The output type will be one of Tuple0 to Tuple25 depending on the spout's declared number of attributes.

Parameters:

spout - The spout to be used.

Throws:

IllegalArgumentException - If the number of declared output attributes is not with range [0;25].

SpoutWrapper

public SpoutWrapper(backtype.storm.topology.IRichSpout spout,
            Integer numberOfInvocations)
             throws IllegalArgumentException

Instantiates a new SpoutWrapper that calls the nextTuple() method of the given spout a finite number of times. The output type will be one of Tuple0 to Tuple25 depending on the spout's declared number of attributes.

Parameters:

spout - The spout to be used.

numberOfInvocations - The number of calls to ISpout.nextTuple(). If value is negative, SpoutWrapper terminates if no tuple was emitted for the first time. If value is null, finite invocation is disabled.

Throws:

IllegalArgumentException - If the number of declared output attributes is not with range [0;25].

SpoutWrapper

public SpoutWrapper(backtype.storm.topology.IRichSpout spout,
            String[] rawOutputs)
             throws IllegalArgumentException

Instantiates a new SpoutWrapper that calls the nextTuple() method of the given spout in an infinite loop. The output type can be any type if parameter rawOutput is true and the spout's number of declared output tuples is 1. If rawOutput is false the output type will be one of Tuple0 to Tuple25 depending on the spout's declared number of attributes.

Parameters:

spout - The spout to be used.

rawOutputs - Contains stream names if a single attribute output stream, should not be of type Tuple1 but be of a raw type. (Can be null.)

Throws:

IllegalArgumentException - If rawOuput is true and the number of declared output attributes is not 1 or if rawOuput is false and the number of declared output attributes is not with range [0;25].

SpoutWrapper

public SpoutWrapper(backtype.storm.topology.IRichSpout spout,
            String[] rawOutputs,
            Integer numberOfInvocations)
             throws IllegalArgumentException

Instantiates a new SpoutWrapper that calls the nextTuple() method of the given spout a finite number of times. The output type can be any type if parameter rawOutput is true and the spout's number of declared output tuples is 1. If rawOutput is false the output type will be one of Tuple0 to Tuple25 depending on the spout's declared number of attributes.

Parameters:

spout - The spout to be used.

rawOutputs - Contains stream names if a single attribute output stream, should not be of type Tuple1 but be of a raw type. (Can be null.)

numberOfInvocations - The number of calls to ISpout.nextTuple(). If value is negative, SpoutWrapper terminates if no tuple was emitted for the first time. If value is null, finite invocation is disabled.

Throws:

IllegalArgumentException - If rawOuput is true and the number of declared output attributes is not 1 or if rawOuput is false and the number of declared output attributes is not with range [0;25].

SpoutWrapper

public SpoutWrapper(backtype.storm.topology.IRichSpout spout,
            Collection<String> rawOutputs)
             throws IllegalArgumentException

Instantiates a new SpoutWrapper that calls the nextTuple() method of the given spout in an infinite loop. The output type can be any type if parameter rawOutput is true and the spout's number of declared output tuples is 1. If rawOutput is false the output type will be one of Tuple0 to Tuple25 depending on the spout's declared number of attributes.

Parameters:

spout - The spout to be used.

rawOutputs - Contains stream names if a single attribute output stream, should not be of type Tuple1 but be of a raw type. (Can be null.)

Throws:

IllegalArgumentException - If rawOuput is true and the number of declared output attributes is not 1 or if rawOuput is false and the number of declared output attributes is not with range [0;25].

SpoutWrapper

public SpoutWrapper(backtype.storm.topology.IRichSpout spout,
            Collection<String> rawOutputs,
            Integer numberOfInvocations)
             throws IllegalArgumentException

Instantiates a new SpoutWrapper that calls the nextTuple() method of the given spout a finite number of times. The output type can be any type if parameter rawOutput is true and the spout's number of declared output tuples is 1. If rawOutput is false the output type will be one of Tuple0 to Tuple25 depending on the spout's declared number of attributes.

Parameters:

spout - The spout to be used.

rawOutputs - Contains stream names if a single attribute output stream, should not be of type Tuple1 but be of a raw type. (Can be null.)

numberOfInvocations - The number of calls to ISpout.nextTuple(). If value is negative, SpoutWrapper terminates if no tuple was emitted for the first time. If value is null, finite invocation is disabled.

Throws:

IllegalArgumentException - If rawOuput is true and the number of declared output attributes is not 1 or if rawOuput is false and the number of declared output attributes is not with range [0;25].

SpoutWrapper

public SpoutWrapper(backtype.storm.topology.IRichSpout spout,
            String name,
            Collection<String> rawOutputs,
            Integer numberOfInvocations)
             throws IllegalArgumentException

Instantiates a new SpoutWrapper that calls the nextTuple() method of the given spout a finite number of times. The output type can be any type if parameter rawOutput is true and the spout's number of declared output tuples is 1. If rawOutput is false the output type will be one of Tuple0 to Tuple25 depending on the spout's declared number of attributes.

Parameters:

spout - The spout to be used.

name - The name of the spout.

rawOutputs - Contains stream names if a single attribute output stream, should not be of type Tuple1 but be of a raw type. (Can be null.)

numberOfInvocations - The number of calls to ISpout.nextTuple(). If value is negative, SpoutWrapper terminates if no tuple was emitted for the first time. If value is null, finite invocation is disabled.

Throws:

IllegalArgumentException - If rawOuput is true and the number of declared output attributes is not 1 or if rawOuput is false and the number of declared output attributes is not with range [0;25].

Method Detail

setStormTopology

public void setStormTopology(backtype.storm.generated.StormTopology stormTopology)

Sets the original Storm topology.

Parameters:

stormTopology - The original Storm topology.

run

public final void run(SourceFunction.SourceContext<OUT> ctx)
               throws Exception

Specified by:

run in interface SourceFunction<OUT>

Throws:

Exception

cancel

public void cancel()

Sets the isRunning flag to false.

Specified by:

cancel in interface SourceFunction<OUT>

stop

public void stop()

Sets the isRunning flag to false.

Specified by:

stop in interface org.apache.flink.api.common.functions.StoppableFunction

close

public void close()
           throws Exception

Specified by:

close in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

close in class org.apache.flink.api.common.functions.AbstractRichFunction

Throws:

Exception