org.apache.flink.streaming.api.functions.source

Interface SourceFunction<T>

Type Parameters:

T - The type of the elements produced by this source.

All Superinterfaces:

org.apache.flink.api.common.functions.Function, Serializable

All Known Subinterfaces:

ParallelSourceFunction<OUT>

All Known Implementing Classes:

ConnectorSource, FileMonitoringFunction, FileSourceFunction, FromElementsFunction, FromIteratorFunction, FromSplittableIteratorFunction, RichParallelSourceFunction, RichSourceFunction, SocketTextStreamFunction

public interface SourceFunction<T>
extends org.apache.flink.api.common.functions.Function, Serializable

Base interface for all stream data sources in Flink. The contract of a stream source is the following: When the source should start emitting elements the run(org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext<T>) method is called with a Collector that can be used for emitting elements. The run method can run for as long as necessary. The source must, however, react to an invocation of cancel() by breaking out of its main loop. Note about checkpointed sources

Sources that also implement the Checkpointed interface must ensure that state checkpointing, updating of internal state and emission of elements are not done concurrently. This is achieved by using the provided checkpointing lock object to protect update of state and emission of elements in a synchronized block.

This is the basic pattern one should follow when implementing a (checkpointed) source:

 {@code
  public class ExampleSource implements SourceFunction, Checkpointed {
      private long count = 0L;
      private volatile boolean isRunning;

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static interface	SourceFunction.SourceContext<T> Interface that source functions use to communicate with the outside world.

Method Summary

Modifier and Type	Method and Description
void	cancel() Cancels the source.
void	run(SourceFunction.SourceContext<T> ctx) Starts the source.

Method Detail

run

void run(SourceFunction.SourceContext<T> ctx)
  throws Exception

Starts the source. You can use the Collector parameter to emit elements. Sources that implement Checkpointed must lock on the checkpoint lock (using a synchronized block) before updating internal state and/or emitting elements. Also, the update of state and emission of elements must happen in the same synchronized block.

Parameters:

ctx - The context for interaction with the outside world.

Throws:

Exception

cancel

void cancel()

Cancels the source. Most sources will have a while loop inside the run(org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext<T>) method. You need to ensure that the source will break out of this loop. This can be achieved by having a volatile field "isRunning" that is checked in the loop and that is set to false in this method.