Package org.apache.camel.component.flink

Class FlinkEndpoint

java.lang.Object
org.apache.camel.support.service.BaseService
org.apache.camel.support.service.ServiceSupport
org.apache.camel.support.DefaultEndpoint
org.apache.camel.component.flink.FlinkEndpoint
All Implemented Interfaces:
AutoCloseable, org.apache.camel.CamelContextAware, org.apache.camel.ComponentAware, org.apache.camel.Endpoint, org.apache.camel.IsSingleton, org.apache.camel.Service, org.apache.camel.ShutdownableService, org.apache.camel.spi.HasCamelContext, org.apache.camel.spi.HasId, org.apache.camel.StatefulService, org.apache.camel.SuspendableService
@UriEndpoint(firstVersion="2.18.0", scheme="flink", title="Flink", syntax="flink:endpointType", producerOnly=true, category={TRANSFORMATION,BIGDATA}, headersClass=FlinkConstants.class) public class FlinkEndpoint extends org.apache.camel.support.DefaultEndpoint
Send DataSet jobs to an Apache Flink cluster.

  • Field Summary

    Fields inherited from class org.apache.camel.support.service.BaseService

    BUILT, FAILED, INITIALIZED, INITIALIZING, lock, NEW, SHUTDOWN, SHUTTING_DOWN, STARTED, STARTING, status, STOPPED, STOPPING, SUSPENDED, SUSPENDING

  • Constructor Summary

    Constructors
    Constructor
    Description
    FlinkEndpoint(String endpointUri, FlinkComponent component, EndpointType endpointType)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.camel.Consumer
    createConsumer(org.apache.camel.Processor processor)
     
    org.apache.camel.Producer
    createProducer()
     
    protected void
    doInit()
     
    String
    getCheckpointingMode()
     
    Long
    getCheckpointInterval()
     
    Long
    getCheckpointTimeout()
     
    FlinkComponent
    getComponent()
     
    org.apache.flink.api.java.DataSet
    getDataSet()
     
    DataSetCallback
    getDataSetCallback()
     
    org.apache.flink.streaming.api.datastream.DataStream
    getDataStream()
     
    DataStreamCallback
    getDataStreamCallback()
     
    String
    getExecutionMode()
     
    String
    getJobName()
     
    Integer
    getMaxParallelism()
     
    Long
    getMinPauseBetweenCheckpoints()
     
    Integer
    getParallelism()
     
    boolean
    isCollect()
     
    void
    setCheckpointingMode(String checkpointingMode)
    Checkpointing mode: EXACTLY_ONCE (default) or AT_LEAST_ONCE.
    void
    setCheckpointInterval(Long checkpointInterval)
    Interval in milliseconds between checkpoints.
    void
    setCheckpointTimeout(Long checkpointTimeout)
    Timeout in milliseconds for checkpoints.
    void
    setCollect(boolean collect)
    Indicates if results should be collected or counted.
    void
    setDataSet(org.apache.flink.api.java.DataSet ds)
    Deprecated.
    The DataSet API is deprecated since Flink 1.12.
    void
    setDataSetCallback(DataSetCallback dataSetCallback)
    Deprecated.
    The DataSet API is deprecated since Flink 1.12.
    void
    setDataStream(org.apache.flink.streaming.api.datastream.DataStream ds)
    DataStream to compute against.
    void
    setDataStreamCallback(DataStreamCallback dataStreamCallback)
    Function performing action against a DataStream.
    void
    setEndpointType(EndpointType endpointType)
    Type of the endpoint (dataset, datastream).
    void
    setExecutionMode(String executionMode)
    Execution mode for the Flink job.
    void
    setJobName(String jobName)
    Name for the Flink job.
    void
    setMaxParallelism(Integer maxParallelism)
    Maximum parallelism for the Flink job.
    void
    setMinPauseBetweenCheckpoints(Long minPauseBetweenCheckpoints)
    Minimum pause in milliseconds between consecutive checkpoints.
    void
    setParallelism(Integer parallelism)
    Parallelism for the Flink job.

    Methods inherited from class org.apache.camel.support.DefaultEndpoint

    configureConsumer, configureExchange, configurePollingConsumer, configureProperties, createAsyncProducer, createEndpointUri, createExchange, createExchange, createPollingConsumer, equals, getCamelContext, getEndpointKey, getEndpointUri, getExceptionHandler, getExchangePattern, getId, getPollingConsumerBlockTimeout, getPollingConsumerQueueSize, hashCode, isAutowiredEnabled, isBridgeErrorHandler, isLazyStartProducer, isLenientProperties, isPollingConsumerBlockWhenFull, isPollingConsumerCopy, isSingleton, setAutowiredEnabled, setBridgeErrorHandler, setCamelContext, setComponent, setEndpointUri, setEndpointUriIfNotSpecified, setExceptionHandler, setExchangePattern, setLazyStartProducer, setPollingConsumerBlockTimeout, setPollingConsumerBlockWhenFull, setPollingConsumerCopy, setPollingConsumerQueueSize, setProperties, toString

    Methods inherited from class org.apache.camel.support.service.BaseService

    build, doBuild, doFail, doLifecycleChange, doResume, doShutdown, doStart, doStop, doSuspend, fail, getInternalLock, getStatus, init, isBuild, isInit, isNew, isRunAllowed, isShutdown, isStarted, isStarting, isStartingOrStarted, isStopped, isStopping, isStoppingOrStopped, isSuspended, isSuspending, isSuspendingOrSuspended, resume, shutdown, start, stop, suspend

    Methods inherited from class java.lang.Object

    clone, finalize, getClass, notify, notifyAll, wait, wait, wait

    Methods inherited from interface org.apache.camel.ComponentAware

    getComponent

    Methods inherited from interface org.apache.camel.Endpoint

    getEndpointBaseUri, isRemote, isSingletonProducer

    Methods inherited from interface org.apache.camel.Service

    build, close, init, start, stop

    Methods inherited from interface org.apache.camel.ShutdownableService

    shutdown

    Methods inherited from interface org.apache.camel.StatefulService

    getStatus, isRunAllowed, isStarted, isStarting, isStopped, isStopping, isSuspending

    Methods inherited from interface org.apache.camel.SuspendableService

    isSuspended, resume, suspend

  • Constructor Details

  • Method Details

    • doInit

      protected void doInit() throws Exception
      Overrides:
      doInit in class org.apache.camel.support.DefaultEndpoint
      Throws:
      Exception

    • createProducer

      public org.apache.camel.Producer createProducer() throws Exception
      Throws:
      Exception

    • createConsumer

      public org.apache.camel.Consumer createConsumer(org.apache.camel.Processor processor) throws Exception
      Throws:
      Exception

    • getComponent

      public FlinkComponent getComponent()
      Specified by:
      getComponent in interface org.apache.camel.ComponentAware
      Overrides:
      getComponent in class org.apache.camel.support.DefaultEndpoint

    • setEndpointType

      public void setEndpointType(EndpointType endpointType)
      Type of the endpoint (dataset, datastream).

    • getDataSet

      public org.apache.flink.api.java.DataSet getDataSet()

    • getDataStream

      public org.apache.flink.streaming.api.datastream.DataStream getDataStream()

    • setDataSet

      @Deprecated(since="4.16.0") public void setDataSet(org.apache.flink.api.java.DataSet ds)
      Deprecated.
      The DataSet API is deprecated since Flink 1.12. Use the DataStream API with bounded streams instead.
      DataSet to compute against.

    • setDataStream

      public void setDataStream(org.apache.flink.streaming.api.datastream.DataStream ds)
      DataStream to compute against.

    • getDataSetCallback

      public DataSetCallback getDataSetCallback()

    • getDataStreamCallback

      public DataStreamCallback getDataStreamCallback()

    • setDataSetCallback

      @Deprecated(since="4.16.0") public void setDataSetCallback(DataSetCallback dataSetCallback)
      Deprecated.
      The DataSet API is deprecated since Flink 1.12. Use the DataStream API with bounded streams instead.
      Function performing action against a DataSet.

    • setDataStreamCallback

      public void setDataStreamCallback(DataStreamCallback dataStreamCallback)
      Function performing action against a DataStream.

    • isCollect

      public boolean isCollect()

    • setCollect

      public void setCollect(boolean collect)
      Indicates if results should be collected or counted.

    • getExecutionMode

      public String getExecutionMode()

    • setExecutionMode

      public void setExecutionMode(String executionMode)
      Execution mode for the Flink job. Options: STREAMING (default), BATCH, AUTOMATIC. BATCH mode is recommended for bounded streams (batch processing).

    • getCheckpointInterval

      public Long getCheckpointInterval()

    • setCheckpointInterval

      public void setCheckpointInterval(Long checkpointInterval)
      Interval in milliseconds between checkpoints. Enables checkpointing when set. Recommended for streaming jobs to ensure fault tolerance.

    • getCheckpointingMode

      public String getCheckpointingMode()

    • setCheckpointingMode

      public void setCheckpointingMode(String checkpointingMode)
      Checkpointing mode: EXACTLY_ONCE (default) or AT_LEAST_ONCE. EXACTLY_ONCE provides stronger guarantees but may have higher overhead.

    • getParallelism

      public Integer getParallelism()

    • setParallelism

      public void setParallelism(Integer parallelism)
      Parallelism for the Flink job. If not set, uses the default parallelism of the execution environment.

    • getMaxParallelism

      public Integer getMaxParallelism()

    • setMaxParallelism

      public void setMaxParallelism(Integer maxParallelism)
      Maximum parallelism for the Flink job. Defines the upper bound for dynamic scaling and the number of key groups for stateful operators.

    • getJobName

      public String getJobName()

    • setJobName

      public void setJobName(String jobName)
      Name for the Flink job. Useful for identification in the Flink UI and logs.

    • getCheckpointTimeout

      public Long getCheckpointTimeout()

    • setCheckpointTimeout

      public void setCheckpointTimeout(Long checkpointTimeout)
      Timeout in milliseconds for checkpoints. Checkpoints that take longer will be aborted.

    • getMinPauseBetweenCheckpoints

      public Long getMinPauseBetweenCheckpoints()

    • setMinPauseBetweenCheckpoints

      public void setMinPauseBetweenCheckpoints(Long minPauseBetweenCheckpoints)
      Minimum pause in milliseconds between consecutive checkpoints. Helps prevent checkpoint storms under heavy load.