kafka

Class KafkaSample

  • java.lang.Object
    • kafka.KafkaSample


  • public class KafkaSample
    extends java.lang.Object
    Demonstrate integrating with the Apache Kafka messaging system http://kafka.apache.org.

    Connectors are used to create a bridge between topology streams and a Kafka cluster:

    • KafkaConsumer - subscribe to Kafka topics and create streams of messages.
    • KafkaProducer - publish streams of messages to Kafka topics.

    The sample publishes some messages to a Kafka topic. It also subscribes to the topic and reports the messages received. The messages received may include messages from prior runs of the sample.

    The sample requires a running Kafka cluster with the following characteristics:

    • the kafka topic (@code kafkaSampleTopic} has been created. e.g.
      ${KAFKA_HOME}/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic kafkaSampleTopic
    • the Kafka cluster's zookeeper connection is localhost:2181
    • the Kafka cluster's brokers addresses is localhost:9092

    Required IBM Streams environment variables:

    • STREAMS_INSTALL - the Streams installation directory
    • STREAMS_DOMAIN_ID - the Streams domain to use for context DISTRIBUTED
    • STREAMS_INSTANCE_ID - the Streams instance to use for context DISTRIBUTED

    See the Apache Kafka link above for information about setting up a Kafka cluster and creating a topic.

    This may be executed from the samples/java/functional directory as:

    • ant run.kafka.distributed - Using Apache Ant, this will run in distributed mode.
    • ant run.kafka - Using Apache Ant, this will run in standalone mode.
    • java -cp functionalsamples.jar:../../../com.ibm.streamsx.topology/lib/com.ibm.streamsx.topology.jar:$STREAMS_INSTALL/lib/com.ibm.streams.operator.samples.jar kafka.KafkaSample CONTEXT_TYPE - Run directly from the command line.
    • CONTEXT_TYPE is one of:
      • DISTRIBUTED - Run as an IBM Streams distributed application.
      • STANDALONE - Run as an IBM Streams standalone application.
      • BUNDLE - Create an IBM Streams application bundle.
      • TOOLKIT - Create an IBM Streams application toolkit.
    • An application execution within your IDE once you set the class path to include the correct jars.
    • Constructor Summary

      Constructors 
      Constructor and Description
      KafkaSample() 
    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method and Description
      static void main(java.lang.String[] args) 
      void publishSubscribe(java.lang.String contextType)
      Publish some messages to a topic, scribe to the topic and report received messages.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Method Detail

      • main

        public static void main(java.lang.String[] args)
                         throws java.lang.Exception
        Throws:
        java.lang.Exception
      • publishSubscribe

        public void publishSubscribe(java.lang.String contextType)
                              throws java.lang.Exception
        Publish some messages to a topic, scribe to the topic and report received messages.
        Parameters:
        contextType - string value of a StreamsContext.Type
        Throws:
        java.lang.Exception
streamsx.topology @ IBMStreams GitHub