Package org.apache.pulsar.reactive.client.internal.api

Class GroupOrderedMessageProcessors

java.lang.Object
org.apache.pulsar.reactive.client.internal.api.GroupOrderedMessageProcessors
public final class GroupOrderedMessageProcessors extends Object
Functions for implementing In-order parallel processing for Pulsar messages using Project Reactor.

A processing group is resolved for each message based on the message's key. The message flux is split into group fluxes based on the processing group. Each group flux is processes messages in order (one-by-one). Multiple group fluxes are processed in parallel.

  • Method Summary

    Modifier and Type
    Method
    Description
    static <T> reactor.core.publisher.Flux<reactor.core.publisher.GroupedFlux<Integer,org.apache.pulsar.client.api.Message<T>>>
    groupByProcessingGroup(reactor.core.publisher.Flux<org.apache.pulsar.client.api.Message<T>> messageFlux, MessageGroupingFunction groupingFunction, int numberOfGroups)
    Splits the flux of messages by message key into the given number of groups.
    static <T, R> reactor.core.publisher.Flux<R>
    processGroupsInOrderConcurrently(reactor.core.publisher.Flux<org.apache.pulsar.client.api.Message<T>> messageFlux, MessageGroupingFunction groupingFunction, Function<? super org.apache.pulsar.client.api.Message<T>,? extends org.reactivestreams.Publisher<? extends R>> messageHandler, reactor.core.scheduler.Scheduler scheduler, int concurrency)
    Processes the messages concurrently with the targeted concurrency.
    static <T, R> reactor.core.publisher.Flux<R>
    processGroupsInOrderInParallel(reactor.core.publisher.Flux<org.apache.pulsar.client.api.Message<T>> messageFlux, MessageGroupingFunction groupingFunction, Function<? super org.apache.pulsar.client.api.Message<T>,? extends org.reactivestreams.Publisher<? extends R>> messageHandler, reactor.core.scheduler.Scheduler scheduler, int parallelism)
    Processes the messages in parallel with the targeted parallelism.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • groupByProcessingGroup

      public static <T> reactor.core.publisher.Flux<reactor.core.publisher.GroupedFlux<Integer,org.apache.pulsar.client.api.Message<T>>> groupByProcessingGroup(reactor.core.publisher.Flux<org.apache.pulsar.client.api.Message<T>> messageFlux, MessageGroupingFunction groupingFunction, int numberOfGroups)
      Splits the flux of messages by message key into the given number of groups.
      Type Parameters:
      T - message payload type
      Parameters:
      messageFlux - flux of messages
      groupingFunction - function to use for resolving the group for a message
      numberOfGroups - number of processing groups
      Returns:
      the grouped flux of messages

    • processGroupsInOrderConcurrently

      public static <T, R> reactor.core.publisher.Flux<R> processGroupsInOrderConcurrently(reactor.core.publisher.Flux<org.apache.pulsar.client.api.Message<T>> messageFlux, MessageGroupingFunction groupingFunction, Function<? super org.apache.pulsar.client.api.Message<T>,? extends org.reactivestreams.Publisher<? extends R>> messageHandler, reactor.core.scheduler.Scheduler scheduler, int concurrency)
      Processes the messages concurrently with the targeted concurrency. Uses ".flatMap" in the implementation
      Type Parameters:
      T - message payload type
      R - message handler's resulting type
      Parameters:
      messageFlux - the flux of messages
      groupingFunction - function to use for resolving the group for a message
      messageHandler - message handler function
      scheduler - scheduler to use for subscribing to inner publishers
      concurrency - targeted concurrency level
      Returns:
      flux of message handler results

    • processGroupsInOrderInParallel

      public static <T, R> reactor.core.publisher.Flux<R> processGroupsInOrderInParallel(reactor.core.publisher.Flux<org.apache.pulsar.client.api.Message<T>> messageFlux, MessageGroupingFunction groupingFunction, Function<? super org.apache.pulsar.client.api.Message<T>,? extends org.reactivestreams.Publisher<? extends R>> messageHandler, reactor.core.scheduler.Scheduler scheduler, int parallelism)
      Processes the messages in parallel with the targeted parallelism. Uses ".parallel" in the implementation
      Type Parameters:
      T - message payload type
      R - message handler's resulting type
      Parameters:
      messageFlux - the flux of messages
      groupingFunction - function to use for resolving the group for a message
      messageHandler - message handler function
      scheduler - scheduler to use for subscribing to inner publishers
      parallelism - targeted level of parallelism
      Returns:
      flux of message handler results