A B C D E F G H I J K L M N O P R S T U V W

A

abort() - Method in interface org.apache.beam.runners.fnexecution.state.StateDelegator.Registration

De-registers the handler for all future requests for state for the registered process bundle instruction id.

accessPattern() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.SideInputSpec

addLengthPrefixedCoder(String, RunnerApi.Components.Builder, boolean) - Static method in class org.apache.beam.runners.fnexecution.wire.LengthPrefixUnknownCoders

Recursively traverses the coder tree and wraps the first unknown coder in every branch with a LengthPrefixCoder unless an ancestor coder is itself a LengthPrefixCoder.

addRunnerWireCoder(PipelineNode.PCollectionNode, RunnerApi.Components.Builder, RunnerApi.ExecutableStagePayload.WireCoderSetting) - Static method in class org.apache.beam.runners.fnexecution.wire.WireCoders

Creates a runner-side wire coder for a port read/write for the given PCollection.

addSdkWireCoder(PipelineNode.PCollectionNode, RunnerApi.Components.Builder, RunnerApi.ExecutableStagePayload.WireCoderSetting) - Static method in class org.apache.beam.runners.fnexecution.wire.WireCoders

Creates an SDK-side wire coder for a port read/write for the given PCollection.

append(K, W, Iterator<V>) - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.BagUserStateHandler

Appends the values to the bag user state for the given key and window.

ArtifactDestination() - Constructor for class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.ArtifactDestination

ArtifactRetrievalService - Class in org.apache.beam.runners.fnexecution.artifact

An ArtifactRetrievalService that uses FileSystems as its backing storage.

ArtifactRetrievalService() - Constructor for class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService

ArtifactRetrievalService(ArtifactResolver) - Constructor for class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService

ArtifactRetrievalService(int) - Constructor for class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService

ArtifactRetrievalService(ArtifactResolver, int) - Constructor for class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService

ArtifactStagingService - Class in org.apache.beam.runners.fnexecution.artifact

ArtifactStagingService(ArtifactStagingService.ArtifactDestinationProvider) - Constructor for class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService

ArtifactStagingService.ArtifactDestination - Class in org.apache.beam.runners.fnexecution.artifact

A pairing of a newly created artifact type and an ouptut stream that will be readable at that type.

ArtifactStagingService.ArtifactDestinationProvider - Interface in org.apache.beam.runners.fnexecution.artifact

Provides a concrete location to which artifacts can be staged on retrieval.

asResponseObserver() - Method in class org.apache.beam.runners.fnexecution.control.FnApiControlClient

B

BagUserStateSpec() - Constructor for class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.BagUserStateSpec

BatchSideInputHandlerFactory - Class in org.apache.beam.runners.fnexecution.translation

StateRequestHandler that uses a BatchSideInputHandlerFactory.SideInputGetter to access side inputs.

BatchSideInputHandlerFactory.SideInputGetter - Interface in org.apache.beam.runners.fnexecution.translation

Returns the value for the side input with the given PCollection id from the runner.

beamFilesystemArtifactDestinationProvider(String) - Static method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService

An ArtifactDestinationProvider that places new artifacts as files in a Beam filesystem.

BeamWorkerStatusGrpcService - Class in org.apache.beam.runners.fnexecution.status

A Fn Status service which can collect run-time status information from SDK harnesses for debugging purpose.

BundleCheckpointHandler - Interface in org.apache.beam.runners.fnexecution.control

A handler which is invoked when the SDK returns BeamFnApi.DelayedBundleApplications as part of the bundle completion.

BundleCheckpointHandlers - Class in org.apache.beam.runners.fnexecution.control

Utility methods for creating BundleCheckpointHandlers.

BundleCheckpointHandlers() - Constructor for class org.apache.beam.runners.fnexecution.control.BundleCheckpointHandlers

BundleCheckpointHandlers.StateAndTimerBundleCheckpointHandler<T> - Class in org.apache.beam.runners.fnexecution.control

A BundleCheckpointHandler which uses TimerInternals.TimerData ans ValueState to reschedule BeamFnApi.DelayedBundleApplication.

BundleFinalizationHandler - Interface in org.apache.beam.runners.fnexecution.control

A handler for the runner when a finalization request has been received.

BundleFinalizationHandlers - Class in org.apache.beam.runners.fnexecution.control

Utility methods for creating BundleFinalizationHandlers.

BundleFinalizationHandlers() - Constructor for class org.apache.beam.runners.fnexecution.control.BundleFinalizationHandlers

BundleFinalizationHandlers.InMemoryFinalizer - Class in org.apache.beam.runners.fnexecution.control

See BundleFinalizationHandlers.inMemoryFinalizer(InstructionRequestHandler) for details.

BundleProgressHandler - Interface in org.apache.beam.runners.fnexecution.control

A handler for bundle progress messages, both during bundle execution and on its completion.

BundleSplitHandler - Interface in org.apache.beam.runners.fnexecution.control

A handler which is invoked whenever an active bundle is split.

ByteStringCoder - Class in org.apache.beam.runners.fnexecution.wire

A duplicate of ByteStringCoder that uses the Apache Beam vendored protobuf.

C

clear(K, W) - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.BagUserStateHandler

Clears the bag user state for the given key and window.

close() - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService

close() - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService

close() - Method in class org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext

close() - Method in class org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory

close() - Method in class org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.WrappedSdkHarnessClient

close() - Method in class org.apache.beam.runners.fnexecution.control.FnApiControlClient

close() - Method in class org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService

close() - Method in interface org.apache.beam.runners.fnexecution.control.RemoteBundle

Closes this bundle.

close() - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor.ActiveBundle

Blocks until bundle processing is finished.

close() - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient

close() - Method in class org.apache.beam.runners.fnexecution.control.SingleEnvironmentInstanceJobBundleFactory

Deprecated.

close() - Method in class org.apache.beam.runners.fnexecution.data.GrpcDataService

close() - Method in class org.apache.beam.runners.fnexecution.environment.ProcessEnvironment

close() - Method in interface org.apache.beam.runners.fnexecution.environment.RemoteEnvironment

.

close() - Method in class org.apache.beam.runners.fnexecution.environment.StaticRemoteEnvironment

close() - Method in class org.apache.beam.runners.fnexecution.logging.GrpcLoggingService

close() - Method in class org.apache.beam.runners.fnexecution.provisioning.StaticGrpcProvisionService

close() - Method in class org.apache.beam.runners.fnexecution.state.GrpcStateService

close() - Method in class org.apache.beam.runners.fnexecution.status.BeamWorkerStatusGrpcService

coder() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.TimerSpec

consistentWithEquals() - Method in class org.apache.beam.runners.fnexecution.wire.ByteStringCoder

control(StreamObserver<BeamFnApi.InstructionRequest>) - Method in class org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService

Called by gRPC for each incoming connection from an SDK harness, and enqueue an available SDK harness client.

ControlClientPool - Interface in org.apache.beam.runners.fnexecution.control

A pool of control clients that brokers incoming SDK harness connections (in the form of InstructionRequestHandlers.

ControlClientPool.Sink - Interface in org.apache.beam.runners.fnexecution.control

A sink for InstructionRequestHandlers keyed by worker id.

ControlClientPool.Source - Interface in org.apache.beam.runners.fnexecution.control

A source of InstructionRequestHandlers.

create(String, ByteString, OutputStream) - Static method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.ArtifactDestination

create(JobInfo) - Static method in class org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext

create(JobInfo) - Static method in class org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory

create(JobInfo, Map<String, EnvironmentFactory.Provider>) - Static method in class org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory

create() - Static method in class org.apache.beam.runners.fnexecution.control.MapControlClientPool

Creates a MapControlClientPool.

create(String) - Method in interface org.apache.beam.runners.fnexecution.control.OutputReceiverFactory

Get a new FnDataReceiver for an output PCollection.

create(ReferenceCountingExecutableStageContextFactory.Creator, SerializableFunction<Object, Boolean>) - Static method in class org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory

create(EnvironmentFactory, GrpcFnServer<GrpcDataService>, GrpcFnServer<GrpcStateService>, IdGenerator) - Static method in class org.apache.beam.runners.fnexecution.control.SingleEnvironmentInstanceJobBundleFactory

Deprecated.

create(String, String) - Method in class org.apache.beam.runners.fnexecution.control.TimerReceiverFactory

create(PipelineOptions, ExecutorService, OutboundObserverFactory) - Static method in class org.apache.beam.runners.fnexecution.data.GrpcDataService

create(PipelineOptions, GrpcFnServer<GrpcLoggingService>, GrpcFnServer<FnApiControlClientPoolService>, ControlClientPool.Source) - Static method in class org.apache.beam.runners.fnexecution.environment.EmbeddedEnvironmentFactory

create(GrpcFnServer<FnApiControlClientPoolService>, GrpcFnServer<GrpcLoggingService>, GrpcFnServer<ArtifactRetrievalService>, GrpcFnServer<StaticGrpcProvisionService>, ControlClientPool.Source, IdGenerator) - Static method in class org.apache.beam.runners.fnexecution.environment.ExternalEnvironmentFactory

create(ProcessManager, RunnerApi.Environment, String, InstructionRequestHandler) - Static method in class org.apache.beam.runners.fnexecution.environment.ProcessEnvironment

create(ProcessManager, GrpcFnServer<StaticGrpcProvisionService>, ControlClientPool.Source, IdGenerator, PipelineOptions) - Static method in class org.apache.beam.runners.fnexecution.environment.ProcessEnvironmentFactory

create() - Static method in class org.apache.beam.runners.fnexecution.environment.ProcessManager

create(String, String, String, Struct) - Static method in class org.apache.beam.runners.fnexecution.provisioning.JobInfo

create(ProvisionApi.ProvisionInfo, HeaderAccessor) - Static method in class org.apache.beam.runners.fnexecution.provisioning.StaticGrpcProvisionService

create() - Static method in class org.apache.beam.runners.fnexecution.state.GrpcStateService

Create a new GrpcStateService.

create(Endpoints.ApiServiceDescriptor, HeaderAccessor) - Static method in class org.apache.beam.runners.fnexecution.status.BeamWorkerStatusGrpcService

Create new instance of BeamWorkerStatusGrpcService.

createEnvironment(RunnerApi.Environment, String) - Method in class org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory

Creates a new, active RemoteEnvironment backed by a local Docker container.

createEnvironment(RunnerApi.Environment, String) - Method in class org.apache.beam.runners.fnexecution.environment.EmbeddedEnvironmentFactory

createEnvironment(RunnerApi.Environment, String) - Method in interface org.apache.beam.runners.fnexecution.environment.EnvironmentFactory

Creates an active RunnerApi.Environment and returns a handle to it.

createEnvironment(RunnerApi.Environment, String) - Method in class org.apache.beam.runners.fnexecution.environment.ExternalEnvironmentFactory

Creates a new, active RemoteEnvironment backed by an unmanaged worker.

createEnvironment(RunnerApi.Environment, String) - Method in class org.apache.beam.runners.fnexecution.environment.ProcessEnvironmentFactory

Creates a new, active RemoteEnvironment backed by a forked process.

createEnvironment(RunnerApi.Environment, String) - Method in class org.apache.beam.runners.fnexecution.environment.StaticRemoteEnvironmentFactory

createEnvironmentFactory(GrpcFnServer<FnApiControlClientPoolService>, GrpcFnServer<GrpcLoggingService>, GrpcFnServer<ArtifactRetrievalService>, GrpcFnServer<StaticGrpcProvisionService>, ControlClientPool, IdGenerator) - Method in class org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.Provider

createEnvironmentFactory(GrpcFnServer<FnApiControlClientPoolService>, GrpcFnServer<GrpcLoggingService>, GrpcFnServer<ArtifactRetrievalService>, GrpcFnServer<StaticGrpcProvisionService>, ControlClientPool, IdGenerator) - Method in class org.apache.beam.runners.fnexecution.environment.EmbeddedEnvironmentFactory.Provider

createEnvironmentFactory(GrpcFnServer<FnApiControlClientPoolService>, GrpcFnServer<GrpcLoggingService>, GrpcFnServer<ArtifactRetrievalService>, GrpcFnServer<StaticGrpcProvisionService>, ControlClientPool, IdGenerator) - Method in interface org.apache.beam.runners.fnexecution.environment.EnvironmentFactory.Provider

Creates EnvironmentFactory for the provided GrpcServices.

createEnvironmentFactory(GrpcFnServer<FnApiControlClientPoolService>, GrpcFnServer<GrpcLoggingService>, GrpcFnServer<ArtifactRetrievalService>, GrpcFnServer<StaticGrpcProvisionService>, ControlClientPool, IdGenerator) - Method in class org.apache.beam.runners.fnexecution.environment.ExternalEnvironmentFactory.Provider

createEnvironmentFactory(GrpcFnServer<FnApiControlClientPoolService>, GrpcFnServer<GrpcLoggingService>, GrpcFnServer<ArtifactRetrievalService>, GrpcFnServer<StaticGrpcProvisionService>, ControlClientPool, IdGenerator) - Method in class org.apache.beam.runners.fnexecution.environment.ProcessEnvironmentFactory.Provider

createEnvironmentFactory(GrpcFnServer<FnApiControlClientPoolService>, GrpcFnServer<GrpcLoggingService>, GrpcFnServer<ArtifactRetrievalService>, GrpcFnServer<StaticGrpcProvisionService>, ControlClientPool, IdGenerator) - Method in class org.apache.beam.runners.fnexecution.environment.StaticRemoteEnvironmentFactory.Provider

createOutputMap(Iterable<String>) - Static method in class org.apache.beam.runners.fnexecution.translation.PipelineTranslatorUtils

Creates a mapping from PCollection id to output tag integer.

D

data(StreamObserver<BeamFnApi.Elements>) - Method in class org.apache.beam.runners.fnexecution.data.GrpcDataService

decode(InputStream) - Method in class org.apache.beam.runners.fnexecution.wire.ByteStringCoder

decode(InputStream, Coder.Context) - Method in class org.apache.beam.runners.fnexecution.wire.ByteStringCoder

decodeTimerDataTimerId(String) - Static method in class org.apache.beam.runners.fnexecution.control.TimerReceiverFactory

Decodes a string into the transform and timer family ids.

DEFAULT_BUFFER_SIZE - Static variable in class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService

DefaultExecutableStageContext - Class in org.apache.beam.runners.fnexecution.control

Implementation of a ExecutableStageContext.

DefaultJobBundleFactory - Class in org.apache.beam.runners.fnexecution.control

A JobBundleFactory for which the implementation can specify a custom EnvironmentFactory for environment management.

DefaultJobBundleFactory.ServerInfo - Class in org.apache.beam.runners.fnexecution.control

A container for EnvironmentFactory and its corresponding Grpc servers.

DefaultJobBundleFactory.WrappedSdkHarnessClient - Class in org.apache.beam.runners.fnexecution.control

Holder for an SdkHarnessClient along with its associated state and data servers.

delegateBasedUponType(EnumMap<BeamFnApi.StateKey.TypeCase, StateRequestHandler>) - Static method in class org.apache.beam.runners.fnexecution.state.StateRequestHandlers

Returns a StateRequestHandler which delegates to the supplied handler depending on the BeamFnApi.StateRequests type.

deregister() - Method in interface org.apache.beam.runners.fnexecution.state.StateDelegator.Registration

De-registers the handler for all future requests for state for the registered process bundle instruction id.

DockerEnvironmentFactory - Class in org.apache.beam.runners.fnexecution.environment

An EnvironmentFactory that creates docker containers by shelling out to docker.

DockerEnvironmentFactory.Provider - Class in org.apache.beam.runners.fnexecution.environment

Provider for DockerEnvironmentFactory.

E

elementCoder() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.SideInputSpec

elementCoder() - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.IterableSideInputHandler

Returns the Coder to use for the elements of the resulting values iterable.

EMBEDDED_ARTIFACT_URN - Static variable in class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService

EmbeddedEnvironmentFactory - Class in org.apache.beam.runners.fnexecution.environment

An EnvironmentFactory that communicates to a FnHarness which is executing in the same process.

EmbeddedEnvironmentFactory.Provider - Class in org.apache.beam.runners.fnexecution.environment

Provider of EmbeddedEnvironmentFactory.

encode(ByteString, OutputStream) - Method in class org.apache.beam.runners.fnexecution.wire.ByteStringCoder

encode(ByteString, OutputStream, Coder.Context) - Method in class org.apache.beam.runners.fnexecution.wire.ByteStringCoder

encodeToTimerDataTimerId(String, String) - Static method in class org.apache.beam.runners.fnexecution.control.TimerReceiverFactory

Encodes transform and timer family ids into a single string which retains the human readable format len(transformId):transformId:timerId.

EnvironmentFactory - Interface in org.apache.beam.runners.fnexecution.environment

Creates environments which communicate to an SdkHarnessClient.

EnvironmentFactory.Provider - Interface in org.apache.beam.runners.fnexecution.environment

Provider for a EnvironmentFactory and ServerFactory for the environment.

ExecutableProcessBundleDescriptor() - Constructor for class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.ExecutableProcessBundleDescriptor

ExecutableStageContext - Interface in org.apache.beam.runners.fnexecution.control

The context required in order to execute stages.

ExecutableStageContext.Factory - Interface in org.apache.beam.runners.fnexecution.control

Creates ExecutableStageContext instances.

ExternalEnvironmentFactory - Class in org.apache.beam.runners.fnexecution.environment

An EnvironmentFactory which requests workers via the given URL in the Environment.

ExternalEnvironmentFactory.Provider - Class in org.apache.beam.runners.fnexecution.environment

Provider of ExternalEnvironmentFactory.

F

FILE_ARTIFACT_URN - Static variable in class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService

finalizeAllOutstandingBundles() - Method in class org.apache.beam.runners.fnexecution.control.BundleFinalizationHandlers.InMemoryFinalizer

All finalization requests will be sent without waiting for the responses.

fireEligibleTimers(InMemoryTimerInternals, Map<KV<String, String>, FnDataReceiver<Timer>>, Object) - Static method in class org.apache.beam.runners.fnexecution.translation.PipelineTranslatorUtils

Fires all timers which are ready to be fired.

FnApiControlClient - Class in org.apache.beam.runners.fnexecution.control

A client for the control plane of an SDK harness, which can issue requests to it over the Fn API.

FnApiControlClientPoolService - Class in org.apache.beam.runners.fnexecution.control

A Fn API control service which adds incoming SDK harness connections to a sink.

FnDataService - Interface in org.apache.beam.runners.fnexecution.data

The FnDataService is able to forward inbound elements to a consumer and is also a consumer of outbound elements.

forBagUserStateHandlerFactory(ProcessBundleDescriptors.ExecutableProcessBundleDescriptor, StateRequestHandlers.BagUserStateHandlerFactory) - Static method in class org.apache.beam.runners.fnexecution.state.StateRequestHandlers

Returns an adapter which converts a StateRequestHandlers.BagUserStateHandlerFactory to a StateRequestHandler.

forHandler(RunnerApi.Environment, InstructionRequestHandler) - Static method in interface org.apache.beam.runners.fnexecution.environment.RemoteEnvironment

Create a new RemoteEnvironment for the provided RunnerApi.Environment and AutoCloseable InstructionRequestHandler.

forIterableSideInput(String, String, Coder<V>, Coder<W>) - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.SideInputHandlerFactory

Returns an StateRequestHandlers.IterableSideInputHandler for the given pTransformId, sideInputId.

forIterableSideInput(String, String, Coder<V>, Coder<W>) - Method in class org.apache.beam.runners.fnexecution.translation.BatchSideInputHandlerFactory

forIterableSideInput(String, String, Coder<V>, Coder<W>) - Method in class org.apache.beam.runners.fnexecution.translation.StreamingSideInputHandlerFactory

forMultimapSideInput(String, String, KvCoder<K, V>, Coder<W>) - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.SideInputHandlerFactory

Returns a StateRequestHandlers.MultimapSideInputHandler for the given pTransformId, sideInputId.

forMultimapSideInput(String, String, KvCoder<K, V>, Coder<W>) - Method in class org.apache.beam.runners.fnexecution.translation.BatchSideInputHandlerFactory

forMultimapSideInput(String, String, KvCoder<K, V>, Coder<W>) - Method in class org.apache.beam.runners.fnexecution.translation.StreamingSideInputHandlerFactory

forRequestObserver(String, StreamObserver<BeamFnApi.InstructionRequest>, ConcurrentMap<String, BeamFnApi.ProcessBundleDescriptor>) - Static method in class org.apache.beam.runners.fnexecution.control.FnApiControlClient

Returns a FnApiControlClient which will submit its requests to the provided observer.

forService(InstructionRequestHandler) - Static method in class org.apache.beam.runners.fnexecution.environment.StaticRemoteEnvironmentFactory

forSideInputHandlerFactory(Map<String, Map<String, ProcessBundleDescriptors.SideInputSpec>>, StateRequestHandlers.SideInputHandlerFactory) - Static method in class org.apache.beam.runners.fnexecution.state.StateRequestHandlers

Returns an adapter which converts a StateRequestHandlers.SideInputHandlerFactory to a StateRequestHandler.

forStage(ExecutableStage) - Method in class org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory

forStage(ExecutableStage) - Method in interface org.apache.beam.runners.fnexecution.control.JobBundleFactory

forStage(ExecutableStage) - Method in class org.apache.beam.runners.fnexecution.control.SingleEnvironmentInstanceJobBundleFactory

Deprecated.

forStage(ExecutableStage, BatchSideInputHandlerFactory.SideInputGetter) - Static method in class org.apache.beam.runners.fnexecution.translation.BatchSideInputHandlerFactory

Creates a new state handler for the given stage.

forStage(ExecutableStage, Map<RunnerApi.ExecutableStagePayload.SideInputId, PCollectionView<?>>, SideInputHandler) - Static method in class org.apache.beam.runners.fnexecution.translation.StreamingSideInputHandlerFactory

Creates a new state handler for the given stage.

forUserState(String, String, Coder<K>, Coder<V>, Coder<W>) - Method in class org.apache.beam.runners.fnexecution.state.InMemoryBagUserStateFactory

forUserState(String, String, Coder<K>, Coder<V>, Coder<W>) - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.BagUserStateHandlerFactory

forWriter(LogWriter) - Static method in class org.apache.beam.runners.fnexecution.logging.GrpcLoggingService

fromExecutableStage(String, ExecutableStage, Endpoints.ApiServiceDescriptor, Endpoints.ApiServiceDescriptor) - Static method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors

Note that the BeamFnApi.ProcessBundleDescriptor is constructed by: Adding gRPC read and write nodes wiring them to the specified data endpoint.

fromExecutableStage(String, ExecutableStage, Endpoints.ApiServiceDescriptor) - Static method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors

fromFile(String) - Static method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.ArtifactDestination

fromFile(String, OutputStream) - Static method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.ArtifactDestination

G

get(JobInfo) - Method in interface org.apache.beam.runners.fnexecution.control.ExecutableStageContext.Factory

Get or create ExecutableStageContext for given JobInfo.

get(JobInfo) - Method in class org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory

get(K, W) - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.BagUserStateHandler

Returns an Iterable of values representing the bag user state for the given key and window.

get(W) - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.IterableSideInputHandler

Returns an Iterable of values representing the side input for the given window.

get(W) - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.MultimapSideInputHandler

Returns an Iterable of keys representing the side input for the given window.

get(K, W) - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.MultimapSideInputHandler

Returns an Iterable of values representing the side input for the given key and window.

getAllWorkerStatuses(long, TimeUnit) - Method in class org.apache.beam.runners.fnexecution.status.BeamWorkerStatusGrpcService

Get all the statuses from all connected SDK harnesses within specified timeout.

getArtifact(ArtifactApi.GetArtifactRequest, StreamObserver<ArtifactApi.GetArtifactResponse>) - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService

getArtifact(RunnerApi.ArtifactInformation) - Static method in class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService

getBagUserStateSpecs() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.ExecutableProcessBundleDescriptor

Get a mapping from PTransform id to user state input id to bag user states that are used during execution.

getBundle(OutputReceiverFactory, StateRequestHandler, BundleProgressHandler) - Method in interface org.apache.beam.runners.fnexecution.control.StageBundleFactory

Get a new bundle for processing the data in an executable stage.

getBundle(OutputReceiverFactory, StateRequestHandler, BundleProgressHandler, BundleFinalizationHandler) - Method in interface org.apache.beam.runners.fnexecution.control.StageBundleFactory

getBundle(OutputReceiverFactory, StateRequestHandler, BundleProgressHandler, BundleFinalizationHandler, BundleCheckpointHandler) - Method in interface org.apache.beam.runners.fnexecution.control.StageBundleFactory

getBundle(OutputReceiverFactory, TimerReceiverFactory, StateRequestHandler, BundleProgressHandler) - Method in interface org.apache.beam.runners.fnexecution.control.StageBundleFactory

Get a new bundle for processing the data in an executable stage.

getBundle(OutputReceiverFactory, TimerReceiverFactory, StateRequestHandler, BundleProgressHandler, BundleFinalizationHandler) - Method in interface org.apache.beam.runners.fnexecution.control.StageBundleFactory

getBundle(OutputReceiverFactory, TimerReceiverFactory, StateRequestHandler, BundleProgressHandler, BundleFinalizationHandler, BundleCheckpointHandler) - Method in interface org.apache.beam.runners.fnexecution.control.StageBundleFactory

getCacheTokens() - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandler

Retrieves a list of valid cache tokens.

getCoder() - Method in class org.apache.beam.runners.fnexecution.control.RemoteOutputReceiver

getCoder() - Method in class org.apache.beam.runners.fnexecution.data.RemoteInputDestination

getDefault() - Static method in class org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter

getDestination(String, String) - Method in interface org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.ArtifactDestinationProvider

getEncodedElementByteSize(ByteString) - Method in class org.apache.beam.runners.fnexecution.wire.ByteStringCoder

getEncodedTypeDescriptor() - Method in class org.apache.beam.runners.fnexecution.wire.ByteStringCoder

getEnvironment() - Method in class org.apache.beam.runners.fnexecution.environment.ProcessEnvironment

getEnvironment() - Method in interface org.apache.beam.runners.fnexecution.environment.RemoteEnvironment

Return the environment that the remote handles.

getEnvironment() - Method in class org.apache.beam.runners.fnexecution.environment.RemoteEnvironment.SimpleRemoteEnvironment

getEnvironment() - Method in class org.apache.beam.runners.fnexecution.environment.StaticRemoteEnvironment

getExecutableStageIntermediateId(PipelineNode.PTransformNode) - Static method in class org.apache.beam.runners.fnexecution.translation.PipelineTranslatorUtils

getId() - Method in interface org.apache.beam.runners.fnexecution.control.RemoteBundle

Get an id used to represent this bundle.

getId() - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor.ActiveBundle

Returns an id used to represent this bundle.

getInputId(PipelineNode.PTransformNode) - Static method in class org.apache.beam.runners.fnexecution.translation.PipelineTranslatorUtils

getInputReceivers() - Method in interface org.apache.beam.runners.fnexecution.control.RemoteBundle

Get a map of PCollection ids to receivers which consume input elements, forwarding them to the remote environment.

getInputReceivers() - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor.ActiveBundle

Get a map of PCollection ids to receivers which consume input elements, forwarding them to the remote environment.

getInstructionRequestHandler() - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient

getInstructionRequestHandler() - Method in interface org.apache.beam.runners.fnexecution.control.StageBundleFactory

getInstructionRequestHandler() - Method in class org.apache.beam.runners.fnexecution.environment.ProcessEnvironment

getInstructionRequestHandler() - Method in interface org.apache.beam.runners.fnexecution.environment.RemoteEnvironment

Return an InstructionRequestHandler which can communicate with the environment.

getInstructionRequestHandler() - Method in class org.apache.beam.runners.fnexecution.environment.RemoteEnvironment.SimpleRemoteEnvironment

getInstructionRequestHandler() - Method in class org.apache.beam.runners.fnexecution.environment.StaticRemoteEnvironment

getOutputId(PipelineNode.PTransformNode) - Static method in class org.apache.beam.runners.fnexecution.translation.PipelineTranslatorUtils

getOutputStream() - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.ArtifactDestination

getProcessBundleDescriptor(String) - Method in class org.apache.beam.runners.fnexecution.control.FnApiControlClient

getProcessBundleDescriptor(BeamFnApi.GetProcessBundleDescriptorRequest, StreamObserver<BeamFnApi.ProcessBundleDescriptor>) - Method in class org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService

getProcessBundleDescriptor() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.ExecutableProcessBundleDescriptor

getProcessBundleDescriptor() - Method in interface org.apache.beam.runners.fnexecution.control.StageBundleFactory

getProcessor(BeamFnApi.ProcessBundleDescriptor, List<RemoteInputDestination>) - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient

Provides SdkHarnessClient.BundleProcessor that is capable of processing bundles not containing timers or state accesses such as: Side inputs User state Remote references

getProcessor(BeamFnApi.ProcessBundleDescriptor, List<RemoteInputDestination>, StateDelegator) - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient

Provides SdkHarnessClient.BundleProcessor that is capable of processing bundles not containing timers.

getProcessor(BeamFnApi.ProcessBundleDescriptor, List<RemoteInputDestination>, StateDelegator, Map<String, Map<String, ProcessBundleDescriptors.TimerSpec>>) - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient

Provides SdkHarnessClient.BundleProcessor that is capable of processing bundles containing timers and state accesses such as: Side inputs User state Remote references

getProvisionInfo(ProvisionApi.GetProvisionInfoRequest, StreamObserver<ProvisionApi.GetProvisionInfoResponse>) - Method in class org.apache.beam.runners.fnexecution.provisioning.StaticGrpcProvisionService

getPTransformId() - Method in class org.apache.beam.runners.fnexecution.data.RemoteInputDestination

getReceiver() - Method in class org.apache.beam.runners.fnexecution.control.RemoteOutputReceiver

getRemoteInputDestinations() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.ExecutableProcessBundleDescriptor

Get RemoteInputDestinations that input data are sent to the BeamFnApi.ProcessBundleDescriptor over.

getRemoteOutputCoders() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.ExecutableProcessBundleDescriptor

Get all of the transforms materialized by this ProcessBundleDescriptors.ExecutableProcessBundleDescriptor and the Java Coder for the wire format of that transform.

getServerFactory() - Method in class org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.Provider

getServerFactory() - Method in class org.apache.beam.runners.fnexecution.environment.EmbeddedEnvironmentFactory.Provider

getServerFactory() - Method in interface org.apache.beam.runners.fnexecution.environment.EnvironmentFactory.Provider

Create the ServerFactory applicable to this environment.

getServerFactory() - Method in class org.apache.beam.runners.fnexecution.environment.ExternalEnvironmentFactory.Provider

getSideInput(String) - Method in interface org.apache.beam.runners.fnexecution.translation.BatchSideInputHandlerFactory.SideInputGetter

getSideInputs(ExecutableStage) - Static method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors

getSideInputSpecs() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.ExecutableProcessBundleDescriptor

Get a mapping from PTransform id to side input id to side inputs that are used during execution.

getSingleWorkerStatus(String, long, TimeUnit) - Method in class org.apache.beam.runners.fnexecution.status.BeamWorkerStatusGrpcService

Get the latest SDK worker status from the client's corresponding SDK harness.

getSink() - Method in interface org.apache.beam.runners.fnexecution.control.ControlClientPool

Sink for control clients.

getSink() - Method in class org.apache.beam.runners.fnexecution.control.MapControlClientPool

getSource() - Method in interface org.apache.beam.runners.fnexecution.control.ControlClientPool

Source of control clients.

getSource() - Method in class org.apache.beam.runners.fnexecution.control.MapControlClientPool

getStageBundleFactory(ExecutableStage) - Method in class org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext

getStageBundleFactory(ExecutableStage) - Method in interface org.apache.beam.runners.fnexecution.control.ExecutableStageContext

getStagedArtifacts(String) - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService

Returns the rewritten artifacts associated with this job, keyed by environment.

getTimerReceivers() - Method in interface org.apache.beam.runners.fnexecution.control.RemoteBundle

Get a map of (transform id, timer id) to receivers which consume timers, forwarding them to the remote environment.

getTimerReceivers() - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor.ActiveBundle

getTimerSpec() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.TimerSpec

getTimerSpecs() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.ExecutableProcessBundleDescriptor

Get a mapping from PTransform id to timer id to timer specs that are used during execution.

getTypePayload() - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.ArtifactDestination

getTypeUrn() - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.ArtifactDestination

getWindowedValueCoder(String, RunnerApi.Components) - Static method in class org.apache.beam.runners.fnexecution.translation.PipelineTranslatorUtils

getWindowingStrategy(String, RunnerApi.Components) - Static method in class org.apache.beam.runners.fnexecution.translation.PipelineTranslatorUtils

getWorkerId() - Method in class org.apache.beam.runners.fnexecution.control.FnApiControlClient

GrpcDataService - Class in org.apache.beam.runners.fnexecution.data

A FnDataService implemented via gRPC.

GrpcDataService() - Constructor for class org.apache.beam.runners.fnexecution.data.GrpcDataService

Deprecated.

This constructor is for migrating Dataflow purpose only.

GrpcLoggingService - Class in org.apache.beam.runners.fnexecution.logging

An implementation of the Beam Fn Logging Service over gRPC.

GrpcStateService - Class in org.apache.beam.runners.fnexecution.state

An implementation of the Beam Fn State service.

H

handle(BeamFnApi.InstructionRequest) - Method in class org.apache.beam.runners.fnexecution.control.FnApiControlClient

handle(BeamFnApi.InstructionRequest) - Method in interface org.apache.beam.runners.fnexecution.control.InstructionRequestHandler

handle(BeamFnApi.StateRequest) - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandler

Handle a BeamFnApi.StateRequest asynchronously.

hasUnboundedPCollections(RunnerApi.Pipeline) - Static method in class org.apache.beam.runners.fnexecution.translation.PipelineTranslatorUtils

Indicates whether the given pipeline has any unbounded PCollections.

I

ignored() - Static method in interface org.apache.beam.runners.fnexecution.control.BundleProgressHandler

Returns a handler that ignores metrics.

INHERIT_IO_FILE - Static variable in class org.apache.beam.runners.fnexecution.environment.ProcessManager

A symbolic file to indicate that we want to inherit I/O of parent process.

InMemoryBagUserStateFactory<K,V,W extends org.apache.beam.sdk.transforms.windowing.BoundedWindow> - Class in org.apache.beam.runners.fnexecution.state

Holds user state in memory.

InMemoryBagUserStateFactory() - Constructor for class org.apache.beam.runners.fnexecution.state.InMemoryBagUserStateFactory

inMemoryFinalizer(InstructionRequestHandler) - Static method in class org.apache.beam.runners.fnexecution.control.BundleFinalizationHandlers

A bundle finalizer that stores all bundle finalization requests in memory.

instantiateCoder(String, RunnerApi.Components) - Static method in class org.apache.beam.runners.fnexecution.translation.PipelineTranslatorUtils

Creates a coder for a given PCollection id from the Proto definition.

instantiateRunnerWireCoder(PipelineNode.PCollectionNode, RunnerApi.Components) - Static method in class org.apache.beam.runners.fnexecution.wire.WireCoders

Instantiates a runner-side wire coder for the given PCollection.

instantiateRunnerWireCoder(PipelineNode.PCollectionNode, RunnerApi.Components, RunnerApi.ExecutableStagePayload.WireCoderSetting) - Static method in class org.apache.beam.runners.fnexecution.wire.WireCoders

Instantiates a runner-side wire coder for the given PCollection.

InstructionRequestHandler - Interface in org.apache.beam.runners.fnexecution.control

Interface for any function that can handle a Fn API BeamFnApi.InstructionRequest.

isAliveOrThrow() - Method in class org.apache.beam.runners.fnexecution.environment.ProcessManager.RunningProcess

Checks if the underlying process is still running.

isRegisterByteSizeObserverCheap(ByteString) - Method in class org.apache.beam.runners.fnexecution.wire.ByteStringCoder

isSdfTimer(String) - Static method in class org.apache.beam.runners.fnexecution.control.BundleCheckpointHandlers.StateAndTimerBundleCheckpointHandler

A helper function to help check whether the given timer is the timer which is set for rescheduling BeamFnApi.DelayedBundleApplication.

J

JobBundleFactory - Interface in org.apache.beam.runners.fnexecution.control

A factory that has all job-scoped information, and can be combined with stage-scoped information to create a StageBundleFactory.

jobId() - Method in class org.apache.beam.runners.fnexecution.provisioning.JobInfo

JobInfo - Class in org.apache.beam.runners.fnexecution.provisioning

A subset of ProvisionApi.ProvisionInfo that specifies a unique job, while omitting fields that are not known to the runner operator.

JobInfo() - Constructor for class org.apache.beam.runners.fnexecution.provisioning.JobInfo

jobName() - Method in class org.apache.beam.runners.fnexecution.provisioning.JobInfo

K

keyCoder() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.BagUserStateSpec

keyCoder() - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.MultimapSideInputHandler

Returns the Coder to use for the elements of the resulting keys iterable.

L

LengthPrefixUnknownCoders - Class in org.apache.beam.runners.fnexecution.wire

Utilities for replacing or wrapping unknown coders with LengthPrefixCoder.

LengthPrefixUnknownCoders() - Constructor for class org.apache.beam.runners.fnexecution.wire.LengthPrefixUnknownCoders

log(BeamFnApi.LogEntry) - Method in interface org.apache.beam.runners.fnexecution.logging.LogWriter

Write the contents of the Log Entry to some logging backend.

log(BeamFnApi.LogEntry) - Method in class org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter

logging(StreamObserver<BeamFnApi.LogControl>) - Method in class org.apache.beam.runners.fnexecution.logging.GrpcLoggingService

LogWriter - Interface in org.apache.beam.runners.fnexecution.logging

A consumer of Beam Log Entries.

M

MapControlClientPool - Class in org.apache.beam.runners.fnexecution.control

A ControlClientPool backed by a client map.

N

NeedsDocker - Interface in org.apache.beam.runners.fnexecution.environment.testing

Category for integration tests that require Docker.

newBundle(Map<String, RemoteOutputReceiver<?>>, BundleProgressHandler) - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor

Start a new bundle for the given BeamFnApi.ProcessBundleDescriptor identifier.

newBundle(Map<String, RemoteOutputReceiver<?>>, StateRequestHandler, BundleProgressHandler) - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor

Start a new bundle for the given BeamFnApi.ProcessBundleDescriptor identifier.

newBundle(Map<String, RemoteOutputReceiver<?>>, Map<KV<String, String>, RemoteOutputReceiver<Timer<?>>>, StateRequestHandler, BundleProgressHandler, BundleFinalizationHandler, BundleCheckpointHandler) - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor

Start a new bundle for the given BeamFnApi.ProcessBundleDescriptor identifier.

newBundle(Map<String, RemoteOutputReceiver<?>>, Map<KV<String, String>, RemoteOutputReceiver<Timer<?>>>, StateRequestHandler, BundleProgressHandler, BundleSplitHandler, BundleCheckpointHandler, BundleFinalizationHandler) - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor

Start a new bundle for the given BeamFnApi.ProcessBundleDescriptor identifier.

O

of(BeamFnApi.ProcessBundleDescriptor, List<RemoteInputDestination>, Map<String, Coder>, Map<String, Map<String, ProcessBundleDescriptors.SideInputSpec>>, Map<String, Map<String, ProcessBundleDescriptors.BagUserStateSpec>>, Map<String, Map<String, ProcessBundleDescriptors.TimerSpec>>) - Static method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.ExecutableProcessBundleDescriptor

of(String, String, RunnerApi.FunctionSpec, Coder<T>, Coder<W>) - Static method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.SideInputSpec

of(Coder<T>, FnDataReceiver<T>) - Static method in class org.apache.beam.runners.fnexecution.control.RemoteOutputReceiver

of(Coder<T>, String) - Static method in class org.apache.beam.runners.fnexecution.data.RemoteInputDestination

of() - Static method in class org.apache.beam.runners.fnexecution.wire.ByteStringCoder

offer(ArtifactRetrievalService, ArtifactStagingServiceGrpc.ArtifactStagingServiceStub, String) - Static method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService

Lazily stages artifacts by letting an ArtifactStagingService resolve and request artifacts.

offeringClientsToPool(ControlClientPool.Sink, HeaderAccessor) - Static method in class org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService

Creates a new FnApiControlClientPoolService which will enqueue and vend new SDK harness connections.

onCheckpoint(BeamFnApi.ProcessBundleResponse) - Method in interface org.apache.beam.runners.fnexecution.control.BundleCheckpointHandler

onCheckpoint(BeamFnApi.ProcessBundleResponse) - Method in class org.apache.beam.runners.fnexecution.control.BundleCheckpointHandlers.StateAndTimerBundleCheckpointHandler

onClose(Consumer<FnApiControlClient>) - Method in class org.apache.beam.runners.fnexecution.control.FnApiControlClient

onCompleted(BeamFnApi.ProcessBundleResponse) - Method in interface org.apache.beam.runners.fnexecution.control.BundleProgressHandler

Handles the bundle's completion report.

onProgress(BeamFnApi.ProcessBundleProgressResponse) - Method in interface org.apache.beam.runners.fnexecution.control.BundleProgressHandler

Handles a progress report from the bundle while it is executing.

org.apache.beam.runners.fnexecution.artifact - package org.apache.beam.runners.fnexecution.artifact

Pipeline execution-time artifact-management services, including abstract implementations of the Artifact Retrieval Service.

org.apache.beam.runners.fnexecution.control - package org.apache.beam.runners.fnexecution.control

Utilities for a Beam runner to interact with the Fn API Control Service via java abstractions.

org.apache.beam.runners.fnexecution.data - package org.apache.beam.runners.fnexecution.data

Utilities for a Beam runner to interact with the Fn API Data Service via java abstractions.

org.apache.beam.runners.fnexecution.environment - package org.apache.beam.runners.fnexecution.environment

Classes used to instantiate and manage SDK harness environments.

org.apache.beam.runners.fnexecution.environment.testing - package org.apache.beam.runners.fnexecution.environment.testing

Test utilities for the environment management package.

org.apache.beam.runners.fnexecution.logging - package org.apache.beam.runners.fnexecution.logging

Classes used to log informational messages over the Beam Fn Logging Service.

org.apache.beam.runners.fnexecution.provisioning - package org.apache.beam.runners.fnexecution.provisioning

Provision api services.

org.apache.beam.runners.fnexecution.state - package org.apache.beam.runners.fnexecution.state

State API services.

org.apache.beam.runners.fnexecution.status - package org.apache.beam.runners.fnexecution.status

Worker Status API services.

org.apache.beam.runners.fnexecution.translation - package org.apache.beam.runners.fnexecution.translation

Shared utilities for a Beam runner to translate portable pipelines.

org.apache.beam.runners.fnexecution.wire - package org.apache.beam.runners.fnexecution.wire

Wire coders for communications between runner and SDK harness.

OutputReceiverFactory - Interface in org.apache.beam.runners.fnexecution.control

A factory that can create output receivers during an executable stage.

P

pipelineOptions() - Method in class org.apache.beam.runners.fnexecution.provisioning.JobInfo

PipelineTranslatorUtils - Class in org.apache.beam.runners.fnexecution.translation

Utilities for pipeline translation.

ProcessBundleDescriptors - Class in org.apache.beam.runners.fnexecution.control

Utility methods for creating BeamFnApi.ProcessBundleDescriptor instances.

ProcessBundleDescriptors() - Constructor for class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors

ProcessBundleDescriptors.BagUserStateSpec<K,V,W extends org.apache.beam.sdk.transforms.windowing.BoundedWindow> - Class in org.apache.beam.runners.fnexecution.control

A container type storing references to the key, value, and window Coder used when handling bag user state requests.

ProcessBundleDescriptors.ExecutableProcessBundleDescriptor - Class in org.apache.beam.runners.fnexecution.control

ProcessBundleDescriptors.SideInputSpec<T,W extends org.apache.beam.sdk.transforms.windowing.BoundedWindow> - Class in org.apache.beam.runners.fnexecution.control

A container type storing references to the value, and window Coder used when handling side input state requests.

ProcessBundleDescriptors.TimerSpec<K,V,W extends org.apache.beam.sdk.transforms.windowing.BoundedWindow> - Class in org.apache.beam.runners.fnexecution.control

A container type storing references to the key, timer and payload coders and the remote input destination used when handling timer requests.

ProcessEnvironment - Class in org.apache.beam.runners.fnexecution.environment

Environment for process-based execution.

ProcessEnvironmentFactory - Class in org.apache.beam.runners.fnexecution.environment

An EnvironmentFactory which forks processes based on the parameters in the Environment.

ProcessEnvironmentFactory.Provider - Class in org.apache.beam.runners.fnexecution.environment

Provider of ProcessEnvironmentFactory.

ProcessManager - Class in org.apache.beam.runners.fnexecution.environment

A simple process manager which forks processes and kills them if necessary.

ProcessManager.RunningProcess - Class in org.apache.beam.runners.fnexecution.environment

Provider(PipelineOptions) - Constructor for class org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.Provider

Provider(PipelineOptions) - Constructor for class org.apache.beam.runners.fnexecution.environment.EmbeddedEnvironmentFactory.Provider

Provider() - Constructor for class org.apache.beam.runners.fnexecution.environment.ExternalEnvironmentFactory.Provider

Provider(PipelineOptions) - Constructor for class org.apache.beam.runners.fnexecution.environment.ProcessEnvironmentFactory.Provider

Provider(InstructionRequestHandler) - Constructor for class org.apache.beam.runners.fnexecution.environment.StaticRemoteEnvironmentFactory.Provider

put(String, InstructionRequestHandler) - Method in interface org.apache.beam.runners.fnexecution.control.ControlClientPool.Sink

Puts an InstructionRequestHandler into a client pool.

R

receive(LogicalEndpoint, Coder<T>, FnDataReceiver<T>) - Method in interface org.apache.beam.runners.fnexecution.data.FnDataService

Registers a receiver to be notified upon any incoming elements.

receive(LogicalEndpoint, Coder<T>, FnDataReceiver<T>) - Method in class org.apache.beam.runners.fnexecution.data.GrpcDataService

ReferenceCountingExecutableStageContextFactory - Class in org.apache.beam.runners.fnexecution.control

ExecutableStageContext.Factory which counts ExecutableStageContext reference for book keeping.

ReferenceCountingExecutableStageContextFactory.Creator - Interface in org.apache.beam.runners.fnexecution.control

Interface for creator which extends Serializable.

registerEnvironment(String, RunnerApi.Environment) - Method in class org.apache.beam.runners.fnexecution.provisioning.StaticGrpcProvisionService

registerForProcessBundleInstructionId(String, StateRequestHandler) - Method in class org.apache.beam.runners.fnexecution.state.GrpcStateService

registerForProcessBundleInstructionId(String, StateRequestHandler) - Method in interface org.apache.beam.runners.fnexecution.state.StateDelegator

Registers the supplied handler for the given process bundle instruction id for all BeamFnApi.StateRequests with a matching id.

registerJob(String, Map<String, List<RunnerApi.ArtifactInformation>>) - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService

Registers a set of artifacts to be staged with this service.

registerProcessBundleDescriptor(BeamFnApi.ProcessBundleDescriptor) - Method in class org.apache.beam.runners.fnexecution.control.FnApiControlClient

registerProcessBundleDescriptor(BeamFnApi.ProcessBundleDescriptor) - Method in interface org.apache.beam.runners.fnexecution.control.InstructionRequestHandler

RemoteBundle - Interface in org.apache.beam.runners.fnexecution.control

A bundle capable of handling input data elements for a bundle descriptor by forwarding them to a remote environment for processing.

RemoteEnvironment - Interface in org.apache.beam.runners.fnexecution.environment

A handle to an available remote RunnerApi.Environment.

RemoteEnvironment.SimpleRemoteEnvironment - Class in org.apache.beam.runners.fnexecution.environment

A RemoteEnvironment which uses the default RemoteEnvironment.close() behavior.

RemoteInputDestination<T> - Class in org.apache.beam.runners.fnexecution.data

A pair of Coder and BeamFnApi.Target which specifies the arguments to a FnDataService to send data to a remote harness.

RemoteInputDestination() - Constructor for class org.apache.beam.runners.fnexecution.data.RemoteInputDestination

RemoteOutputReceiver<T> - Class in org.apache.beam.runners.fnexecution.control

A pair of Coder and FnDataReceiver which can be registered to receive elements for a LogicalEndpoint.

RemoteOutputReceiver() - Constructor for class org.apache.beam.runners.fnexecution.control.RemoteOutputReceiver

removeStagedArtifacts(String) - Method in interface org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.ArtifactDestinationProvider

removeStagedArtifacts(String) - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService

requestProgress() - Method in interface org.apache.beam.runners.fnexecution.control.RemoteBundle

Ask the remote bundle for progress.

requestProgress() - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor.ActiveBundle

requestsFinalization(String) - Method in interface org.apache.beam.runners.fnexecution.control.BundleFinalizationHandler

This callback is invoked whenever an inflight bundle that is being processed requests finalization.

requestsFinalization(String) - Method in class org.apache.beam.runners.fnexecution.control.BundleFinalizationHandlers.InMemoryFinalizer

resetForNewKey() - Method in class org.apache.beam.runners.fnexecution.state.InMemoryBagUserStateFactory

Prepares previous emitted state handlers for processing a new key.

resolveArtifacts(ArtifactApi.ResolveArtifactsRequest, StreamObserver<ArtifactApi.ResolveArtifactsResponse>) - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService

retrievalToken() - Method in class org.apache.beam.runners.fnexecution.provisioning.JobInfo

reverseArtifactRetrievalService(StreamObserver<ArtifactApi.ArtifactRequestWrapper>) - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService

S

SDF_PREFIX - Static variable in class org.apache.beam.runners.fnexecution.control.BundleCheckpointHandlers.StateAndTimerBundleCheckpointHandler

SdkHarnessClient - Class in org.apache.beam.runners.fnexecution.control

A high-level client for an SDK harness.

SdkHarnessClient.BundleProcessor - Class in org.apache.beam.runners.fnexecution.control

A processor capable of creating bundles for some registered BeamFnApi.ProcessBundleDescriptor.

SdkHarnessClient.BundleProcessor.ActiveBundle - Class in org.apache.beam.runners.fnexecution.control

An active bundle for a particular BeamFnApi.ProcessBundleDescriptor.

send(LogicalEndpoint, Coder<T>) - Method in interface org.apache.beam.runners.fnexecution.data.FnDataService

Deprecated.

Migrate to use BeamFnDataOutboundAggregator directly for sending outbound data.

send(LogicalEndpoint, Coder<T>) - Method in class org.apache.beam.runners.fnexecution.data.GrpcDataService

ServerInfo() - Constructor for class org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.ServerInfo

sideInputId() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.SideInputSpec

SideInputSpec() - Constructor for class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.SideInputSpec

SimpleRemoteEnvironment() - Constructor for class org.apache.beam.runners.fnexecution.environment.RemoteEnvironment.SimpleRemoteEnvironment

SingleEnvironmentInstanceJobBundleFactory - Class in org.apache.beam.runners.fnexecution.control

Deprecated.

replace with a DefaultJobBundleFactory when appropriate if the EnvironmentFactory is a DockerEnvironmentFactory, or create an InProcessJobBundleFactory and inline the creation of the environment if appropriate.

Slf4jLogWriter - Class in org.apache.beam.runners.fnexecution.logging

A LogWriter which uses an SLF4J Logger as the underlying log backend.

split(BeamFnApi.ProcessBundleSplitResponse) - Method in interface org.apache.beam.runners.fnexecution.control.BundleSplitHandler

split(double) - Method in interface org.apache.beam.runners.fnexecution.control.RemoteBundle

Ask the remote bundle to split its current processing based upon its knowledge of remaining work.

split(double) - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor.ActiveBundle

StageBundleFactory - Interface in org.apache.beam.runners.fnexecution.control

A bundle factory scoped to a particular ExecutableStage, which has all of the resources it needs to provide new RemoteBundles.

STAGING_TO_ARTIFACT_URN - Static variable in class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService

startProcess(String, String, List<String>, Map<String, String>) - Method in class org.apache.beam.runners.fnexecution.environment.ProcessManager

Forks a process with the given command, arguments, and additional environment variables.

startProcess(String, String, List<String>, Map<String, String>, File) - Method in class org.apache.beam.runners.fnexecution.environment.ProcessManager

state(StreamObserver<BeamFnApi.StateResponse>) - Method in class org.apache.beam.runners.fnexecution.state.GrpcStateService

StateAndTimerBundleCheckpointHandler(TimerInternalsFactory<T>, StateInternalsFactory<T>, Coder<WindowedValue<T>>, Coder) - Constructor for class org.apache.beam.runners.fnexecution.control.BundleCheckpointHandlers.StateAndTimerBundleCheckpointHandler

StateDelegator - Interface in org.apache.beam.runners.fnexecution.state

The StateDelegator is able to delegate BeamFnApi.StateRequests to a set of registered handlers.

StateDelegator.Registration - Interface in org.apache.beam.runners.fnexecution.state

Allows callers to deregister from receiving further state requests.

StateRequestHandler - Interface in org.apache.beam.runners.fnexecution.state

Handler for StateRequests.

StateRequestHandlers - Class in org.apache.beam.runners.fnexecution.state

A set of utility methods which construct StateRequestHandlers.

StateRequestHandlers() - Constructor for class org.apache.beam.runners.fnexecution.state.StateRequestHandlers

StateRequestHandlers.BagUserStateHandler<K,V,W extends org.apache.beam.sdk.transforms.windowing.BoundedWindow> - Interface in org.apache.beam.runners.fnexecution.state

A handler for bag user state.

StateRequestHandlers.BagUserStateHandlerFactory<K,V,W extends org.apache.beam.sdk.transforms.windowing.BoundedWindow> - Interface in org.apache.beam.runners.fnexecution.state

A factory which constructs StateRequestHandlers.BagUserStateHandlers.

StateRequestHandlers.IterableSideInputHandler<V,W extends org.apache.beam.sdk.transforms.windowing.BoundedWindow> - Interface in org.apache.beam.runners.fnexecution.state

A handler for iterable side inputs.

StateRequestHandlers.MultimapSideInputHandler<K,V,W extends org.apache.beam.sdk.transforms.windowing.BoundedWindow> - Interface in org.apache.beam.runners.fnexecution.state

A handler for multimap side inputs.

StateRequestHandlers.SideInputHandler - Interface in org.apache.beam.runners.fnexecution.state

Marker interface that denotes some type of side input handler.

StateRequestHandlers.SideInputHandlerFactory - Interface in org.apache.beam.runners.fnexecution.state

A factory which constructs StateRequestHandlers.MultimapSideInputHandlers.

StaticGrpcProvisionService - Class in org.apache.beam.runners.fnexecution.provisioning

A provision service that returns a static response to all calls.

StaticRemoteEnvironment - Class in org.apache.beam.runners.fnexecution.environment

A RemoteEnvironment that connects to Dataflow runner harness.

StaticRemoteEnvironmentFactory - Class in org.apache.beam.runners.fnexecution.environment

An EnvironmentFactory that creates StaticRemoteEnvironment used by a runner harness that would like to use an existing InstructionRequestHandler.

StaticRemoteEnvironmentFactory.Provider - Class in org.apache.beam.runners.fnexecution.environment

Provider for StaticRemoteEnvironmentFactory.

stopProcess(String) - Method in class org.apache.beam.runners.fnexecution.environment.ProcessManager

Stops a previously started process identified by its unique id.

StreamingSideInputHandlerFactory - Class in org.apache.beam.runners.fnexecution.translation

StateRequestHandler that uses SideInputHandler to access the broadcast state that represents side inputs.

T

take(String, Duration) - Method in interface org.apache.beam.runners.fnexecution.control.ControlClientPool.Source

Retrieves the InstructionRequestHandler for the given worker id, blocking until available or the request times out.

timerId() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.TimerSpec

TimerReceiverFactory - Class in org.apache.beam.runners.fnexecution.control

A factory that passes timers to TimerReceiverFactory.timerDataConsumer.

TimerReceiverFactory(StageBundleFactory, BiConsumer<Timer<?>, TimerInternals.TimerData>, Coder) - Constructor for class org.apache.beam.runners.fnexecution.control.TimerReceiverFactory

TimerSpec() - Constructor for class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.TimerSpec

toProvisionInfo() - Method in class org.apache.beam.runners.fnexecution.provisioning.JobInfo

transformId() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.BagUserStateSpec

transformId() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.SideInputSpec

transformId() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.TimerSpec

U

unsupported() - Static method in interface org.apache.beam.runners.fnexecution.control.BundleSplitHandler

Returns a bundle split handler that throws on any split response.

unsupported() - Static method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandler

unsupported() - Static method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.BagUserStateHandlerFactory

Throws a UnsupportedOperationException on the first access.

unsupported() - Static method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.SideInputHandlerFactory

Throws a UnsupportedOperationException on the first access.

URL_ARTIFACT_URN - Static variable in class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService

userStateId() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.BagUserStateSpec

usingFnApiClient(InstructionRequestHandler, FnDataService) - Static method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient

Creates a client for a particular SDK harness.

V

valueCoder() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.BagUserStateSpec

valueCoder() - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.MultimapSideInputHandler

Returns the Coder to use for the elements of the resulting values iterable.

verifyDeterministic() - Method in class org.apache.beam.runners.fnexecution.wire.ByteStringCoder

W

windowCoder() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.BagUserStateSpec

windowCoder() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.SideInputSpec

WireCoders - Class in org.apache.beam.runners.fnexecution.wire

Helpers to construct coders for gRPC port reads and writes.

withIdGenerator(IdGenerator) - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient

workerStatus(StreamObserver<BeamFnApi.WorkerStatusRequest>) - Method in class org.apache.beam.runners.fnexecution.status.BeamWorkerStatusGrpcService

A B C D E F G H I J K L M N O P R S T U V W