org.apache.beam.sdk.options

Interface GcsOptions

All Superinterfaces:

ApplicationNameOptions, GcpOptions, GoogleApiDebugOptions, HasDisplayData, PipelineOptions

public interface GcsOptions
extends ApplicationNameOptions, GcpOptions, PipelineOptions

Options used to configure Google Cloud Storage.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	GcsOptions.ExecutorServiceFactory Returns the default ExecutorService to use within the Dataflow SDK.

Nested classes/interfaces inherited from interface org.apache.beam.sdk.options.GcpOptions

GcpOptions.CredentialDirFactory, GcpOptions.DefaultProjectFactory, GcpOptions.GcpTempLocationFactory, GcpOptions.GcpUserCredentialsFactory

Nested classes/interfaces inherited from interface org.apache.beam.sdk.options.GoogleApiDebugOptions

GoogleApiDebugOptions.GoogleApiTracer

Nested classes/interfaces inherited from interface org.apache.beam.sdk.options.PipelineOptions

PipelineOptions.CheckEnabled, PipelineOptions.DirectRunner

Method Summary

Modifier and Type	Method and Description
ExecutorService	getExecutorService() The ExecutorService instance to use to create threads, can be overridden to specify an ExecutorService that is compatible with the users environment.
String	getGcsEndpoint() GCS endpoint to use.
Integer	getGcsUploadBufferSizeBytes() The buffer size (in bytes) to use when uploading files to GCS.
org.apache.beam.sdk.util.GcsUtil	getGcsUtil() The GcsUtil instance that should be used to communicate with Google Cloud Storage.
void	setExecutorService(ExecutorService value)
void	setGcsEndpoint(String value)
void	setGcsUploadBufferSizeBytes(Integer bytes)
void	setGcsUtil(org.apache.beam.sdk.util.GcsUtil value)

Methods inherited from interface org.apache.beam.sdk.options.ApplicationNameOptions

getAppName, setAppName

Methods inherited from interface org.apache.beam.sdk.options.GcpOptions

getAuthorizationServerEncodedUrl, getCredentialDir, getCredentialFactoryClass, getCredentialId, getGcpCredential, getGcpTempLocation, getProject, getSecretsFile, getServiceAccountKeyfile, getServiceAccountName, getTokenServerUrl, setAuthorizationServerEncodedUrl, setCredentialDir, setCredentialFactoryClass, setCredentialId, setGcpCredential, setGcpTempLocation, setProject, setSecretsFile, setServiceAccountKeyfile, setServiceAccountName, setTokenServerUrl

Methods inherited from interface org.apache.beam.sdk.options.GoogleApiDebugOptions

getGoogleApiTrace, setGoogleApiTrace

Methods inherited from interface org.apache.beam.sdk.options.PipelineOptions

as, cloneAs, getRunner, getStableUniqueNames, getTempLocation, setRunner, setStableUniqueNames, setTempLocation

Methods inherited from interface org.apache.beam.sdk.transforms.display.HasDisplayData

populateDisplayData

Method Detail

getGcsUtil

@Default.InstanceFactory(value=org.apache.beam.sdk.util.GcsUtil.GcsUtilFactory.class)
 @Hidden
org.apache.beam.sdk.util.GcsUtil getGcsUtil()

The GcsUtil instance that should be used to communicate with Google Cloud Storage.

setGcsUtil

void setGcsUtil(org.apache.beam.sdk.util.GcsUtil value)

getExecutorService

@Default.InstanceFactory(value=GcsOptions.ExecutorServiceFactory.class)
 @Hidden
ExecutorService getExecutorService()

The ExecutorService instance to use to create threads, can be overridden to specify an ExecutorService that is compatible with the users environment. If unset, the default is to create an ExecutorService with an unbounded number of threads; this is compatible with Google AppEngine.

setExecutorService

void setExecutorService(ExecutorService value)

getGcsEndpoint

@Hidden
String getGcsEndpoint()

GCS endpoint to use. If unspecified, uses the default endpoint.

setGcsEndpoint

void setGcsEndpoint(String value)

getGcsUploadBufferSizeBytes

Integer getGcsUploadBufferSizeBytes()

The buffer size (in bytes) to use when uploading files to GCS. Please see the documentation for AbstractGoogleAsyncWriteChannel.setUploadBufferSize(int) for more information on the restrictions and performance implications of this value.

setGcsUploadBufferSizeBytes

void setGcsUploadBufferSizeBytes(Integer bytes)