org.apache.beam.sdk.options

Interface GcpOptions

All Superinterfaces:

GoogleApiDebugOptions, HasDisplayData, PipelineOptions

All Known Subinterfaces:

BigQueryOptions, GcsOptions, PubsubOptions, StreamingOptions

public interface GcpOptions
extends GoogleApiDebugOptions, PipelineOptions

Options used to configure Google Cloud Platform project and credentials.

These options configure which of the following three different mechanisms for obtaining a credential are used:

1. It can fetch the application default credentials.
2. The user can specify a client secrets file and go through the OAuth2 webflow. The credential will then be cached in the user's home directory for reuse.
3. The user can specify a file containing a service account private key along with the service account name.

The default mechanism is to use the application default credentials. The other options can be used by setting the corresponding properties.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	GcpOptions.CredentialDirFactory Returns the default credential directory of ${user.home}/.store/data-flow.
static class	GcpOptions.DefaultProjectFactory Attempts to infer the default project based upon the environment this application is executing within.
static class	GcpOptions.GcpTempLocationFactory Returns PipelineOptions.getTempLocation() as the default GCP temp location.
static class	GcpOptions.GcpUserCredentialsFactory Attempts to load the GCP credentials.

Nested classes/interfaces inherited from interface org.apache.beam.sdk.options.GoogleApiDebugOptions

GoogleApiDebugOptions.GoogleApiTracer

Nested classes/interfaces inherited from interface org.apache.beam.sdk.options.PipelineOptions

PipelineOptions.CheckEnabled, PipelineOptions.DirectRunner

Method Summary

Modifier and Type	Method and Description
String	getAuthorizationServerEncodedUrl() The authorization server URL to use for OAuth 2 authentication.
String	getCredentialDir() Directory for storing dataflow credentials after execution of the OAuth 2 webflow.
Class<? extends org.apache.beam.sdk.util.CredentialFactory>	getCredentialFactoryClass() The class of the credential factory that should be created and used to create credentials.
String	getCredentialId() This option controls which credential store to use when creating the credentials using the OAuth 2 webflow.
Credential	getGcpCredential() The credential instance that should be used to authenticate against GCP services.
String	getGcpTempLocation() A GCS path for storing temporary files in GCP.
String	getProject() Project id to use when launching jobs.
String	getSecretsFile() This option controls which file to use when attempting to create the credentials using the OAuth 2 webflow.
String	getServiceAccountKeyfile() This option controls which file to use when attempting to create the credentials using the service account method.
String	getServiceAccountName() This option controls which service account to use when attempting to create the credentials using the service account method.
String	getTokenServerUrl() The token server URL to use for OAuth 2 authentication.
void	setAuthorizationServerEncodedUrl(String value)
void	setCredentialDir(String value)
void	setCredentialFactoryClass(Class<? extends org.apache.beam.sdk.util.CredentialFactory> credentialFactoryClass)
void	setCredentialId(String value)
void	setGcpCredential(Credential value)
void	setGcpTempLocation(String value)
void	setProject(String value)
void	setSecretsFile(String value)
void	setServiceAccountKeyfile(String value)
void	setServiceAccountName(String value)
void	setTokenServerUrl(String value)

Methods inherited from interface org.apache.beam.sdk.options.GoogleApiDebugOptions

getGoogleApiTrace, setGoogleApiTrace

Methods inherited from interface org.apache.beam.sdk.options.PipelineOptions

as, cloneAs, getRunner, getStableUniqueNames, getTempLocation, setRunner, setStableUniqueNames, setTempLocation

Methods inherited from interface org.apache.beam.sdk.transforms.display.HasDisplayData

populateDisplayData

Method Detail

getProject

@Default.InstanceFactory(value=GcpOptions.DefaultProjectFactory.class)
String getProject()

Project id to use when launching jobs.

setProject

void setProject(String value)

getServiceAccountKeyfile

String getServiceAccountKeyfile()

This option controls which file to use when attempting to create the credentials using the service account method.

This option if specified, needs be combined with the serviceAccountName.

setServiceAccountKeyfile

void setServiceAccountKeyfile(String value)

getServiceAccountName

String getServiceAccountName()

This option controls which service account to use when attempting to create the credentials using the service account method.

This option if specified, needs be combined with the serviceAccountKeyfile.

setServiceAccountName

void setServiceAccountName(String value)

getSecretsFile

String getSecretsFile()

This option controls which file to use when attempting to create the credentials using the OAuth 2 webflow. After the OAuth2 webflow, the credentials will be stored within credentialDir.

setSecretsFile

void setSecretsFile(String value)

getCredentialId

@Default.String(value="cloud_dataflow")
String getCredentialId()

This option controls which credential store to use when creating the credentials using the OAuth 2 webflow.

setCredentialId

void setCredentialId(String value)

getCredentialDir

@Default.InstanceFactory(value=GcpOptions.CredentialDirFactory.class)
String getCredentialDir()

Directory for storing dataflow credentials after execution of the OAuth 2 webflow. Defaults to using the $HOME/.store/data-flow directory.

setCredentialDir

void setCredentialDir(String value)

getCredentialFactoryClass

@Default.Class(value=org.apache.beam.sdk.util.GcpCredentialFactory.class)
Class<? extends org.apache.beam.sdk.util.CredentialFactory> getCredentialFactoryClass()

The class of the credential factory that should be created and used to create credentials. If gcpCredential has not been set explicitly, an instance of this class will be constructed and used as a credential factory.

setCredentialFactoryClass

void setCredentialFactoryClass(Class<? extends org.apache.beam.sdk.util.CredentialFactory> credentialFactoryClass)

getGcpCredential

@Default.InstanceFactory(value=GcpOptions.GcpUserCredentialsFactory.class)
 @Hidden
Credential getGcpCredential()

The credential instance that should be used to authenticate against GCP services. If no credential has been set explicitly, the default is to use the instance factory that constructs a credential based upon the currently set credentialFactoryClass.

setGcpCredential

void setGcpCredential(Credential value)

getTokenServerUrl

@Default.String(value="https://accounts.google.com/o/oauth2/token")
 @Hidden
String getTokenServerUrl()

The token server URL to use for OAuth 2 authentication. Normally, the default is sufficient, but some specialized use cases may want to override this value.

setTokenServerUrl

void setTokenServerUrl(String value)

getAuthorizationServerEncodedUrl

@Default.String(value="https://accounts.google.com/o/oauth2/auth")
 @Hidden
String getAuthorizationServerEncodedUrl()

The authorization server URL to use for OAuth 2 authentication. Normally, the default is sufficient, but some specialized use cases may want to override this value.

setAuthorizationServerEncodedUrl

void setAuthorizationServerEncodedUrl(String value)

getGcpTempLocation

@Default.InstanceFactory(value=GcpOptions.GcpTempLocationFactory.class)
String getGcpTempLocation()

A GCS path for storing temporary files in GCP.

Its default to PipelineOptions.getTempLocation().

setGcpTempLocation

void setGcpTempLocation(String value)