org.apache.beam.sdk.io.jdbc

Class JdbcIO.WriteVoid<T>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<T>,org.apache.beam.sdk.values.PCollection<java.lang.Void>>

org.apache.beam.sdk.io.jdbc.JdbcIO.WriteVoid<T>

All Implemented Interfaces:

java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

Enclosing class:

JdbcIO

public abstract static class JdbcIO.WriteVoid<T>
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<T>,org.apache.beam.sdk.values.PCollection<java.lang.Void>>

A PTransform to write to a JDBC datasource.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name, resourceHints

Constructor Summary

Constructors

Constructor and Description
WriteVoid()

Method Summary

Modifier and Type	Method and Description
org.apache.beam.sdk.values.PCollection<java.lang.Void>	expand(org.apache.beam.sdk.values.PCollection<T> input)
JdbcIO.WriteVoid<T>	withBatchSize(long batchSize) Provide a maximum size in number of SQL statement for the batch.
JdbcIO.WriteVoid<T>	withDataSourceConfiguration(JdbcIO.DataSourceConfiguration config)
JdbcIO.WriteVoid<T>	withDataSourceProviderFn(org.apache.beam.sdk.transforms.SerializableFunction<java.lang.Void,javax.sql.DataSource> dataSourceProviderFn)
JdbcIO.WriteVoid<T>	withPreparedStatementSetter(JdbcIO.PreparedStatementSetter<T> setter)
JdbcIO.WriteVoid<T>	withRetryConfiguration(JdbcIO.RetryConfiguration retryConfiguration) When a SQL exception occurs, JdbcIO.Write uses this JdbcIO.RetryConfiguration to exponentially back off and retry the statements based on the JdbcIO.RetryConfiguration mentioned.
JdbcIO.WriteVoid<T>	withRetryStrategy(JdbcIO.RetryStrategy retryStrategy) When a SQL exception occurs, JdbcIO.Write uses this JdbcIO.RetryStrategy to determine if it will retry the statements.
JdbcIO.WriteVoid<T>	withStatement(java.lang.String statement)
JdbcIO.WriteVoid<T>	withStatement(org.apache.beam.sdk.options.ValueProvider<java.lang.String> statement)
JdbcIO.WriteVoid<T>	withTable(java.lang.String table)

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setResourceHints, toString, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

WriteVoid

public WriteVoid()

Method Detail

withDataSourceConfiguration

public JdbcIO.WriteVoid<T> withDataSourceConfiguration(JdbcIO.DataSourceConfiguration config)

withDataSourceProviderFn

public JdbcIO.WriteVoid<T> withDataSourceProviderFn(org.apache.beam.sdk.transforms.SerializableFunction<java.lang.Void,javax.sql.DataSource> dataSourceProviderFn)

withStatement

public JdbcIO.WriteVoid<T> withStatement(java.lang.String statement)

withStatement

public JdbcIO.WriteVoid<T> withStatement(org.apache.beam.sdk.options.ValueProvider<java.lang.String> statement)

withPreparedStatementSetter

public JdbcIO.WriteVoid<T> withPreparedStatementSetter(JdbcIO.PreparedStatementSetter<T> setter)

withBatchSize

public JdbcIO.WriteVoid<T> withBatchSize(long batchSize)

Provide a maximum size in number of SQL statement for the batch. Default is 1000.

Parameters:

batchSize - maximum batch size in number of statements

withRetryStrategy

public JdbcIO.WriteVoid<T> withRetryStrategy(JdbcIO.RetryStrategy retryStrategy)

When a SQL exception occurs, JdbcIO.Write uses this JdbcIO.RetryStrategy to determine if it will retry the statements. If JdbcIO.RetryStrategy.apply(SQLException) returns true, then JdbcIO.Write retries the statements.

withRetryConfiguration

public JdbcIO.WriteVoid<T> withRetryConfiguration(JdbcIO.RetryConfiguration retryConfiguration)

When a SQL exception occurs, JdbcIO.Write uses this JdbcIO.RetryConfiguration to exponentially back off and retry the statements based on the JdbcIO.RetryConfiguration mentioned.

Usage of RetryConfiguration -

 pipeline.apply(JdbcIO.<T>write())
    .withDataSourceConfiguration(...)
    .withRetryStrategy(...)
    .withRetryConfiguration(JdbcIO.RetryConfiguration.
        create(5, Duration.standardSeconds(5), Duration.standardSeconds(1))

 

maxDuration and initialDuration are Nullable

 pipeline.apply(JdbcIO.<T>write())
    .withDataSourceConfiguration(...)
    .withRetryStrategy(...)
    .withRetryConfiguration(JdbcIO.RetryConfiguration.
        create(5, null, null)

 

withTable

public JdbcIO.WriteVoid<T> withTable(java.lang.String table)

expand

public org.apache.beam.sdk.values.PCollection<java.lang.Void> expand(org.apache.beam.sdk.values.PCollection<T> input)

Specified by:

expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<T>,org.apache.beam.sdk.values.PCollection<java.lang.Void>>