org.apache.beam.sdk.extensions.sql.meta.provider.pubsub

Class PubsubMessageToRow

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.io.gcp.pubsub.PubsubMessage>,org.apache.beam.sdk.values.PCollectionTuple>

org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubMessageToRow

All Implemented Interfaces:

java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

@Internal
 @Experimental
public abstract class PubsubMessageToRow
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.io.gcp.pubsub.PubsubMessage>,org.apache.beam.sdk.values.PCollectionTuple>
implements java.io.Serializable

Read side converter for PubsubMessage with JSON payload.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name

Constructor Summary

Constructors

Constructor and Description
PubsubMessageToRow()

Method Summary

Modifier and Type	Method and Description
static org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubMessageToRow.Builder	builder()
org.apache.beam.sdk.values.PCollectionTuple	expand(org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.io.gcp.pubsub.PubsubMessage> input)
abstract org.apache.beam.sdk.schemas.Schema	messageSchema() Schema of the Pubsub message.
abstract boolean	useDlq()
abstract boolean	useFlatSchema()

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, populateDisplayData, toString, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

PubsubMessageToRow

public PubsubMessageToRow()

Method Detail

messageSchema

public abstract org.apache.beam.sdk.schemas.Schema messageSchema()

Schema of the Pubsub message.

Required to have at least 'event_timestamp' field of type Schema.FieldType#DATETIME.

If useFlatSchema() is set every other field is assumed to be part of the payload. Otherwise, the schema must contain exactly:

• 'attributes' of type MAP<VARCHAR,VARCHAR>
• 'payload' of type ROW<...>

Only UTF-8 JSON objects are supported.

useDlq

public abstract boolean useDlq()

useFlatSchema

public abstract boolean useFlatSchema()

builder

public static org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubMessageToRow.Builder builder()

expand

public org.apache.beam.sdk.values.PCollectionTuple expand(org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.io.gcp.pubsub.PubsubMessage> input)

Specified by:

expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.io.gcp.pubsub.PubsubMessage>,org.apache.beam.sdk.values.PCollectionTuple>