net.snowflake.ingest.streaming

Interface SnowflakeStreamingIngestChannel

public interface SnowflakeStreamingIngestChannel

A logical partition that represents a connection to a single Snowflake table, data will be ingested into the channel, and then flush to Snowflake table periodically in the background. Note that only one client (or thread) could write to a channel at a given time, so if there are multiple clients (or multiple threads in the same client) try to ingest using the same channel, the latest client (or thread) that opens the channel will win and all the other opened channels will be invalid

Method Summary

Modifier and Type	Method and Description
CompletableFuture<Void>	close() Close the channel, this function will make sure all the data in this channel is committed
String	getDBName() Get the database name
String	getFullyQualifiedName() Get the fully qualified channel name
String	getFullyQualifiedTableName() Get the fully qualified table name that the channel belongs to
String	getLatestCommittedOffsetToken() Get the latest committed offset token from Snowflake
String	getName() Get the name of the channel
String	getSchemaName() Get the schema name
String	getTableName() Get the table name
InsertValidationResponse	insertRow(Map<String,Object> row, String offsetToken) Insert one row into the channel, the row is represented using Map where the key is column name and the value is a row of data
InsertValidationResponse	insertRows(Iterable<Map<String,Object>> rows, String offsetToken) Insert a batch of rows into the channel, each row is represented using Map where the key is column name and the value is a row of data
boolean	isClosed()
boolean	isValid()

Method Detail

getFullyQualifiedName

String getFullyQualifiedName()

Get the fully qualified channel name

Returns:

fully qualified name of the channel, in the format of dbName.schemaName.tableName.channelName

getName

String getName()

Get the name of the channel

Returns:

name of the channel

getDBName

String getDBName()

Get the database name

Returns:

name of the database

getSchemaName

String getSchemaName()

Get the schema name

Returns:

name of the schema

getTableName

String getTableName()

Get the table name

Returns:

name of the table

getFullyQualifiedTableName

String getFullyQualifiedTableName()

Get the fully qualified table name that the channel belongs to

Returns:

fully qualified table name, in the format of dbName.schemaName.tableName

isValid

boolean isValid()

Returns:

a boolean which indicates whether the channel is valid

isClosed

boolean isClosed()

Returns:

a boolean which indicates whether the channel is closed

close

CompletableFuture<Void> close()

Close the channel, this function will make sure all the data in this channel is committed

Returns:

a completable future which will be completed when the channel is closed

insertRow

InsertValidationResponse insertRow(Map<String,Object> row,
                                   @Nullable
                                   String offsetToken)

Insert one row into the channel, the row is represented using Map where the key is column name and the value is a row of data

Parameters:

row - object data to write

offsetToken - offset of given row, used for replay in case of failures. It could be null if you don't plan on replaying or can't replay

Returns:

insert response that possibly contains errors because of insertion failures

insertRows

InsertValidationResponse insertRows(Iterable<Map<String,Object>> rows,
                                    @Nullable
                                    String offsetToken)

Insert a batch of rows into the channel, each row is represented using Map where the key is column name and the value is a row of data

Parameters:

rows - object data to write

offsetToken - offset of last row in the row-set, used for replay in case of failures, It could be null if you don't plan on replaying or can't replay

Returns:

insert response that possibly contains errors because of insertion failures

getLatestCommittedOffsetToken

String getLatestCommittedOffsetToken()

Get the latest committed offset token from Snowflake

Returns:

the latest committed offset token