cn.tenmg.clink.operator.job.generator

类 AbstractDataSyncJobGenerator

java.lang.Object

cn.tenmg.clink.operator.job.generator.AbstractDataSyncJobGenerator

所有已实现的接口:

DataSyncJobGenerator

直接已知子类:

FromSourceFactoryDataSyncJobGenerator, SingleTableDataSyncJobGenerator

public abstract class AbstractDataSyncJobGenerator
extends Object
implements DataSyncJobGenerator

抽象数据同步作业生成器

从以下版本开始:

1.6.0

作者:

June wjzhao@aliyun.com

嵌套类概要

嵌套类

限定符和类型	类和说明
protected static class	AbstractDataSyncJobGenerator.ColumnConvert 列转换配置参数
protected static class	AbstractDataSyncJobGenerator.DefaultPrimaryKeysCollector 默认主键收集器
protected static interface	AbstractDataSyncJobGenerator.PrimaryKeysCollector 主键收集器
protected static class	AbstractDataSyncJobGenerator.Strategy 同步策略

字段概要

字段

限定符和类型	字段和说明
protected static String	AUTO_COLUMNS
protected static String	AUTO_COLUMNS_SPLITER
protected static String	COLUMN_NAME
protected static Map<String,AbstractDataSyncJobGenerator.ColumnConvert>	columnConverts
protected static String	GROUP_ID_KEY
protected org.slf4j.Logger	log
protected static Pattern	METADATA_PATTERN
protected static String	SCRIPT_KEY_SUFFIX
protected static String	STRATEGY_KEY_SUFFIX
protected static boolean	TO_LOWERCASE
protected static String	TOPIC_KEY
protected static String	TYPE_KEY_PREFIX

构造器概要

构造器

构造器和说明
AbstractDataSyncJobGenerator()

方法概要

限定符和类型	方法和说明
protected static void	addSmartLoadColumns(String connector, List<Column> columns, Map<String,MetaDataGetter.TableMetaData.ColumnType> columnTypes, Map<String,Object> params, Map<String,String> autoColumnses)
protected static void	collationCustom(List<Column> columns, Map<String,Object> params, Map<String,String> autoColumnses)
protected static void	collationPartlyCustom(String connector, List<Column> columns, Map<String,Object> params, Map<String,MetaDataGetter.TableMetaData.ColumnType> columnsTypes, Map<String,String> autoColumnses)
Object	generate(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env, DataSync dataSync, Map<String,String> sourceDataSource, Map<String,String> sinkDataSource, Map<String,Object> params) 生成数据同步任务
protected static String	getDefaultAutoColumns()
protected static String	getDefaultFromType(String columnName)
protected static String	getDefaultScript(String columnName)
protected static String	getDefaultStrategy(String columnName)
protected static String	getDefaultToType(String columnName)
protected static String	insertSQL(String toTable, String fromTable, List<Column> columns, Map<String,Object> params)
protected static String	sinkTableSQL(Map<String,String> dataSource, String table, List<Column> columns, Set<String> primaryKeys, Map<String,Object> params) 生成创建数据汇表的 Flink SQL
protected static String	sinkTableSQL(Map<String,String> sinkDataSource, String table, List<Column> columns, Set<String> primaryKeys, Map<String,Object> params, AbstractDataSyncJobGenerator.PrimaryKeysCollector collector) 生成创建数据汇表的 Flink SQL
protected static Map<String,String>	toMap(boolean toLowercase, String... strings)

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

TOPIC_KEY

protected static final String TOPIC_KEY

另请参阅:

常量字段值

GROUP_ID_KEY

protected static final String GROUP_ID_KEY

另请参阅:

常量字段值

AUTO_COLUMNS

protected static final String AUTO_COLUMNS

另请参阅:

常量字段值

AUTO_COLUMNS_SPLITER

protected static final String AUTO_COLUMNS_SPLITER

另请参阅:

常量字段值

TYPE_KEY_PREFIX

protected static final String TYPE_KEY_PREFIX

另请参阅:

常量字段值

SCRIPT_KEY_SUFFIX

protected static final String SCRIPT_KEY_SUFFIX

另请参阅:

常量字段值

STRATEGY_KEY_SUFFIX

protected static final String STRATEGY_KEY_SUFFIX

另请参阅:

常量字段值

COLUMN_NAME

protected static final String COLUMN_NAME

另请参阅:

常量字段值

TO_LOWERCASE

protected static final boolean TO_LOWERCASE

columnConverts

protected static final Map<String,AbstractDataSyncJobGenerator.ColumnConvert> columnConverts

METADATA_PATTERN

protected static final Pattern METADATA_PATTERN

log

protected org.slf4j.Logger log

构造器详细资料

AbstractDataSyncJobGenerator

public AbstractDataSyncJobGenerator()

方法详细资料

generate

public Object generate(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env,
                       DataSync dataSync,
                       Map<String,String> sourceDataSource,
                       Map<String,String> sinkDataSource,
                       Map<String,Object> params)
                throws Exception

从接口复制的说明: DataSyncJobGenerator

生成数据同步任务

指定者:

generate 在接口中 DataSyncJobGenerator

参数:

env - 流执行环境

dataSync - 同步配置对象

sourceDataSource - 源数据源

sinkDataSource - 汇数据源

params - 参数查找表

返回:

生成数据同步任务的结果

抛出:

Exception - 发生异常

sinkTableSQL

protected static String sinkTableSQL(Map<String,String> dataSource,
                                     String table,
                                     List<Column> columns,
                                     Set<String> primaryKeys,
                                     Map<String,Object> params)
                              throws IOException

生成创建数据汇表的 Flink SQL

参数:

dataSource - 数据源

table - 表名

columns - 列

primaryKeys - 主键

params - 运行参数

返回:

创建数据汇表的 Flink SQL

抛出:

IOException - I/O 异常

sinkTableSQL

protected static String sinkTableSQL(Map<String,String> sinkDataSource,
                                     String table,
                                     List<Column> columns,
                                     Set<String> primaryKeys,
                                     Map<String,Object> params,
                                     AbstractDataSyncJobGenerator.PrimaryKeysCollector collector)
                              throws IOException

生成创建数据汇表的 Flink SQL

参数:

sinkDataSource - 汇数据源

table - 表名

columns - 列

primaryKeys - 主键

params - 运行参数

collector - 主键收集器

返回:

创建数据汇表的 Flink SQL

抛出:

IOException - I/O 异常

addSmartLoadColumns

protected static void addSmartLoadColumns(String connector,
                                          List<Column> columns,
                                          Map<String,MetaDataGetter.TableMetaData.ColumnType> columnTypes,
                                          Map<String,Object> params,
                                          Map<String,String> autoColumnses)

collationPartlyCustom

protected static void collationPartlyCustom(String connector,
                                            List<Column> columns,
                                            Map<String,Object> params,
                                            Map<String,MetaDataGetter.TableMetaData.ColumnType> columnsTypes,
                                            Map<String,String> autoColumnses)

collationCustom

protected static void collationCustom(List<Column> columns,
                                      Map<String,Object> params,
                                      Map<String,String> autoColumnses)

toMap

protected static final Map<String,String> toMap(boolean toLowercase,
                                                String... strings)

insertSQL

protected static String insertSQL(String toTable,
                                  String fromTable,
                                  List<Column> columns,
                                  Map<String,Object> params)

getDefaultAutoColumns

protected static String getDefaultAutoColumns()

getDefaultStrategy

protected static String getDefaultStrategy(String columnName)

getDefaultFromType

protected static String getDefaultFromType(String columnName)

getDefaultScript

protected static String getDefaultScript(String columnName)

getDefaultToType

protected static String getDefaultToType(String columnName)