org.pentaho.di.trans.steps.mysqlbulkloader

类 MySQLBulkLoaderMeta

java.lang.Object

org.pentaho.di.trans.step.BaseStepMeta

org.pentaho.di.trans.steps.mysqlbulkloader.MySQLBulkLoaderMeta

所有已实现的接口:

Cloneable, ProvidesDatabaseConnectionInformation, StepAttributesInterface, StepMetaInterface

public class MySQLBulkLoaderMeta
extends BaseStepMeta
implements StepMetaInterface, ProvidesDatabaseConnectionInformation

Here are the steps that we need to take to make streaming loading possible for MySQL:

The following steps are carried out by the step at runtime:

- create a unique FIFO file (using mkfifo, LINUX ONLY FOLKS!)
- Create a target table using standard Kettle SQL generation
- Execute the LOAD DATA SQL Command to bulk load in a separate SQL thread in the background:
- Write to the FIFO file
- At the end, close the output stream to the FIFO file
* At the end, remove the FIFO file
Created on 24-oct-2007

作者:

Matt Casters

嵌套类概要

嵌套类

限定符和类型	类和说明
static class	MySQLBulkLoaderMeta.FieldFormatTypeConverter

字段概要

字段

限定符和类型	字段和说明
static int	FIELD_FORMAT_TYPE_DATE
static int	FIELD_FORMAT_TYPE_NUMBER
static int	FIELD_FORMAT_TYPE_OK
static int	FIELD_FORMAT_TYPE_STRING_ESCAPE
static int	FIELD_FORMAT_TYPE_TIMESTAMP

从类继承的字段 org.pentaho.di.trans.step.BaseStepMeta

attributes, databases, log, loggingObject, parentStepMeta, repository, STEP_ATTRIBUTES_FILE

构造器概要

构造器

构造器和说明
MySQLBulkLoaderMeta()

方法概要

限定符和类型	方法和说明
void	afterInjectionSynchronization() If we use injection we can have different arrays lengths.
void	allocate(int nrvalues)
void	analyseImpact(List<DatabaseImpact> impact, TransMeta transMeta, StepMeta stepMeta, org.pentaho.di.core.row.RowMetaInterface prev, String[] input, String[] output, org.pentaho.di.core.row.RowMetaInterface info, Repository repository, org.pentaho.metastore.api.IMetaStore metaStore) Each step must be able to report on the impact it has on a database, table field, etc.
void	check(List<org.pentaho.di.core.CheckResultInterface> remarks, TransMeta transMeta, StepMeta stepMeta, org.pentaho.di.core.row.RowMetaInterface prev, String[] input, String[] output, org.pentaho.di.core.row.RowMetaInterface info, org.pentaho.di.core.variables.VariableSpace space, Repository repository, org.pentaho.metastore.api.IMetaStore metaStore) Checks the settings of this step and puts the findings in a remarks List.
Object	clone() Make an exact copy of this step, make sure to explicitly copy Collections etc.
String	getBulkSize()
org.pentaho.di.core.database.DatabaseMeta	getDatabaseMeta() Returns the database meta for this step
String	getDelimiter()
String	getEnclosure()
String	getEncoding()
String	getEscapeChar()
int[]	getFieldFormatType()
static int	getFieldFormatType(String codeOrDescription)
static String	getFieldFormatTypeCode(int type)
static String[]	getFieldFormatTypeCodes()
static String	getFieldFormatTypeDescription(int type)
static String[]	getFieldFormatTypeDescriptions()
void	getFields(org.pentaho.di.core.row.RowMetaInterface rowMeta, String origin, org.pentaho.di.core.row.RowMetaInterface[] info, StepMeta nextStep, org.pentaho.di.core.variables.VariableSpace space, Repository repository, org.pentaho.metastore.api.IMetaStore metaStore) Gets the fields.
String[]	getFieldStream()
String[]	getFieldTable()
String	getFifoFileName()
String	getMissingDatabaseConnectionInformationMessage() Provides a way for this object to return a custom message when database connection information is incomplete or missing.
org.pentaho.di.core.row.RowMetaInterface	getRequiredFields(org.pentaho.di.core.variables.VariableSpace space) The natural way of data flow in a transformation is source-to-target.
String	getSchemaName() Returns the schema name for this step.
org.pentaho.di.core.SQLStatement	getSQLStatements(TransMeta transMeta, StepMeta stepMeta, org.pentaho.di.core.row.RowMetaInterface prev, Repository repository, org.pentaho.metastore.api.IMetaStore metaStore) Standard method to return an SQLStatement object with SQL statements that the step needs in order to work correctly.
StepInterface	getStep(StepMeta stepMeta, StepDataInterface stepDataInterface, int cnr, TransMeta transMeta, Trans trans) Get the executing step, needed by Trans to launch a step.
StepDataInterface	getStepData() Get a new instance of the appropriate data class.
String	getTableName() Returns the table name for this step
org.pentaho.di.core.database.DatabaseMeta[]	getUsedDatabaseConnections() This method returns all the database connections that are used by the step.
String	getXML() Produces the XML string that describes this step's information.
boolean	isIgnoringErrors()
boolean	isLocalFile()
boolean	isReplacingData()
void	loadXML(Node stepnode, List<org.pentaho.di.core.database.DatabaseMeta> databases, org.pentaho.metastore.api.IMetaStore metaStore) Load the values for this step from an XML Node
void	readRep(Repository rep, org.pentaho.metastore.api.IMetaStore metaStore, org.pentaho.di.repository.ObjectId id_step, List<org.pentaho.di.core.database.DatabaseMeta> databases) Read the steps information from a Kettle repository
void	saveRep(Repository rep, org.pentaho.metastore.api.IMetaStore metaStore, org.pentaho.di.repository.ObjectId id_transformation, org.pentaho.di.repository.ObjectId id_step) Save the steps data into a Kettle repository
void	setBulkSize(String bulkSize)
void	setDatabaseMeta(org.pentaho.di.core.database.DatabaseMeta database)
void	setDefault() Set default values
void	setDelimiter(String delimiter)
void	setEnclosure(String enclosure)
void	setEncoding(String encoding)
void	setEscapeChar(String escapeChar)
void	setFieldFormatType(int[] fieldFormatType)
void	setFieldStream(String[] fieldStream)
void	setFieldTable(String[] fieldTable)
void	setFifoFileName(String fifoFileName)
void	setIgnoringErrors(boolean ignoringErrors)
void	setLocalFile(boolean localFile)
void	setReplacingData(boolean replacingData)
void	setSchemaName(String schemaName)
void	setTableName(String tableName)

从类继承的方法 org.pentaho.di.trans.step.BaseStepMeta

analyseImpact, cancelQueries, check, check, createEntry, excludeFromCopyDistributeVerification, excludeFromRowLayoutVerification, exportResources, exportResources, extractStepMetadataEntries, findAttribute, findParent, findParentEntry, getActiveReferencedObjectDescription, getDescription, getDialogClassName, getFields, getLog, getLogChannelId, getName, getObjectCopy, getObjectId, getObjectRevision, getObjectType, getOptionalStreams, getParent, getParentStepMeta, getReferencedObjectDescriptions, getRepCode, getRepositoryDirectory, getRequiredFields, getResourceDependencies, getSQLStatements, getStepInjectionMetadataEntries, getStepIOMeta, getStepIOMeta, getStepMetaInjectionInterface, getSupportedTransformationTypes, getTableFields, getTooltip, getUsedArguments, getUsedLibraries, getXmlCode, handleStreamSelection, hasChanged, hasRepositoryReferences, isBasic, isDebug, isDetailed, isReferencedObjectEnabled, isRowLevel, loadReferencedObject, loadReferencedObject, loadStepAttributes, loadXML, loadXML, logBasic, logBasic, logDebug, logDebug, logDetailed, logDetailed, logError, logError, logError, logMinimal, logMinimal, logRowlevel, logRowlevel, lookupRepositoryReferences, readRep, readRep, resetStepIoMeta, saveRep, searchInfoAndTargetSteps, setChanged, setChanged, setParentStepMeta, setStepIOMeta, supportsErrorHandling

从类继承的方法 java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.pentaho.di.trans.step.StepMetaInterface

analyseImpact, cancelQueries, check, cleanAfterHopFromRemove, cleanAfterHopFromRemove, cleanAfterHopToRemove, excludeFromCopyDistributeVerification, excludeFromRowLayoutVerification, exportResources, exportResources, extractStepMetadataEntries, fetchTransMeta, getActiveReferencedObjectDescription, getDialogClassName, getFields, getOptionalStreams, getParentStepMeta, getReferencedObjectDescriptions, getResourceDependencies, getSQLStatements, getStepIOMeta, getStepMetaInjectionInterface, getSupportedTransformationTypes, getTableFields, getUsedArguments, getUsedLibraries, handleStreamSelection, hasChanged, hasRepositoryReferences, isReferencedObjectEnabled, loadReferencedObject, loadXML, lookupRepositoryReferences, passDataToServletOutput, readRep, resetStepIoMeta, saveRep, searchInfoAndTargetSteps, setChanged, setParentStepMeta, supportsErrorHandling

字段详细资料

FIELD_FORMAT_TYPE_OK

public static final int FIELD_FORMAT_TYPE_OK

另请参阅:

常量字段值

FIELD_FORMAT_TYPE_DATE

public static final int FIELD_FORMAT_TYPE_DATE

另请参阅:

常量字段值

FIELD_FORMAT_TYPE_TIMESTAMP

public static final int FIELD_FORMAT_TYPE_TIMESTAMP

另请参阅:

常量字段值

FIELD_FORMAT_TYPE_NUMBER

public static final int FIELD_FORMAT_TYPE_NUMBER

另请参阅:

常量字段值

FIELD_FORMAT_TYPE_STRING_ESCAPE

public static final int FIELD_FORMAT_TYPE_STRING_ESCAPE

另请参阅:

常量字段值

构造器详细资料

MySQLBulkLoaderMeta

public MySQLBulkLoaderMeta()

方法详细资料

getDatabaseMeta

public org.pentaho.di.core.database.DatabaseMeta getDatabaseMeta()

从接口复制的说明: ProvidesDatabaseConnectionInformation

Returns the database meta for this step

指定者:

getDatabaseMeta 在接口中 ProvidesDatabaseConnectionInformation

返回:

Returns the database.

setDatabaseMeta

public void setDatabaseMeta(org.pentaho.di.core.database.DatabaseMeta database)

参数:

database - The database to set.

getTableName

public String getTableName()

从接口复制的说明: ProvidesDatabaseConnectionInformation

Returns the table name for this step

指定者:

getTableName 在接口中 ProvidesDatabaseConnectionInformation

返回:

Returns the tableName.

setTableName

public void setTableName(String tableName)

参数:

tableName - The tableName to set.

getFieldTable

public String[] getFieldTable()

返回:

Returns the fieldTable.

setFieldTable

public void setFieldTable(String[] fieldTable)

参数:

fieldTable - The fieldTable to set.

getFieldStream

public String[] getFieldStream()

返回:

Returns the fieldStream.

setFieldStream

public void setFieldStream(String[] fieldStream)

参数:

fieldStream - The fieldStream to set.

loadXML

public void loadXML(Node stepnode,
                    List<org.pentaho.di.core.database.DatabaseMeta> databases,
                    org.pentaho.metastore.api.IMetaStore metaStore)
             throws org.pentaho.di.core.exception.KettleXMLException

从接口复制的说明: StepMetaInterface

Load the values for this step from an XML Node

指定者:

loadXML 在接口中 StepMetaInterface

覆盖:

loadXML 在类中 BaseStepMeta

参数:

stepnode - the Node to get the info from

databases - The available list of databases to reference to

metaStore - the metastore to optionally load external reference metadata from

抛出:

org.pentaho.di.core.exception.KettleXMLException - When an unexpected XML error occurred. (malformed etc.)

allocate

public void allocate(int nrvalues)

clone

public Object clone()

从接口复制的说明: StepMetaInterface

Make an exact copy of this step, make sure to explicitly copy Collections etc.

指定者:

clone 在接口中 StepMetaInterface

覆盖:

clone 在类中 BaseStepMeta

返回:

an exact copy of this step

setDefault

public void setDefault()

从接口复制的说明: StepMetaInterface

Set default values

指定者:

setDefault 在接口中 StepMetaInterface

getXML

public String getXML()

从类复制的说明: BaseStepMeta

Produces the XML string that describes this step's information.

指定者:

getXML 在接口中 StepMetaInterface

覆盖:

getXML 在类中 BaseStepMeta

返回:

String containing the XML describing this step.

readRep

public void readRep(Repository rep,
                    org.pentaho.metastore.api.IMetaStore metaStore,
                    org.pentaho.di.repository.ObjectId id_step,
                    List<org.pentaho.di.core.database.DatabaseMeta> databases)
             throws org.pentaho.di.core.exception.KettleException

从接口复制的说明: StepMetaInterface

Read the steps information from a Kettle repository

指定者:

readRep 在接口中 StepMetaInterface

覆盖:

readRep 在类中 BaseStepMeta

参数:

rep - The repository to read from

metaStore - The MetaStore to read external information from

id_step - The step ID

databases - The databases to reference

抛出:

org.pentaho.di.core.exception.KettleException - When an unexpected error occurred (database, network, etc)

saveRep

public void saveRep(Repository rep,
                    org.pentaho.metastore.api.IMetaStore metaStore,
                    org.pentaho.di.repository.ObjectId id_transformation,
                    org.pentaho.di.repository.ObjectId id_step)
             throws org.pentaho.di.core.exception.KettleException

从接口复制的说明: StepMetaInterface

Save the steps data into a Kettle repository

指定者:

saveRep 在接口中 StepMetaInterface

覆盖:

saveRep 在类中 BaseStepMeta

参数:

rep - The Kettle repository to save to

metaStore - the metaStore to optionally write to

id_transformation - The transformation ID

id_step - The step ID

抛出:

org.pentaho.di.core.exception.KettleException - When an unexpected error occurred (database, network, etc)

getFields

public void getFields(org.pentaho.di.core.row.RowMetaInterface rowMeta,
                      String origin,
                      org.pentaho.di.core.row.RowMetaInterface[] info,
                      StepMeta nextStep,
                      org.pentaho.di.core.variables.VariableSpace space,
                      Repository repository,
                      org.pentaho.metastore.api.IMetaStore metaStore)
               throws org.pentaho.di.core.exception.KettleStepException

从类复制的说明: BaseStepMeta

Gets the fields.

指定者:

getFields 在接口中 StepMetaInterface

覆盖:

getFields 在类中 BaseStepMeta

参数:

rowMeta - the input row meta that is modified in this method to reflect the output row metadata of the step

origin - Name of the step to use as input for the origin field in the values

info - Fields used as extra lookup information

nextStep - the next step that is targeted

space - the space The variable space to use to replace variables

repository - the repository to use to load Kettle metadata objects impacting the output fields

metaStore - the MetaStore to use to load additional external data or metadata impacting the output fields

抛出:

org.pentaho.di.core.exception.KettleStepException - the kettle step exception

check

public void check(List<org.pentaho.di.core.CheckResultInterface> remarks,
                  TransMeta transMeta,
                  StepMeta stepMeta,
                  org.pentaho.di.core.row.RowMetaInterface prev,
                  String[] input,
                  String[] output,
                  org.pentaho.di.core.row.RowMetaInterface info,
                  org.pentaho.di.core.variables.VariableSpace space,
                  Repository repository,
                  org.pentaho.metastore.api.IMetaStore metaStore)

从接口复制的说明: StepMetaInterface

Checks the settings of this step and puts the findings in a remarks List.

指定者:

check 在接口中 StepMetaInterface

覆盖:

check 在类中 BaseStepMeta

参数:

remarks - The list to put the remarks in @see org.pentaho.di.core.CheckResult

stepMeta - The stepMeta to help checking

prev - The fields coming from the previous step

input - The input step names

output - The output step names

info - The fields that are used as information by the step

space - the variable space to resolve variable expressions with

repository - the repository to use to load Kettle metadata objects impacting the output fields

metaStore - the MetaStore to use to load additional external data or metadata impacting the output fields

getSQLStatements

public org.pentaho.di.core.SQLStatement getSQLStatements(TransMeta transMeta,
                                                         StepMeta stepMeta,
                                                         org.pentaho.di.core.row.RowMetaInterface prev,
                                                         Repository repository,
                                                         org.pentaho.metastore.api.IMetaStore metaStore)
                                                  throws org.pentaho.di.core.exception.KettleStepException

从类复制的说明: BaseStepMeta

Standard method to return an SQLStatement object with SQL statements that the step needs in order to work correctly. This can mean "create table", "create index" statements but also "alter table ... add/drop/modify" statements.

指定者:

getSQLStatements 在接口中 StepMetaInterface

覆盖:

getSQLStatements 在类中 BaseStepMeta

参数:

transMeta - TransInfo object containing the complete transformation

stepMeta - StepMeta object containing the complete step

prev - Row containing meta-data for the input fields (no data)

repository - the repository to use to load Kettle metadata objects impacting the output fields

metaStore - the MetaStore to use to load additional external data or metadata impacting the output fields

返回:

The SQL Statements for this step. If nothing has to be done, the SQLStatement.getSQL() == null. @see SQLStatement

抛出:

org.pentaho.di.core.exception.KettleStepException

analyseImpact

public void analyseImpact(List<DatabaseImpact> impact,
                          TransMeta transMeta,
                          StepMeta stepMeta,
                          org.pentaho.di.core.row.RowMetaInterface prev,
                          String[] input,
                          String[] output,
                          org.pentaho.di.core.row.RowMetaInterface info,
                          Repository repository,
                          org.pentaho.metastore.api.IMetaStore metaStore)
                   throws org.pentaho.di.core.exception.KettleStepException

从类复制的说明: BaseStepMeta

Each step must be able to report on the impact it has on a database, table field, etc.

指定者:

analyseImpact 在接口中 StepMetaInterface

覆盖:

analyseImpact 在类中 BaseStepMeta

参数:

impact - The list of impacts @see org.pentaho.di.transMeta.DatabaseImpact

transMeta - The transformation information

stepMeta - The step information

prev - The fields entering this step

input - The previous step names

output - The output step names

info - The fields used as information by this step

repository - the repository to use to load Kettle metadata objects impacting the output fields

metaStore - the MetaStore to use to load additional external data or metadata impacting the output fields

抛出:

org.pentaho.di.core.exception.KettleStepException

getStep

public StepInterface getStep(StepMeta stepMeta,
                             StepDataInterface stepDataInterface,
                             int cnr,
                             TransMeta transMeta,
                             Trans trans)

从接口复制的说明: StepMetaInterface

Get the executing step, needed by Trans to launch a step.

指定者:

getStep 在接口中 StepMetaInterface

参数:

stepMeta - The step info

stepDataInterface - the step data interface linked to this step. Here the step can store temporary data, database connections, etc.

cnr - The copy nr to get

transMeta - The transformation info

trans - The launching transformation

getStepData

public StepDataInterface getStepData()

从接口复制的说明: StepMetaInterface

Get a new instance of the appropriate data class. This data class implements the StepDataInterface. It basically contains the persisting data that needs to live on, even if a worker thread is terminated.

指定者:

getStepData 在接口中 StepMetaInterface

返回:

The appropriate StepDataInterface class.

getUsedDatabaseConnections

public org.pentaho.di.core.database.DatabaseMeta[] getUsedDatabaseConnections()

从类复制的说明: BaseStepMeta

This method returns all the database connections that are used by the step.

指定者:

getUsedDatabaseConnections 在接口中 StepMetaInterface

覆盖:

getUsedDatabaseConnections 在类中 BaseStepMeta

返回:

an array of database connections meta-data. Return an empty array if no connections are used.

getRequiredFields

public org.pentaho.di.core.row.RowMetaInterface getRequiredFields(org.pentaho.di.core.variables.VariableSpace space)
                                                           throws org.pentaho.di.core.exception.KettleException

从类复制的说明: BaseStepMeta

The natural way of data flow in a transformation is source-to-target. However, this makes mapping to target tables difficult to do. To help out here, we supply information to the transformation meta-data model about which fields are required for a step. This allows us to automate certain tasks like the mapping to pre-defined tables. The Table Output step in this case will output the fields in the target table using this method.

This default implementation returns an empty row meaning that no fields are required for this step to operate.

指定者:

getRequiredFields 在接口中 StepMetaInterface

覆盖:

getRequiredFields 在类中 BaseStepMeta

参数:

space - the variable space to use to do variable substitution.

返回:

the required fields for this steps meta data.

抛出:

org.pentaho.di.core.exception.KettleException - in case the required fields can't be determined

getSchemaName

public String getSchemaName()

从接口复制的说明: ProvidesDatabaseConnectionInformation

Returns the schema name for this step.

指定者:

getSchemaName 在接口中 ProvidesDatabaseConnectionInformation

返回:

the schemaName

setSchemaName

public void setSchemaName(String schemaName)

参数:

schemaName - the schemaName to set

getEncoding

public String getEncoding()

setEncoding

public void setEncoding(String encoding)

getDelimiter

public String getDelimiter()

setDelimiter

public void setDelimiter(String delimiter)

getEnclosure

public String getEnclosure()

setEnclosure

public void setEnclosure(String enclosure)

getFifoFileName

public String getFifoFileName()

返回:

the fifoFileName

setFifoFileName

public void setFifoFileName(String fifoFileName)

参数:

fifoFileName - the fifoFileName to set

isReplacingData

public boolean isReplacingData()

返回:

the replacingData

setReplacingData

public void setReplacingData(boolean replacingData)

参数:

replacingData - the replacingData to set

getFieldFormatType

public int[] getFieldFormatType()

setFieldFormatType

public void setFieldFormatType(int[] fieldFormatType)

getFieldFormatTypeCodes

public static String[] getFieldFormatTypeCodes()

getFieldFormatTypeDescriptions

public static String[] getFieldFormatTypeDescriptions()

getFieldFormatTypeCode

public static String getFieldFormatTypeCode(int type)

getFieldFormatTypeDescription

public static String getFieldFormatTypeDescription(int type)

getFieldFormatType

public static int getFieldFormatType(String codeOrDescription)

getEscapeChar

public String getEscapeChar()

返回:

the escapeChar

setEscapeChar

public void setEscapeChar(String escapeChar)

参数:

escapeChar - the escapeChar to set

isIgnoringErrors

public boolean isIgnoringErrors()

返回:

the ignoringErrors

setIgnoringErrors

public void setIgnoringErrors(boolean ignoringErrors)

参数:

ignoringErrors - the ignoringErrors to set

getBulkSize

public String getBulkSize()

返回:

the bulkSize

setBulkSize

public void setBulkSize(String bulkSize)

参数:

bulkSize - the bulkSize to set

isLocalFile

public boolean isLocalFile()

返回:

the localFile

setLocalFile

public void setLocalFile(boolean localFile)

参数:

localFile - the localFile to set

getMissingDatabaseConnectionInformationMessage

public String getMissingDatabaseConnectionInformationMessage()

从接口复制的说明: ProvidesDatabaseConnectionInformation

Provides a way for this object to return a custom message when database connection information is incomplete or missing. If this returns null a default message will be displayed for missing information.

指定者:

getMissingDatabaseConnectionInformationMessage 在接口中 ProvidesDatabaseConnectionInformation

返回:

A friendly message that describes that database connection information is missing and, potentially, why.

afterInjectionSynchronization

public void afterInjectionSynchronization()

If we use injection we can have different arrays lengths. We need synchronize them for consistency behavior with UI