de.viadee.ki.sparkimporter.processing.steps.importing

Class ColumnsPreparationStep

java.lang.Object

de.viadee.ki.sparkimporter.processing.steps.importing.ColumnsPreparationStep

All Implemented Interfaces:

PreprocessingStepInterface

public class ColumnsPreparationStep
extends Object
implements PreprocessingStepInterface

Constructor Summary

Constructors

Constructor and Description
ColumnsPreparationStep()

Method Summary

Modifier and Type	Method and Description
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	runPreprocessingStep(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataset, boolean writeStepResultIntoFile, String dataLevel, Map<String,Object> parameters) Defines one processing step and what to do with the data

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ColumnsPreparationStep

public ColumnsPreparationStep()

Method Detail

runPreprocessingStep

public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> runPreprocessingStep(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataset,
                                                                                   boolean writeStepResultIntoFile,
                                                                                   String dataLevel,
                                                                                   Map<String,Object> parameters)

Description copied from interface: PreprocessingStepInterface

Defines one processing step and what to do with the data

Specified by:

runPreprocessingStep in interface PreprocessingStepInterface

Parameters:

dataset - the incoming dataset for this processing step

Returns:

the resulting dataset of the processing step