de.viadee.ki.sparkimporter.processing.steps.userconfig

Class VariableFilterStep

java.lang.Object

de.viadee.ki.sparkimporter.processing.steps.userconfig.VariableFilterStep

All Implemented Interfaces:

PreprocessingStepInterface

public class VariableFilterStep
extends Object
implements PreprocessingStepInterface

Constructor Summary

Constructors

Constructor and Description
VariableFilterStep()

Method Summary

Modifier and Type	Method and Description
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	runPreprocessingStep(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataSet, boolean writeStepResultIntoFile, String dataLevel, Map<String,Object> parameters) Defines one processing step and what to do with the data

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

VariableFilterStep

public VariableFilterStep()

Method Detail

runPreprocessingStep

public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> runPreprocessingStep(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataSet,
                                                                                   boolean writeStepResultIntoFile,
                                                                                   String dataLevel,
                                                                                   Map<String,Object> parameters)

Description copied from interface: PreprocessingStepInterface

Defines one processing step and what to do with the data

Specified by:

runPreprocessingStep in interface PreprocessingStepInterface

Parameters:

dataSet - the incoming dataset for this processing step

Returns:

the resulting dataset of the processing step