Type Members

1.  case class FinalStageConfigIsolation(session: SparkSession) extends Rule[SparkPlan] with Product with Serializable

   This rule split stage into two parts:

   This rule split stage into two parts:

   1. previous stage 2. final stage For final stage, we can inject extra config. It's useful if we use repartition to optimize small files that needs bigger shuffle partition size than previous.

   Let's say we have a query with 3 stages, then the logical machine like:

   Set/Reset Command -> cleanup previousStage config if user set the spark config. Query -> AQE -> stage1 -> preparation (use previousStage to overwrite spark config) -> AQE -> stage2 -> preparation (use spark config) -> AQE -> stage3 -> preparation (use finalStage config to overwrite spark config, store spark config to previousStage.)

   An example of the new finalStage config: spark.sql.adaptive.advisoryPartitionSizeInBytes -> spark.sql.finalStage.adaptive.advisoryPartitionSizeInBytes

2.  case class FinalStageConfigIsolationCleanRule(session: SparkSession) extends Rule[LogicalPlan] with Product with Serializable
3.  class KyuubiSQLExtensionException extends SQLException
4.  class KyuubiSparkSQLAstBuilder extends KyuubiSparkSQLBaseVisitor[AnyRef] with SQLConfHelper
5.  class KyuubiSparkSQLBaseListener extends KyuubiSparkSQLListener

   This class provides an empty implementation of KyuubiSparkSQLListener, which can be extended to create a listener which only needs to handle a subset of the available methods.

6.  class KyuubiSparkSQLBaseVisitor[T] extends AbstractParseTreeVisitor[T] with KyuubiSparkSQLVisitor[T]

   This class provides an empty implementation of KyuubiSparkSQLVisitor, which can be extended to create a visitor which only needs to handle a subset of the available methods.

7.  class KyuubiSparkSQLLexer extends Lexer

   Annotations

   @SuppressWarnings()

8.  trait KyuubiSparkSQLListener extends ParseTreeListener

   This interface defines a complete listener for a parse tree produced by KyuubiSparkSQLParser.

9.  class KyuubiSparkSQLParser extends Parser

   Annotations

   @SuppressWarnings()

10.  trait KyuubiSparkSQLVisitor[T] extends ParseTreeVisitor[T]

    This interface defines a complete generic visitor for a parse tree produced by KyuubiSparkSQLParser.

11.  case class MarkNumOutputColumnsRule(session: SparkSession) extends Rule[LogicalPlan] with Product with Serializable
12.  trait RepartitionBeforeWriteHelper extends Rule[LogicalPlan]
13.  abstract class RepartitionBeforeWritingDatasourceBase extends Rule[LogicalPlan] with RepartitionBuilder

    For datasource table, there two commands can write data to table 1.

    For datasource table, there two commands can write data to table 1. InsertIntoHadoopFsRelationCommand 2. CreateDataSourceTableAsSelectCommand This rule add a repartition node between write and query

14.  abstract class RepartitionBeforeWritingHiveBase extends Rule[LogicalPlan] with RepartitionBuilder

    For Hive table, there two commands can write data to table 1.

    For Hive table, there two commands can write data to table 1. InsertIntoHiveTable 2. CreateHiveTableAsSelectCommand This rule add a repartition node between write and query

15.  trait RepartitionBuilder extends Rule[LogicalPlan] with RepartitionBeforeWriteHelper
16.  trait UnparsedExpressionLogicalPlan extends LogicalPlan

    a logical plan contains an unparsed expression that will be parsed by spark.

17.  case class UnparsedPredicateOptimize(tableIdent: Seq[String], tablePredicate: Option[String], orderExpr: Expression) extends LogicalPlan with UnparsedExpressionLogicalPlan with Product with Serializable