Type Members

1.  case class DropIgnoreNonexistent(session: SparkSession) extends Rule[LogicalPlan] with Product with Serializable
2.  case class FinalStageConfigIsolation(session: SparkSession) extends Rule[SparkPlan] with Product with Serializable

   This rule split stage into two parts:

   This rule split stage into two parts:

   1. previous stage 2. final stage For final stage, we can inject extra config. It's useful if we use repartition to optimize small files that needs bigger shuffle partition size than previous.

   Let's say we have a query with 3 stages, then the logical machine like:

   Set/Reset Command -> cleanup previousStage config if user set the spark config. Query -> AQE -> stage1 -> preparation (use previousStage to overwrite spark config) -> AQE -> stage2 -> preparation (use spark config) -> AQE -> stage3 -> preparation (use finalStage config to overwrite spark config, store spark config to previousStage.)

   An example of the new finalStage config: spark.sql.adaptive.advisoryPartitionSizeInBytes -> spark.sql.finalStage.adaptive.advisoryPartitionSizeInBytes

3.  case class FinalStageConfigIsolationCleanRule(session: SparkSession) extends Rule[LogicalPlan] with Product with Serializable
4.  class KyuubiSQLExtensionException extends SQLException
5.  class KyuubiSparkSQLAstBuilder extends KyuubiSparkSQLBaseVisitor[AnyRef] with SQLConfHelper
6.  class KyuubiSparkSQLBaseListener extends KyuubiSparkSQLListener

   This class provides an empty implementation of KyuubiSparkSQLListener, which can be extended to create a listener which only needs to handle a subset of the available methods.

7.  class KyuubiSparkSQLBaseVisitor[T] extends AbstractParseTreeVisitor[T] with KyuubiSparkSQLVisitor[T]

   This class provides an empty implementation of KyuubiSparkSQLVisitor, which can be extended to create a visitor which only needs to handle a subset of the available methods.

8.  class KyuubiSparkSQLCommonExtension extends (SparkSessionExtensions) ⇒ Unit
9.  class KyuubiSparkSQLExtension extends (SparkSessionExtensions) ⇒ Unit

   Depend on Spark SQL Extension framework, we can use this extension follow steps

   Depend on Spark SQL Extension framework, we can use this extension follow steps

   1. move this jar into $SPARK_HOME/jars 2. add config into spark-defaults.conf: spark.sql.extensions=org.apache.kyuubi.sql.KyuubiSparkSQLExtension
10.  class KyuubiSparkSQLLexer extends Lexer

    Annotations

    @SuppressWarnings()

11.  trait KyuubiSparkSQLListener extends ParseTreeListener

    This interface defines a complete listener for a parse tree produced by KyuubiSparkSQLParser.

12.  class KyuubiSparkSQLParser extends Parser

    Annotations

    @SuppressWarnings()

13.  abstract class KyuubiSparkSQLParserBase extends ParserInterface with SQLConfHelper
14.  trait KyuubiSparkSQLVisitor[T] extends ParseTreeVisitor[T]

    This interface defines a complete generic visitor for a parse tree produced by KyuubiSparkSQLParser.

15.  case class RebalanceBeforeWritingDatasource(session: SparkSession) extends RepartitionBeforeWritingDatasourceBase with RepartitionBuilderWithRebalance with Product with Serializable

    For datasource table, there two commands can write data to table 1.

    For datasource table, there two commands can write data to table 1. InsertIntoHadoopFsRelationCommand 2. CreateDataSourceTableAsSelectCommand This rule add a RebalancePartitions node between write and query

16.  case class RebalanceBeforeWritingHive(session: SparkSession) extends RepartitionBeforeWritingHiveBase with RepartitionBuilderWithRebalance with Product with Serializable

    For Hive table, there two commands can write data to table 1.

    For Hive table, there two commands can write data to table 1. InsertIntoHiveTable 2. CreateHiveTableAsSelectCommand This rule add a RebalancePartitions node between write and query

17.  trait RepartitionBeforeWriteHelper extends Rule[LogicalPlan]
18.  abstract class RepartitionBeforeWritingDatasourceBase extends Rule[LogicalPlan] with RepartitionBuilder

    For datasource table, there two commands can write data to table 1.

    For datasource table, there two commands can write data to table 1. InsertIntoHadoopFsRelationCommand 2. CreateDataSourceTableAsSelectCommand This rule add a repartition node between write and query

19.  abstract class RepartitionBeforeWritingHiveBase extends Rule[LogicalPlan] with RepartitionBuilder

    For Hive table, there two commands can write data to table 1.

    For Hive table, there two commands can write data to table 1. InsertIntoHiveTable 2. CreateHiveTableAsSelectCommand This rule add a repartition node between write and query

20.  trait RepartitionBuilder extends Rule[LogicalPlan] with RepartitionBeforeWriteHelper
21.  trait RepartitionBuilderWithRebalance extends Rule[LogicalPlan] with RepartitionBuilder
22.  class SparkKyuubiSparkSQLParser extends KyuubiSparkSQLParserBase
23.  trait UnparsedExpressionLogicalPlan extends LogicalPlan

    a logical plan contains an unparsed expression that will be parsed by spark.

24.  case class UnparsedPredicateOptimize(tableIdent: Seq[String], tablePredicate: Option[String], orderExpr: Expression) extends LogicalPlan with UnparsedExpressionLogicalPlan with Product with Serializable
25.  class UpperCaseCharStream extends CharStream