execution

Linear Supertypes

AnyRef, Any

case class CustomResourceProfileExec(child: SparkPlan) extends SparkPlan with UnaryExecNode with Product with Serializable
This node wraps the final executed plan and inject custom resource profile to the RDD.
This node wraps the final executed plan and inject custom resource profile to the RDD. It assumes that, the produced RDD would create the ResultStage in DAGScheduler, so it makes resource isolation between previous and final stage.
Note that, Spark does not support config minExecutors for each resource profile. Which means, it would retain minExecutors for each resource profile. So, suggest set spark.dynamicAllocation.minExecutors to 0 if enable this feature.

Packages