org.apache.seatunnel.core.starter.spark

类 SparkStarter

java.lang.Object

org.apache.seatunnel.core.starter.spark.SparkStarter

所有已实现的接口:

Starter

public class SparkStarter
extends Object
implements Starter

A Starter to generate spark-submit command for SeaTunnel job on spark.

字段概要

字段

限定符和类型	字段和说明
protected String[]	args original commandline args
protected SparkCommandArgs	commandArgs args parsed from args
protected List<Path>	files files to be placed in the working directory of each spark executor
protected List<Path>	jars jars to include on the spark driver and executor classpaths
protected Map<String,String>	sparkConf spark configuration properties

方法概要

限定符和类型	方法和说明
protected void	appendAppJar(List<String> commands) append appJar to StringBuilder
protected void	appendFiles(List<String> commands, List<Path> paths) append files option to StringBuilder
protected void	appendJars(List<String> commands, List<Path> paths) append jars option to StringBuilder
protected void	appendOption(List<String> commands, String option, String value) append option to StringBuilder
protected void	appendPaths(List<String> commands, String option, List<Path> paths) append comma-split paths option to StringBuilder
protected void	appendSparkConf(List<String> commands, Map<String,String> sparkConf) append spark configurations to StringBuilder
List<String>	buildCommands()
protected List<String>	buildFinal() build final spark-submit commands
static void	main(String[] args)

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

args

protected String[] args

original commandline args

commandArgs

protected SparkCommandArgs commandArgs

args parsed from args

jars

protected List<Path> jars

jars to include on the spark driver and executor classpaths

files

protected List<Path> files

files to be placed in the working directory of each spark executor

sparkConf

protected Map<String,String> sparkConf

spark configuration properties

方法详细资料

main

public static void main(String[] args)
                 throws IOException

抛出:

IOException

buildCommands

public List<String> buildCommands()
                           throws IOException

指定者:

buildCommands 在接口中 Starter

抛出:

IOException

buildFinal

protected List<String> buildFinal()

build final spark-submit commands

appendOption

protected void appendOption(List<String> commands,
                            String option,
                            String value)

append option to StringBuilder

appendJars

protected void appendJars(List<String> commands,
                          List<Path> paths)

append jars option to StringBuilder

appendFiles

protected void appendFiles(List<String> commands,
                           List<Path> paths)

append files option to StringBuilder

appendPaths

protected void appendPaths(List<String> commands,
                           String option,
                           List<Path> paths)

append comma-split paths option to StringBuilder

appendSparkConf

protected void appendSparkConf(List<String> commands,
                               Map<String,String> sparkConf)

append spark configurations to StringBuilder

appendAppJar

protected void appendAppJar(List<String> commands)

append appJar to StringBuilder