Spark Job Server Client
The Spark Job Server operator schedules a specified Spark job on the Spark job server when it is started and stops the job when the operator is terminated. The associated Spark context can be either preserved or removed based on the configuration. The jar file must already be installed on the server.
Configuration Parameters
|
Parameter |
Type |
Description |
|---|---|---|
|
jobServer |
string |
Mandatory. The Spark job server URL. Default: "http://localhost:8090" |
|
jobProps |
string |
Mandatory. The query parameters used for the Spark job server's job API call. Default: "appName=test-job-name&classPath=test.spark.TestSparkStreamingKafkaJob" |
|
jobSArgs |
string |
Mandatory. The program arguments passed to the Spark job. Default: "input.args= \"localhost:2181 test_group teset_topic_in 1 localhost:9092 test_topic_out\"" |
|
stopSC |
string |
Specifies whether the Spark context associated with the job is terminated when the operator is shut down. Default: "false" |
Input
None
Output
None