Error “org.apache.spark.SparkException: A master URL must be set in your configuration”

15/10/06 17:14:43 ERROR SparkContext: Error initializing SparkContext.
org.apache.spark.SparkException: A master URL must be set in your configuration

SparkException: A master URL must be set in your configurationIf you’ve got such Spark exception in the output, it means you simply forgot to specify the master URL.┬áProbably you’re running Spark locally. The most common mistake of running the application from IDE is the absence of setMaster method.

The example from official site is used to run in console, where you specify the master URL manually:

<SPARK_HOME>/bin/spark-submit --class "<CLASS_NAME>" --master local target/<JAR_FILE>

If you want to run your App in IDE, just set the master URL in SparkConf instance with the value “local“:

SparkConf configuration = new SparkConf()
                .setAppName("Your Application Name")
                .setMaster("local");

Leave a Reply

1 Comment on "Error “org.apache.spark.SparkException: A master URL must be set in your configuration”"


Guest
Aneesha
1 year 4 months ago

thanks :)