nxcals.api.extraction.data.builders.SparkSession.sparkContext

property SparkSession.sparkContext: SparkContext

Returns the underlying SparkContext.

New in version 2.0.0.

Examples

>>> spark.sparkContext
<SparkContext master=... appName=...>

Create an RDD from the Spark context

>>> rdd = spark.sparkContext.parallelize([1, 2, 3])
>>> rdd.collect()
[1, 2, 3]