nxcals.api.extraction.data.builders.SparkSession.newSession

SparkSession.newSession() → SparkSession

Returns a new SparkSession as new session, that has separate SQLConf, registered temporary views and UDFs, but shared SparkContext and table cache.

New in version 2.0.0.

Returns:

Spark session if an active session exists for the current thread

Return type:

SparkSession

Examples

>>> spark.newSession()
<...SparkSession object ...>