nxcals.api.extraction.data.builders.SparkSession.addArtifact
- SparkSession.addArtifact(*path: str, pyfile: bool = False, archive: bool = False, file: bool = False) None
Add artifact(s) to the client session. Currently only local files are supported.
New in version 3.5.0.
- Parameters:
*path (tuple of str) – Artifact’s URIs to add.
pyfile (bool) – Whether to add them as Python dependencies such as .py, .egg, .zip or .jar files. The pyfiles are directly inserted into the path when executing Python functions in executors.
archive (bool) – Whether to add them as archives such as .zip, .jar, .tar.gz, .tgz, or .tar files. The archives are unpacked on the executor side automatically.
file (bool) – Add a file to be downloaded with this Spark job on every node. The
path
passed can only be a local file for now.
Notes
This is an API dedicated to Spark Connect client only. With regular Spark Session, it throws an exception.