nxcals.api.extraction.data.builders.DataFrame.withMetadata

DataFrame.withMetadata(columnName: str, metadata: Dict[str, Any]) → DataFrame

Returns a new DataFrame by updating an existing column with metadata.

New in version 3.3.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters:

• columnName (str) – string, name of the existing column to update the metadata.

• metadata (dict) – dict, new metadata to be assigned to df.schema[columnName].metadata

Returns:

DataFrame with updated metadata column.

Return type:

DataFrame

Examples

>>> df = spark.createDataFrame([(2, "Alice"), (5, "Bob")], schema=["age", "name"])
>>> df_meta = df.withMetadata('age', {'foo': 'bar'})
>>> df_meta.schema['age'].metadata
{'foo': 'bar'}