nxcals.api.extraction.data.builders.DataFrame.hint

DataFrame.hint(name: str, *parameters: Union[PrimitiveType, List[PrimitiveType]]) → DataFrame

Specifies some hint on the current DataFrame.

New in version 2.2.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters:

name (str) – A name of the hint.
parameters (str, list, float or int) – Optional parameters.

Returns:

Hinted DataFrame

Return type:

DataFrame

Examples

>>> df = spark.createDataFrame([(2, "Alice"), (5, "Bob")], schema=["age", "name"])
>>> df2 = spark.createDataFrame([Row(height=80, name="Tom"), Row(height=85, name="Bob")])
>>> df.join(df2, "name").explain()  
== Physical Plan ==
...
... +- SortMergeJoin ...
...

Explicitly trigger the broadcast hashjoin by providing the hint in df2.

>>> df.join(df2.hint("broadcast"), "name").explain()
== Physical Plan ==
...
... +- BroadcastHashJoin ...
...