nxcals.api.extraction.data.builders.DataFrame.sortWithinPartitions

DataFrame.sortWithinPartitions(*cols: Union[str, Column, List[Union[str, Column]]], **kwargs: Any) → DataFrame

Returns a new DataFrame with each partition sorted by the specified column(s).

New in version 1.6.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters:

  • cols (str, list or Column, optional) – list of Column or column names to sort by.

  • ascending (bool or list, optional, default True) – boolean or list of boolean. Sort ascending vs. descending. Specify list for multiple sort orders. If a list is specified, the length of the list must equal the length of the cols.

Returns:

DataFrame sorted by partitions.

Return type:

DataFrame

Examples

>>> df = spark.createDataFrame([(2, "Alice"), (5, "Bob")], schema=["age", "name"])
>>> df.sortWithinPartitions("age", ascending=False)
DataFrame[age: bigint, name: string]