nxcals.api.extraction.data.builders.DataFrame.approxQuantile

DataFrame.approxQuantile(col: str, probabilities: Union[List[float], Tuple[float]], relativeError: float) → List[float]
DataFrame.approxQuantile(col: Union[List[str], Tuple[str]], probabilities: Union[List[float], Tuple[float]], relativeError: float) → List[List[float]]

Calculates the approximate quantiles of numerical columns of a DataFrame.

The result of this algorithm has the following deterministic bound: If the DataFrame has N elements and if we request the quantile at probability p up to error err, then the algorithm will return a sample x from the DataFrame so that the exact rank of x is close to (p * N). More precisely,

floor((p - err) * N) <= rank(x) <= ceil((p + err) * N).

This method implements a variation of the Greenwald-Khanna algorithm (with some speed optimizations). The algorithm was first present in [[https://doi.org/10.1145/375663.375670 Space-efficient Online Computation of Quantile Summaries]] by Greenwald and Khanna.

Note that null values will be ignored in numerical columns before calculation. For columns only containing null values, an empty list is returned.

New in version 2.0.0.

Parameters:

  • col (str, tuple or list) –

    Can be a single column name, or a list of names for multiple columns.

    Changed in version 2.2: Added support for multiple columns.

  • probabilities (list or tuple) – a list of quantile probabilities Each number must belong to [0, 1]. For example 0 is the minimum, 0.5 is the median, 1 is the maximum.

  • relativeError (float) – The relative target precision to achieve (>= 0). If set to zero, the exact quantiles are computed, which could be very expensive. Note that values greater than 1 are accepted but give the same result as 1.

Returns:

the approximate quantiles at the given probabilities. If the input col is a string, the output is a list of floats. If the input col is a list or tuple of strings, the output is also a list, but each element in it is a list of floats, i.e., the output is a list of list of floats.

Return type:

list