nxcals.api.extraction.data.builders.DataQuery
- class nxcals.api.extraction.data.builders.DataQuery(spark, builder_class, builders_package)
A class that provides methods for building queries on data that has been loaded into a Spark dataframe.
Methods
DataQuery.__init__(spark, builder_class, ...)DataQuery.builder(spark)DataQuery.getAsPivot(spark, start_time, ...)Create a dataset with variable names as columns. Values are joined on timestamps. Experimental, may be changed in the future. Doesn't support variables pointing to whole entities. :param spark: spark session :param start_time: begin of extraction time window :param end_time: end of extraction time window :param system: system name, where variables will be searched from variable_names :param variables: iterable containing variable names, which must be registered in system. All must exist, throws exception otherwise :param variables_like: iterable containing variable name patterns, which must be registered in system :return: PySpark DataFrame, with one column "nxcals_timestamp" and other columns named after variables, e.g. "nxcals_timestamp", "BLM1:arr", "BLM2:voltage".
DataQuery.getForEntities(spark, start_time, ...)DataQuery.getForVariables(spark, start_time, ...)