Package cern.nxcals.api.extraction.data.builders

Class DataQuery

java.lang.Object

cern.nxcals.api.extraction.data.builders.fluent.v2.AbstractDataQuery<org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>>

cern.nxcals.api.extraction.data.builders.LegacyBuildersDataQuery

cern.nxcals.api.extraction.data.builders.DataQuery

public class DataQuery
extends LegacyBuildersDataQuery

Field Summary

Fields inherited from class cern.nxcals.api.extraction.data.builders.LegacyBuildersDataQuery

session

Method Summary

Modifier and Type	Method	Description
static DataQuery	builder(@NonNull org.apache.spark.sql.SparkSession session)
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	getAsPivot(@NonNull org.apache.spark.sql.SparkSession spark, @NonNull TimeWindow timeWindow, @NonNull java.util.Collection<Variable> variables)	Create a dataset with variable names as columns.
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	getFor(@NonNull org.apache.spark.sql.SparkSession sparkSession, @NonNull TimeWindow timeWindow, @NonNull Entity entity)
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	getFor(@NonNull org.apache.spark.sql.SparkSession sparkSession, @NonNull TimeWindow timeWindow, @NonNull Variable variable)
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	getFor(@NonNull org.apache.spark.sql.SparkSession spark, @NonNull TimeWindow timeWindow, @NonNull java.lang.String system, @NonNull EntityQuery... entitiesQueries)
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	getFor(@NonNull org.apache.spark.sql.SparkSession spark, @NonNull TimeWindow timeWindow, @NonNull java.lang.String system, @NonNull java.lang.String... variables)
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	getFor(@NonNull org.apache.spark.sql.SparkSession spark, @NonNull TimeWindow timeWindow, @NonNull java.lang.String system, @NonNull java.util.List<java.lang.String> variables)
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	getFor(@NonNull org.apache.spark.sql.SparkSession spark, @NonNull TimeWindow timeWindow, @NonNull java.lang.String system, @NonNull java.util.List<java.lang.String> variables, @NonNull java.util.List<java.lang.String> variablesLike)
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	getFor(@NonNull org.apache.spark.sql.SparkSession spark, @NonNull TimeWindow timeWindow, @NonNull java.lang.String system, @NonNull java.util.Map<java.lang.String,java.lang.Object>... keyValuesArr)
protected QueryData<org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>>	queryData()

Methods inherited from class cern.nxcals.api.extraction.data.builders.LegacyBuildersDataQuery

byEntities, byVariables

Methods inherited from class cern.nxcals.api.extraction.data.builders.fluent.v2.AbstractDataQuery

entities, variables

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

builder

public static DataQuery builder(@NonNull
                                @NonNull org.apache.spark.sql.SparkSession session)

getFor

public static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor(@NonNull
                                                                            @NonNull org.apache.spark.sql.SparkSession sparkSession,
                                                                            @NonNull
                                                                            @NonNull TimeWindow timeWindow,
                                                                            @NonNull
                                                                            @NonNull Variable variable)

getFor

public static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor(@NonNull
                                                                            @NonNull org.apache.spark.sql.SparkSession sparkSession,
                                                                            @NonNull
                                                                            @NonNull TimeWindow timeWindow,
                                                                            @NonNull
                                                                            @NonNull Entity entity)

getFor

@SafeVarargs
public static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor(@NonNull
                                                                            @NonNull org.apache.spark.sql.SparkSession spark,
                                                                            @NonNull
                                                                            @NonNull TimeWindow timeWindow,
                                                                            @NonNull
                                                                            @NonNull java.lang.String system,
                                                                            @NonNull
                                                                            @NonNull java.util.Map<java.lang.String,java.lang.Object>... keyValuesArr)

getFor

public static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor(@NonNull
                                                                            @NonNull org.apache.spark.sql.SparkSession spark,
                                                                            @NonNull
                                                                            @NonNull TimeWindow timeWindow,
                                                                            @NonNull
                                                                            @NonNull java.lang.String system,
                                                                            @NonNull
                                                                            @NonNull EntityQuery... entitiesQueries)

getFor

public static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor(@NonNull
                                                                            @NonNull org.apache.spark.sql.SparkSession spark,
                                                                            @NonNull
                                                                            @NonNull TimeWindow timeWindow,
                                                                            @NonNull
                                                                            @NonNull java.lang.String system,
                                                                            @NonNull
                                                                            @NonNull java.lang.String... variables)

getFor

public static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor(@NonNull
                                                                            @NonNull org.apache.spark.sql.SparkSession spark,
                                                                            @NonNull
                                                                            @NonNull TimeWindow timeWindow,
                                                                            @NonNull
                                                                            @NonNull java.lang.String system,
                                                                            @NonNull
                                                                            @NonNull java.util.List<java.lang.String> variables)

getFor

public static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor(@NonNull
                                                                            @NonNull org.apache.spark.sql.SparkSession spark,
                                                                            @NonNull
                                                                            @NonNull TimeWindow timeWindow,
                                                                            @NonNull
                                                                            @NonNull java.lang.String system,
                                                                            @NonNull
                                                                            @NonNull java.util.List<java.lang.String> variables,
                                                                            @NonNull
                                                                            @NonNull java.util.List<java.lang.String> variablesLike)

getAsPivot

@Experimental
public static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getAsPivot(@NonNull
                                                                                @NonNull org.apache.spark.sql.SparkSession spark,
                                                                                @NonNull
                                                                                @NonNull TimeWindow timeWindow,
                                                                                @NonNull
                                                                                @NonNull java.util.Collection<Variable> variables)

Create a dataset with variable names as columns. Values are joined on timestamps. Experimental, may be changed in the future.

Parameters:

spark - - active spark session

timeWindow - - for data extraction

variables - - requested variables

Returns:

- dataset with one column "nxcals_timestamp" and other columns named as variables

queryData

protected QueryData<org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>> queryData()

Specified by:

queryData in class AbstractDataQuery<org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>>