Class DataQuery

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      static DataQuery builder​(@NonNull org.apache.spark.sql.SparkSession session)  
      static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getAsPivot​(@NonNull org.apache.spark.sql.SparkSession spark, @NonNull TimeWindow timeWindow, @NonNull java.util.Collection<Variable> variables)
      Create a dataset with variable names as columns.
      static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor​(@NonNull org.apache.spark.sql.SparkSession sparkSession, @NonNull TimeWindow timeWindow, @NonNull Entity entity)  
      static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor​(@NonNull org.apache.spark.sql.SparkSession sparkSession, @NonNull TimeWindow timeWindow, @NonNull Variable variable)  
      static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor​(@NonNull org.apache.spark.sql.SparkSession spark, @NonNull TimeWindow timeWindow, @NonNull java.lang.String system, @NonNull EntityQuery... entitiesQueries)  
      static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor​(@NonNull org.apache.spark.sql.SparkSession spark, @NonNull TimeWindow timeWindow, @NonNull java.lang.String system, @NonNull java.lang.String... variables)  
      static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor​(@NonNull org.apache.spark.sql.SparkSession spark, @NonNull TimeWindow timeWindow, @NonNull java.lang.String system, @NonNull java.util.List<java.lang.String> variables)  
      static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor​(@NonNull org.apache.spark.sql.SparkSession spark, @NonNull TimeWindow timeWindow, @NonNull java.lang.String system, @NonNull java.util.List<java.lang.String> variables, @NonNull java.util.List<java.lang.String> variablesLike)  
      static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor​(@NonNull org.apache.spark.sql.SparkSession spark, @NonNull TimeWindow timeWindow, @NonNull java.lang.String system, @NonNull java.util.Map<java.lang.String,​java.lang.Object>... keyValuesArr)  
      protected QueryData<org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>> queryData()  
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Method Detail

      • builder

        public static DataQuery builder​(@NonNull
                                        @NonNull org.apache.spark.sql.SparkSession session)
      • getFor

        public static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor​(@NonNull
                                                                                    @NonNull org.apache.spark.sql.SparkSession sparkSession,
                                                                                    @NonNull
                                                                                    @NonNull TimeWindow timeWindow,
                                                                                    @NonNull
                                                                                    @NonNull Variable variable)
      • getFor

        public static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor​(@NonNull
                                                                                    @NonNull org.apache.spark.sql.SparkSession sparkSession,
                                                                                    @NonNull
                                                                                    @NonNull TimeWindow timeWindow,
                                                                                    @NonNull
                                                                                    @NonNull Entity entity)
      • getFor

        @SafeVarargs
        public static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor​(@NonNull
                                                                                    @NonNull org.apache.spark.sql.SparkSession spark,
                                                                                    @NonNull
                                                                                    @NonNull TimeWindow timeWindow,
                                                                                    @NonNull
                                                                                    @NonNull java.lang.String system,
                                                                                    @NonNull
                                                                                    @NonNull java.util.Map<java.lang.String,​java.lang.Object>... keyValuesArr)
      • getFor

        public static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor​(@NonNull
                                                                                    @NonNull org.apache.spark.sql.SparkSession spark,
                                                                                    @NonNull
                                                                                    @NonNull TimeWindow timeWindow,
                                                                                    @NonNull
                                                                                    @NonNull java.lang.String system,
                                                                                    @NonNull
                                                                                    @NonNull EntityQuery... entitiesQueries)
      • getFor

        public static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor​(@NonNull
                                                                                    @NonNull org.apache.spark.sql.SparkSession spark,
                                                                                    @NonNull
                                                                                    @NonNull TimeWindow timeWindow,
                                                                                    @NonNull
                                                                                    @NonNull java.lang.String system,
                                                                                    @NonNull
                                                                                    @NonNull java.lang.String... variables)
      • getFor

        public static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor​(@NonNull
                                                                                    @NonNull org.apache.spark.sql.SparkSession spark,
                                                                                    @NonNull
                                                                                    @NonNull TimeWindow timeWindow,
                                                                                    @NonNull
                                                                                    @NonNull java.lang.String system,
                                                                                    @NonNull
                                                                                    @NonNull java.util.List<java.lang.String> variables)
      • getFor

        public static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getFor​(@NonNull
                                                                                    @NonNull org.apache.spark.sql.SparkSession spark,
                                                                                    @NonNull
                                                                                    @NonNull TimeWindow timeWindow,
                                                                                    @NonNull
                                                                                    @NonNull java.lang.String system,
                                                                                    @NonNull
                                                                                    @NonNull java.util.List<java.lang.String> variables,
                                                                                    @NonNull
                                                                                    @NonNull java.util.List<java.lang.String> variablesLike)
      • getAsPivot

        @Experimental
        public static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getAsPivot​(@NonNull
                                                                                        @NonNull org.apache.spark.sql.SparkSession spark,
                                                                                        @NonNull
                                                                                        @NonNull TimeWindow timeWindow,
                                                                                        @NonNull
                                                                                        @NonNull java.util.Collection<Variable> variables)
        Create a dataset with variable names as columns. Values are joined on timestamps. Experimental, may be changed in the future.
        Parameters:
        spark - - active spark session
        timeWindow - - for data extraction
        variables - - requested variables
        Returns:
        - dataset with one column "nxcals_timestamp" and other columns named as variables
      • queryData

        protected QueryData<org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>> queryData()
        Specified by:
        queryData in class AbstractDataQuery<org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>>