Package cern.nxcals.api.extraction.data

Class ExtractionUtils

  • public final class ExtractionUtils
extends java.lang.Object

    • Field Summary

      Fields 
      Modifier and Type Field Description
      static org.apache.avro.Schema STRING_SCHEMA  

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> createEmptyDataFrame​(org.apache.spark.sql.SparkSession session, java.util.Collection<cern.nxcals.common.domain.ColumnMapping> mappings)  
      static java.util.List<org.apache.avro.Schema.Field> extractSchemaFields​(java.lang.String schemaContent)  
      static org.apache.spark.sql.types.DataType getDataTypeFor​(org.apache.avro.Schema schema)  
      static java.lang.String getHbaseTypeNameFor​(org.apache.avro.Schema schema)  
      static org.apache.spark.sql.types.StructType getStructSchemaFor​(java.util.Collection<cern.nxcals.common.domain.ColumnMapping> fields)  
      static java.lang.String getTimestampFieldName​(@NonNull SystemSpec systemData)  
      static org.apache.spark.sql.Column toColumn​(cern.nxcals.common.domain.ColumnMapping mapping)  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • STRING_SCHEMA

        public static final org.apache.avro.Schema STRING_SCHEMA

    • Method Detail

      • getTimestampFieldName

        public static java.lang.String getTimestampFieldName​(@NonNull
                                                     @NonNull SystemSpec systemData)

      • extractSchemaFields

        public static java.util.List<org.apache.avro.Schema.Field> extractSchemaFields​(java.lang.String schemaContent)

      • getDataTypeFor

        public static org.apache.spark.sql.types.DataType getDataTypeFor​(org.apache.avro.Schema schema)

      • getHbaseTypeNameFor

        public static java.lang.String getHbaseTypeNameFor​(org.apache.avro.Schema schema)

      • getStructSchemaFor

        public static org.apache.spark.sql.types.StructType getStructSchemaFor​(java.util.Collection<cern.nxcals.common.domain.ColumnMapping> fields)

      • createEmptyDataFrame

        public static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> createEmptyDataFrame​(org.apache.spark.sql.SparkSession session,
                                                                                          java.util.Collection<cern.nxcals.common.domain.ColumnMapping> mappings)

      • toColumn

        public static org.apache.spark.sql.Column toColumn​(cern.nxcals.common.domain.ColumnMapping mapping)