Class ExtractionUtils


  • public final class ExtractionUtils
    extends java.lang.Object
    • Field Summary

      Fields 
      Modifier and Type Field Description
      static org.apache.avro.Schema STRING_SCHEMA  
    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> createEmptyDataFrame​(org.apache.spark.sql.SparkSession session, java.util.Collection<cern.nxcals.common.domain.ColumnMapping> mappings)  
      static java.util.List<org.apache.avro.Schema.Field> extractSchemaFields​(java.lang.String schemaContent)  
      static org.apache.spark.sql.types.DataType getDataTypeFor​(org.apache.avro.Schema schema)  
      static java.lang.String getHbaseTypeNameFor​(org.apache.avro.Schema schema)  
      static org.apache.spark.sql.types.StructType getStructSchemaFor​(java.util.Collection<cern.nxcals.common.domain.ColumnMapping> fields)  
      static java.lang.String getTimestampFieldName​(@NonNull SystemSpec systemData)  
      static org.apache.spark.sql.Column toColumn​(cern.nxcals.common.domain.ColumnMapping mapping)  
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Field Detail

      • STRING_SCHEMA

        public static final org.apache.avro.Schema STRING_SCHEMA
    • Method Detail

      • getTimestampFieldName

        public static java.lang.String getTimestampFieldName​(@NonNull
                                                             @NonNull SystemSpec systemData)
      • extractSchemaFields

        public static java.util.List<org.apache.avro.Schema.Field> extractSchemaFields​(java.lang.String schemaContent)
      • getDataTypeFor

        public static org.apache.spark.sql.types.DataType getDataTypeFor​(org.apache.avro.Schema schema)
      • getHbaseTypeNameFor

        public static java.lang.String getHbaseTypeNameFor​(org.apache.avro.Schema schema)
      • getStructSchemaFor

        public static org.apache.spark.sql.types.StructType getStructSchemaFor​(java.util.Collection<cern.nxcals.common.domain.ColumnMapping> fields)
      • createEmptyDataFrame

        public static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> createEmptyDataFrame​(org.apache.spark.sql.SparkSession session,
                                                                                                  java.util.Collection<cern.nxcals.common.domain.ColumnMapping> mappings)
      • toColumn

        public static org.apache.spark.sql.Column toColumn​(cern.nxcals.common.domain.ColumnMapping mapping)