[gradsusr] Creating Data Descriptor File For HDF5 Formatted Data
jimbo32886 at gmail.com
jimbo32886 at gmail.com
Thu Feb 2 12:05:48 EST 2012
Jennifer,
Thanks for the information and analogy! I have no further questions
regarding this data set then. Thanks to everyone for your help!
-Jim
On Thu, Feb 2, 2012 at 11:52 AM, Jennifer Adams <jma at cola.iges.org> wrote:
> The message is harmless. Before GrADS does the I/O, it checks the chunk
> size to make sure the cache is big enough. The message comes out of the HDF
> library because this data set is not chunked and we have asked for its
> chunk size. Not using the chunking in HDF5 is like owning a Ferrari and
> only using it to drive to the grocery store. :-)
> --Jennifer
>
>
> On Feb 2, 2012, at 11:25 AM, James E. Johnson wrote:
>
> On Thursday, February 02, 2012 11:10:49 am jimbo32886 at gmail.com wrote:
>
> James,
>
>
> Thank you very much for the information. I just had to modify a few
>
> things in the control file you had sent me, but it appears to be working
>
> properly. The data set does indeed contain salinity values and not
>
> indices. I do get one error when I first display the data. This is the
>
> error I get:
>
>
> ga-> d sss
>
> HDF5-DIAG: Error detected in HDF5 (1.8.4-patch1) thread 0:
>
> #000: H5Pdcpl.c line 1032 in H5Pget_chunk(): not a chunked storage layout
>
> major: Invalid arguments to routine
>
> minor: Bad value
>
> Contouring at clevs = 25 26 27 28 29 30 31 32 33 34 35 36 37 38
>
>
> After the initial display of the data, the error does not seem to pop up
>
> again. Do you have any idea what this error is referring to, and is it
>
> something to worry about as the data does look to be properly displayed.
>
>
> Thanks,
>
> Jim
>
>
>
>
> Not sure, could be a bug in the Grads code that doesn't like something in
> the
> HDF5 file. Apparently its looking for chunked layout info, and can't find
> it
> and prints an error message. But you say it continues to work, so I guess
> its
> not a fatal flaw. Perhaps have the Grads developers look into the error.
>
>
> ------------------------------------------------------------------------------
> | James E. Johnson | address:
> |
> | Wyle Information Systems, Inc. | NASA Goddard Space Flight
> Center |
> | e-mail: james.johnson at nasa.gov | Distributed Active Archive
> Center |
> | phone: 301-614-5121 | Code 610.2, Bldg 32, Room S130G
> |
> | fax: 301-614-5268 | Greenbelt, MD 20771
> |
>
> ------------------------------------------------------------------------------
>
>
>
> On Thu, Feb 2, 2012 at 10:36 AM, James E. Johnson <James.Johnson at nasa.gov>
>
> wrote:
>
>
> This is not a particularly hard file to write a .ctl file for as it
>
> contains but a single data set array (variable in netCDF speak) named
>
> "l3m_data", you could have done a h5dump -h
>
> Q20112132011243.L3m_MO_EVSCI_V1.2_SSS_1deg | grep DATASET to find it. The
>
> second data set array (variable) is the color palette that the Aquarius
>
> team created for visualizing the data. You could use that to set up a new
>
> grads color table, or just use grads' default color table. But its also
>
> possible that the data are intended to be viewed as an image, since they
>
> included the palette. In that case they should have used the HDF5 image
>
> calls to create the file so that it would be tagged as an image. The
>
> values in the l3m_data array would be the indices to the color palette,
>
> but without the data array to look at I can't say if the values are
>
> indices or actually salinity values.
>
>
> This .ctl should read in the data:
>
>
> dset Q20112132011243.L3m_MO_EVSCI_V1.2_SSS_1deg
>
> dtype hdf5_grid
>
> title Aquarius Level 3 Surface Salinity
>
> xdef 360 linear -180 1.0
>
> ydef 180 linear -90 1.0
>
> tdef 1 linear 1Aug2011 1mn
>
> vars 1
>
> l3m_data=>dat 0 y,x Surface Salinity
>
> endvars
>
>
>
> --------------------------------------------------------------------------
>
> ----
>
>
> | James E. Johnson | address:
>
> | | Wyle Information Systems, Inc. | NASA Goddard Space Flight
>
> | Center | e-mail: james.johnson at nasa.gov | Distributed Active
>
> | Archive Center | phone: 301-614-5121 | Code 610.2,
>
> | Bldg 32, Room S130G | fax: 301-614-5268 |
>
> | Greenbelt, MD 20771 |
>
>
> --------------------------------------------------------------------------
>
> ----
>
>
> On Thursday, February 02, 2012 09:28:02 am jimbo32886 at gmail.com wrote:
>
> Hello,
>
>
> I had a question for anyone who has had experience taking the header
>
> output from an HDF5 data file and then constructing a control file based
>
> off of that information. I am very new to the HDF5 file construct and
>
> could certainly use some help. I am working with the Level 3 mapped Sea
>
> Surface Salinity data from the Aquarius mission. I am using GrADS
>
> version 2.0.a9.oga.1. The "h5dump" for the data file is as follows:
>
>
> HDF5 "Q20112132011243.L3m_MO_EVSCI_V1.2_SSS_1deg" {
>
> GROUP "/" {
>
> ATTRIBUTE "Data Bins" {
>
> DATATYPE H5T_STD_I32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Data Center" {
>
> DATATYPE H5T_STD_I8LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Data Maximum" {
>
> DATATYPE H5T_IEEE_F32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Data Minimum" {
>
> DATATYPE H5T_IEEE_F32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Easternmost Longitude" {
>
> DATATYPE H5T_IEEE_F32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "End Day" {
>
> DATATYPE H5T_STD_I16LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "End Millisec" {
>
> DATATYPE H5T_STD_I32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "End Orbit" {
>
> DATATYPE H5T_STD_I32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "End Time" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 17;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "End Year" {
>
> DATATYPE H5T_STD_I16LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Input Files" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 39;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Input Parameters" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 422;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Intercept" {
>
> DATATYPE H5T_IEEE_F32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "L2 Flag Names" {
>
> DATATYPE H5T_STD_I8LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Latitude Step" {
>
> DATATYPE H5T_IEEE_F32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Latitude Units" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 14;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Longitude Step" {
>
> DATATYPE H5T_IEEE_F32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Longitude Units" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 13;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Map Projection" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 24;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Measure" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 5;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Mission" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 15;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Mission Characteristics" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 137;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Northernmost Latitude" {
>
> DATATYPE H5T_IEEE_F32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Number of Columns" {
>
> DATATYPE H5T_STD_I32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Number of Lines" {
>
> DATATYPE H5T_STD_I32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Orbit" {
>
> DATATYPE H5T_STD_I32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Parameter" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 21;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Period End Day" {
>
> DATATYPE H5T_STD_I16LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Period End Year" {
>
> DATATYPE H5T_STD_I16LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Period Start Day" {
>
> DATATYPE H5T_STD_I16LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Period Start Year" {
>
> DATATYPE H5T_STD_I16LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Processing Control" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 60;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Processing Time" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 17;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Processing Version" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 5;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Product Name" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 43;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Product Type" {
>
> DATATYPE H5T_STD_I8LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "SW Point Latitude" {
>
> DATATYPE H5T_IEEE_F32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "SW Point Longitude" {
>
> DATATYPE H5T_IEEE_F32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Scaling" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 7;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Scaling Equation" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 47;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Sensor" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 9;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Sensor Characteristics" {
>
> DATATYPE H5T_STD_I8LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Sensor Name" {
>
> DATATYPE H5T_STD_I8LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Slope" {
>
> DATATYPE H5T_IEEE_F32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Software Name" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 7;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Software Version" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 5;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Southernmost Latitude" {
>
> DATATYPE H5T_IEEE_F32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Start Day" {
>
> DATATYPE H5T_STD_I16LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Start Millisec" {
>
> DATATYPE H5T_STD_I32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Start Orbit" {
>
> DATATYPE H5T_STD_I32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Start Time" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 17;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Start Year" {
>
> DATATYPE H5T_STD_I16LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Station Latitude" {
>
> DATATYPE H5T_IEEE_F32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Station Longitude" {
>
> DATATYPE H5T_IEEE_F32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Station Name" {
>
> DATATYPE H5T_STD_I8LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Suggested Image Scaling Applied" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 3;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Suggested Image Scaling Maximum" {
>
> DATATYPE H5T_IEEE_F32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Suggested Image Scaling Minimum" {
>
> DATATYPE H5T_IEEE_F32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Suggested Image Scaling Type" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 5;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Title" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 31;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Units" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 4;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Westernmost Longitude" {
>
> DATATYPE H5T_IEEE_F32LE
>
> DATASPACE SCALAR
>
> }
>
> DATASET "l3m_data" {
>
> DATATYPE H5T_IEEE_F32LE
>
> DATASPACE SIMPLE { ( 180, 360 ) / ( 180, 360 ) }
>
> ATTRIBUTE "Intercept" {
>
> DATATYPE H5T_IEEE_F32LE
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Scaling" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 7;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Scaling Equation" {
>
> DATATYPE H5T_STRING {
>
> STRSIZE 47;
>
> STRPAD H5T_STR_NULLTERM;
>
> CSET H5T_CSET_ASCII;
>
> CTYPE H5T_C_S1;
>
> }
>
> DATASPACE SCALAR
>
> }
>
> ATTRIBUTE "Slope" {
>
> DATATYPE H5T_IEEE_F32LE
>
> DATASPACE SCALAR
>
> }
>
> }
>
> DATASET "palette" {
>
> DATATYPE H5T_STD_U8LE
>
> DATASPACE SIMPLE { ( 3, 256 ) / ( 3, 256 ) }
>
> }
>
> }
>
> }
>
>
>
>
> Any input would be greatly appreciated!
>
>
> Thanks,
>
>
>
> ------------------------------------------------------------------------------
> | James E. Johnson | address:
> |
> | Wyle Information Systems, Inc. | NASA Goddard Space Flight
> Center |
> | e-mail: james.johnson at nasa.gov | Distributed Active Archive
> Center |
> | phone: 301-614-5121 | Code 610.2, Bldg 32, Room S130G
> |
> | fax: 301-614-5268 | Greenbelt, MD 20771
> |
>
> ------------------------------------------------------------------------------
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
>
>
> --
> Jennifer M. Adams
> IGES/COLA
> 4041 Powder Mill Road, Suite 302
> Calverton, MD 20705
> jma at cola.iges.org
>
>
>
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20120202/ca9ce1ef/attachment-0003.html
More information about the gradsusr
mailing list