[gradsusr] Creating Data Descriptor File For HDF5 Formatted Data
Jennifer Adams
jma at cola.iges.org
Thu Feb 2 11:52:03 EST 2012
The message is harmless. Before GrADS does the I/O, it checks the chunk size to make sure the cache is big enough. The message comes out of the HDF library because this data set is not chunked and we have asked for its chunk size. Not using the chunking in HDF5 is like owning a Ferrari and only using it to drive to the grocery store. :-)
--Jennifer
On Feb 2, 2012, at 11:25 AM, James E. Johnson wrote:
> On Thursday, February 02, 2012 11:10:49 am jimbo32886 at gmail.com wrote:
>> James,
>>
>> Thank you very much for the information. I just had to modify a few
>> things in the control file you had sent me, but it appears to be working
>> properly. The data set does indeed contain salinity values and not
>> indices. I do get one error when I first display the data. This is the
>> error I get:
>>
>> ga-> d sss
>> HDF5-DIAG: Error detected in HDF5 (1.8.4-patch1) thread 0:
>> #000: H5Pdcpl.c line 1032 in H5Pget_chunk(): not a chunked storage layout
>> major: Invalid arguments to routine
>> minor: Bad value
>> Contouring at clevs = 25 26 27 28 29 30 31 32 33 34 35 36 37 38
>>
>> After the initial display of the data, the error does not seem to pop up
>> again. Do you have any idea what this error is referring to, and is it
>> something to worry about as the data does look to be properly displayed.
>>
>> Thanks,
>> Jim
>>
>
>
> Not sure, could be a bug in the Grads code that doesn't like something in the
> HDF5 file. Apparently its looking for chunked layout info, and can't find it
> and prints an error message. But you say it continues to work, so I guess its
> not a fatal flaw. Perhaps have the Grads developers look into the error.
>
> ------------------------------------------------------------------------------
> | James E. Johnson | address: |
> | Wyle Information Systems, Inc. | NASA Goddard Space Flight Center |
> | e-mail: james.johnson at nasa.gov | Distributed Active Archive Center |
> | phone: 301-614-5121 | Code 610.2, Bldg 32, Room S130G |
> | fax: 301-614-5268 | Greenbelt, MD 20771 |
> ------------------------------------------------------------------------------
>
>>
>>
>> On Thu, Feb 2, 2012 at 10:36 AM, James E. Johnson <James.Johnson at nasa.gov>
>> wrote:
>>
>> This is not a particularly hard file to write a .ctl file for as it
>> contains but a single data set array (variable in netCDF speak) named
>> "l3m_data", you could have done a h5dump -h
>> Q20112132011243.L3m_MO_EVSCI_V1.2_SSS_1deg | grep DATASET to find it. The
>> second data set array (variable) is the color palette that the Aquarius
>> team created for visualizing the data. You could use that to set up a new
>> grads color table, or just use grads' default color table. But its also
>> possible that the data are intended to be viewed as an image, since they
>> included the palette. In that case they should have used the HDF5 image
>> calls to create the file so that it would be tagged as an image. The
>> values in the l3m_data array would be the indices to the color palette,
>> but without the data array to look at I can't say if the values are
>> indices or actually salinity values.
>>
>> This .ctl should read in the data:
>>
>> dset Q20112132011243.L3m_MO_EVSCI_V1.2_SSS_1deg
>> dtype hdf5_grid
>> title Aquarius Level 3 Surface Salinity
>> xdef 360 linear -180 1.0
>> ydef 180 linear -90 1.0
>> tdef 1 linear 1Aug2011 1mn
>> vars 1
>> l3m_data=>dat 0 y,x Surface Salinity
>> endvars
>>
>>
>> --------------------------------------------------------------------------
>> ----
>>
>> | James E. Johnson | address:
>> | | Wyle Information Systems, Inc. | NASA Goddard Space Flight
>> | Center | e-mail: james.johnson at nasa.gov | Distributed Active
>> | Archive Center | phone: 301-614-5121 | Code 610.2,
>> | Bldg 32, Room S130G | fax: 301-614-5268 |
>> | Greenbelt, MD 20771 |
>>
>> --------------------------------------------------------------------------
>> ----
>>
>> On Thursday, February 02, 2012 09:28:02 am jimbo32886 at gmail.com wrote:
>>> Hello,
>>>
>>> I had a question for anyone who has had experience taking the header
>>> output from an HDF5 data file and then constructing a control file based
>>> off of that information. I am very new to the HDF5 file construct and
>>> could certainly use some help. I am working with the Level 3 mapped Sea
>>> Surface Salinity data from the Aquarius mission. I am using GrADS
>>> version 2.0.a9.oga.1. The "h5dump" for the data file is as follows:
>>>
>>> HDF5 "Q20112132011243.L3m_MO_EVSCI_V1.2_SSS_1deg" {
>>> GROUP "/" {
>>> ATTRIBUTE "Data Bins" {
>>> DATATYPE H5T_STD_I32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Data Center" {
>>> DATATYPE H5T_STD_I8LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Data Maximum" {
>>> DATATYPE H5T_IEEE_F32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Data Minimum" {
>>> DATATYPE H5T_IEEE_F32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Easternmost Longitude" {
>>> DATATYPE H5T_IEEE_F32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "End Day" {
>>> DATATYPE H5T_STD_I16LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "End Millisec" {
>>> DATATYPE H5T_STD_I32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "End Orbit" {
>>> DATATYPE H5T_STD_I32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "End Time" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 17;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "End Year" {
>>> DATATYPE H5T_STD_I16LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Input Files" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 39;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Input Parameters" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 422;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Intercept" {
>>> DATATYPE H5T_IEEE_F32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "L2 Flag Names" {
>>> DATATYPE H5T_STD_I8LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Latitude Step" {
>>> DATATYPE H5T_IEEE_F32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Latitude Units" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 14;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Longitude Step" {
>>> DATATYPE H5T_IEEE_F32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Longitude Units" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 13;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Map Projection" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 24;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Measure" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 5;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Mission" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 15;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Mission Characteristics" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 137;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Northernmost Latitude" {
>>> DATATYPE H5T_IEEE_F32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Number of Columns" {
>>> DATATYPE H5T_STD_I32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Number of Lines" {
>>> DATATYPE H5T_STD_I32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Orbit" {
>>> DATATYPE H5T_STD_I32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Parameter" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 21;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Period End Day" {
>>> DATATYPE H5T_STD_I16LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Period End Year" {
>>> DATATYPE H5T_STD_I16LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Period Start Day" {
>>> DATATYPE H5T_STD_I16LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Period Start Year" {
>>> DATATYPE H5T_STD_I16LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Processing Control" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 60;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Processing Time" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 17;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Processing Version" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 5;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Product Name" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 43;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Product Type" {
>>> DATATYPE H5T_STD_I8LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "SW Point Latitude" {
>>> DATATYPE H5T_IEEE_F32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "SW Point Longitude" {
>>> DATATYPE H5T_IEEE_F32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Scaling" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 7;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Scaling Equation" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 47;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Sensor" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 9;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Sensor Characteristics" {
>>> DATATYPE H5T_STD_I8LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Sensor Name" {
>>> DATATYPE H5T_STD_I8LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Slope" {
>>> DATATYPE H5T_IEEE_F32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Software Name" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 7;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Software Version" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 5;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Southernmost Latitude" {
>>> DATATYPE H5T_IEEE_F32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Start Day" {
>>> DATATYPE H5T_STD_I16LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Start Millisec" {
>>> DATATYPE H5T_STD_I32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Start Orbit" {
>>> DATATYPE H5T_STD_I32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Start Time" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 17;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Start Year" {
>>> DATATYPE H5T_STD_I16LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Station Latitude" {
>>> DATATYPE H5T_IEEE_F32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Station Longitude" {
>>> DATATYPE H5T_IEEE_F32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Station Name" {
>>> DATATYPE H5T_STD_I8LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Suggested Image Scaling Applied" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 3;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Suggested Image Scaling Maximum" {
>>> DATATYPE H5T_IEEE_F32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Suggested Image Scaling Minimum" {
>>> DATATYPE H5T_IEEE_F32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Suggested Image Scaling Type" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 5;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Title" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 31;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Units" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 4;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Westernmost Longitude" {
>>> DATATYPE H5T_IEEE_F32LE
>>> DATASPACE SCALAR
>>> }
>>> DATASET "l3m_data" {
>>> DATATYPE H5T_IEEE_F32LE
>>> DATASPACE SIMPLE { ( 180, 360 ) / ( 180, 360 ) }
>>> ATTRIBUTE "Intercept" {
>>> DATATYPE H5T_IEEE_F32LE
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Scaling" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 7;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Scaling Equation" {
>>> DATATYPE H5T_STRING {
>>> STRSIZE 47;
>>> STRPAD H5T_STR_NULLTERM;
>>> CSET H5T_CSET_ASCII;
>>> CTYPE H5T_C_S1;
>>> }
>>> DATASPACE SCALAR
>>> }
>>> ATTRIBUTE "Slope" {
>>> DATATYPE H5T_IEEE_F32LE
>>> DATASPACE SCALAR
>>> }
>>> }
>>> DATASET "palette" {
>>> DATATYPE H5T_STD_U8LE
>>> DATASPACE SIMPLE { ( 3, 256 ) / ( 3, 256 ) }
>>> }
>>> }
>>> }
>>>
>>>
>>>
>>> Any input would be greatly appreciated!
>>>
>>> Thanks,
>
> ------------------------------------------------------------------------------
> | James E. Johnson | address: |
> | Wyle Information Systems, Inc. | NASA Goddard Space Flight Center |
> | e-mail: james.johnson at nasa.gov | Distributed Active Archive Center |
> | phone: 301-614-5121 | Code 610.2, Bldg 32, Room S130G |
> | fax: 301-614-5268 | Greenbelt, MD 20771 |
> ------------------------------------------------------------------------------
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jma at cola.iges.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20120202/8d3165f4/attachment-0003.html
More information about the gradsusr
mailing list