[gradsusr] GPM HDF5 descriptor file
Jennifer M Adams
jadams21 at gmu.edu
Tue Jun 9 11:06:19 EDT 2020
Yes, this is a problem with the HDF5_grid interface in GrADS 2.2.1. Some of the internal variables are handled as ordinary integers but need to be special (longer) data types specific to the HDF5 library. I have fixed this in my dev version except for one lingering issue where the library gets very upset if I try to open a file that doesn't exist. Since these IMERG files are distributed by the GES DIS (where I work), it is important that GrADS be able to handle them. I will work on the final fixes and publish the source code version 2.2.2 as soon as I can. --Jennifer
________________________________
From: gradsusr <gradsusr-bounces at gradsusr.org> on behalf of 姜立智 <jianglizhi at mail.iap.ac.cn>
Sent: Thursday, June 4, 2020 12:19 AM
To: gradsusr at gradsusr.org <gradsusr at gradsusr.org>
Subject: [gradsusr] GPM HDF5 descriptor file
Hello,
I would like to read the GPM IMERG precipitation data (HDF5 format) by using GrADS. After read the page(http://cola.gmu.edu/grads/gadoc/SDFdescriptorfile.html) and gradsusr mailing list(http://gradsusr.org/pipermail/gradsusr/2016-January/039366.html<http://secure-web.cisco.com/1L4INTyKjaI7OuusqLE6gCwJceCXcfOBK-pjU2yl4whiDuZULKQKCbRWiiKA8N2NKI9IkMUskQ_AorJ4c6e1Psh93NkLo0Fy_23kJnx1HbYPh3bILBwuxxlQi84sBVVnNL57DABuuQHPYaEo5s3UFt7iVUhgrwT9Q7fq-LwWtp8TfMdlW7431HuJeDoRh5OZSfZ3WAbN-vQP5BC9e21eGrJR7VBEHsOBn8gxo5nC-VPa8AncIrkkF5MV0v3TEeZ6FTOye7upkvSn7zrewZojyk8C2FAUyaS8peRUA5wEKfAkikEJrKaINtpPI1n9gxMX7tj8ANlMP245zMxg8ROO94EM7fSVUiIDl9EUVRJstg8crAvuUzyYLGnAffqH7L2Pa-3ruwqibjGHzL3s7nB7tohjsQxCBEdUqbNRY0Bwsq6Rz5XX6IAOVXUS9hLBZ9-Bd/http%3A%2F%2Fgradsusr.org%2Fpipermail%2Fgradsusr%2F2016-January%2F039366.html>), I had created the ctl file.
The ctl file can be loaded normally. However, It reports some error while I try to display some variable.
Error message:
ga-> d hqp
HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 23373517074432:
#000: ../../../src/H5D.c line 282 in H5Dopen2(): not a location
major: Invalid arguments to routine
minor: Inappropriate type
#001: ../../../src/H5Gloc.c line 246 in H5G_loc(): invalid object ID
major: Invalid arguments to routine
minor: Bad value
Error: H5Dopen2 failed for variable /Grid/HQprecipitation
Error: Variable /Grid/HQprecipitation not in HDF5 file
Data Request Error: Error for variable 'hqp'
Error ocurred at column 1
DISPLAY error: Invalid expression
Expression = hqp
Here are the config information:
ga-> q config
Config: v2.2.1 little-endian readline grib2 netcdf hdf4-sds hdf5 opendap-grids,stn geotiff shapefile
Grid Analysis and Display System (GrADS) Version 2.2.1
Copyright (C) 1988-2018 by George Mason University
GrADS comes with ABSOLUTELY NO WARRANTY
See file COPYRIGHT for more information
This build of GrADS has the following features:
-+- Byte order is LITTLE ENDIAN
-+- Athena Widget GUI DISABLED
-+- Command line editing ENABLED
-+- GRIB2 interface ENABLED g2clib-1.6.0
-+- NetCDF interface ENABLED netcdf-4.7.3
-+- OPeNDAP gridded data interface ENABLED
-+- OPeNDAP station data interface ENABLED libgadap 2.0
-+- HDF4 interface ENABLED hdf-4.2r14
-+- HDF5 interface ENABLED hdf5-1.10.4
-+- KML contour output ENABLED
-+- GeoTIFF and KML/TIFF output ENABLED
-+- Shapefile interface ENABLED
The 'q gxconfig' command returns Graphics configuration information
Here are the part of h5dump output (see the attathment h5dump.txt for full information):
HDF5 "3B-HHR.MS.MRG.3IMERG.20181113-S000000-E002959.0000.V06B.HDF5" {
GROUP "/Grid/" {
ATTRIBUTE "GridHeader" {
DATATYPE H5T_STRING {
STRSIZE 230;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
}
DATASET "HQobservationTime" {
DATATYPE H5T_STD_I16LE
DATASPACE SIMPLE { ( 1, 3600, 1800 ) / ( H5S_UNLIMITED, 3600, 1800 ) }
ATTRIBUTE "CodeMissingValue" {
DATATYPE H5T_STRING {
STRSIZE 6;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
}
ATTRIBUTE "DIMENSION_LIST" {
DATATYPE H5T_VLEN { H5T_REFERENCE { H5T_STD_REF_OBJECT }}
DATASPACE SIMPLE { ( 3 ) / ( 3 ) }
}
ATTRIBUTE "DimensionNames" {
DATATYPE H5T_STRING {
STRSIZE 13;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
}
ATTRIBUTE "Units" {
DATATYPE H5T_STRING {
STRSIZE 8;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
}
ATTRIBUTE "_FillValue" {
DATATYPE H5T_STD_I16LE
DATASPACE SCALAR
}
ATTRIBUTE "coordinates" {
DATATYPE H5T_STRING {
STRSIZE 13;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
}
ATTRIBUTE "units" {
DATATYPE H5T_STRING {
STRSIZE 8;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
}
}
DATASET "HQprecipSource" {
DATATYPE H5T_STD_I16LE
DATASPACE SIMPLE { ( 1, 3600, 1800 ) / ( H5S_UNLIMITED, 3600, 1800 ) }
ATTRIBUTE "CodeMissingValue" {
DATATYPE H5T_STRING {
STRSIZE 6;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
}
ATTRIBUTE "DIMENSION_LIST" {
DATATYPE H5T_VLEN { H5T_REFERENCE { H5T_STD_REF_OBJECT }}
DATASPACE SIMPLE { ( 3 ) / ( 3 ) }
}
ATTRIBUTE "DimensionNames" {
DATATYPE H5T_STRING {
STRSIZE 13;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
}
ATTRIBUTE "_FillValue" {
DATATYPE H5T_STD_I16LE
DATASPACE SCALAR
}
ATTRIBUTE "coordinates" {
DATATYPE H5T_STRING {
STRSIZE 13;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
}
}
DATASET "HQprecipitation" {
DATATYPE H5T_IEEE_F32LE
DATASPACE SIMPLE { ( 1, 3600, 1800 ) / ( H5S_UNLIMITED, 3600, 1800 ) }
ATTRIBUTE "CodeMissingValue" {
DATATYPE H5T_STRING {
STRSIZE 8;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
}
ATTRIBUTE "DIMENSION_LIST" {
DATATYPE H5T_VLEN { H5T_REFERENCE { H5T_STD_REF_OBJECT }}
DATASPACE SIMPLE { ( 3 ) / ( 3 ) }
}
ATTRIBUTE "DimensionNames" {
DATATYPE H5T_STRING {
STRSIZE 13;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
}
ATTRIBUTE "Units" {
DATATYPE H5T_STRING {
STRSIZE 6;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
}
ATTRIBUTE "_FillValue" {
DATATYPE H5T_IEEE_F32LE
DATASPACE SCALAR
}
ATTRIBUTE "coordinates" {
DATATYPE H5T_STRING {
STRSIZE 13;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
}
ATTRIBUTE "units" {
DATATYPE H5T_STRING {
STRSIZE 6;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
}
}
>From the output, It seem that the size of variable HQprecipitation is (1,3600,1800). So I created the ctl file below:
dset ^3B-HHR.MS.MRG.3IMERG.20181118-S233000-E235959.1410.V06B.HDF5
DTYPE hdf5_grid
TITLE IMERG sample
UNDEF -9999.9
XDEF 3600 linear -179.95 0.1
YDEF 1800 linear -89.95 0.1
ZDEF 1 levels 1 1
TDEF 1 linear 23:30Z18nov2018 30mn
VARS 5
/Grid/HQprecipitation=>hqp 0 t,x,y HQ precipitation
/Grid/IRprecipitation=>irp 0 t,x,y IR precipitation
/Grid/precipitationCal=>calp 0 t,x,y cal precipitation
/Grid/precipitationUncal=>uncalp 0 t,x,y uncal precipitation
/Grid/randomError=>error 0 t,x,y random error
ENDVARS
I am not sure how to read the HDF5 file successfully. Can anyone help? Thanks in advance.
Lizhi Jiang
June 4, 2020
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gradsusr.org/pipermail/gradsusr/attachments/20200609/1ef7dced/attachment-0001.html>
More information about the gradsusr
mailing list