[gradsusr] netcdf group variable

Tue Nov 21 10:48:35 EST 2023

Hi, Mahakur --
Fortunately, you are looking at MLS data, which is archived here at GES DISC, so I was able to check the granule you are trying to access with GrADS. The error you are seeing is because of a space in the Group name in that file, which must be aliased with a ~ in the descriptor. The other issue is that the chunking parameters are set so that the entire variable is one chunk:

        float value(time, lev, lon, lat) ;
                string value:long_name = "Average Value of Cloud Ice" ;
                string value:units = "mg/m!U3!N" ;
                value:_FillValue = -999.99f ;
                string value:comments = "The average value in a bin" ;
                value:_Storage = "chunked" ;
                value:_ChunkSizes = 1, 29, 72, 45 ;
                value:_DeflateLevel = 5 ;
                value:_Endianness = "little" ;

This requires you to override the default chunk cache size – otherwise you would get a warning message like this:
*** WARNING! ***
*** The I/O for this variable will be extremely slow
*** because the chunks are too big to fit in the cache
*** chunk size = 375840 bytes  (45 * 72 * 29 * 1 * 4 bytes)
*** cache size = 25920 bytes  (scale factor)*(file cachesize)
*** There are two ways to control the cache size:
*** add a CACHESIZE entry to the descriptor file
*** or change the scale factor with 'set cachesf'
*** Please read http://cola.gmu.edu/grads/gadoc/compression.html

So … below is a working descriptor file with changes highlighted in green:

dset ^MLS-Aura_L3DB-IWC_v04-24-c01_2022d365.nc
title MLS IWC
dtype hdf5_grid
cachesize 500000
undef -999.99
xdef 72 linear -177.5   5.0
ydef 45 linear -88.0    4.0
zdef 29 levels 1000, 825.4042, 681.2921, 562.3413, 464.1589, 383.1187, 316.2278, 261.0157, 215.4435, 177.8279, 146.7799, 121.1528, 100, 82.54042, 68.1292, 56.23413, 46.41589, 38.31187, 31.62278, 26.10157, 21.54435, 17.78279, 14.67799, 12.11528, 10, 1, 0.1, 0.01, 0.001
tdef 1 linear 31DEC2022 1dy
vars 1
/IWC~PressureGrid/value=>val 29 t,z,x,y IWC (mg/m3)
endvars

--
Jennifer Miletta Adams
Senior Scientific Software Developer
Goddard Earth Sciences Data and Information Services Center (GES DISC)
NASA/GSFC, Code 619
Building 32, Room S159

From: gradsusr <gradsusr-bounces at gradsusr.org> on behalf of M.Mahakur <mmahakur at tropmet.res.in>
Date: Tuesday, November 21, 2023 at 10:04 AM
To: GrADS Users Forum <gradsusr at gradsusr.org>
Subject: Re: [gradsusr] [Possibly Spoofed] Re: [EXTERNAL] [BULK] netcdf group variable
CAUTION: This email originated from outside of NASA.  Please take care when clicking links or opening attachments.  Use the "Report Message" button to report suspicious messages to the NASA SOC.

Respected Jennifer,
Hi, Good morning!

Thanks for your help. I changed the 'dtype' to 'hdf5_grid'. When I opened it i am getting errors while displaying 'val', the parameter; it says not in HDF5 file!
Please suggest..

ga-> d val
HDF5-DIAG: Error detected in HDF5 (1.8.4-patch1) thread 0:
  #000: H5D.c line 321 in H5Dopen2(): not found
    major: Dataset
    minor: Object not found
  #001: H5Gloc.c line 468 in H5G_loc_find(): can't find object
    major: Symbol table
    minor: Object not found
  #002: H5Gtraverse.c line 877 in H5G_traverse(): internal path traversal failed
    major: Symbol table
    minor: Object not found
  #003: H5Gtraverse.c line 776 in H5G_traverse_real(): component not found
    major: Symbol table
    minor: Object not found
Error: H5Dopen2 failed for variable /IWC/ PressureGrid/value
Error: Variable /IWC/ PressureGrid/value not in HDF5 file
Data Request Error:  Error for variable 'val'
  Error ocurred at column 1
DISPLAY error:  Invalid expression
  Expression = val
ga->

ga-> q file
File 1 : MLS IWC
  Descriptor: mls.ctl
  Binary: MLS-Aura_L3DB-IWC_v04-24-c01_2022d365.nc
  Type = Gridded
  Xsize = 72  Ysize = 45  Zsize = 29  Tsize = 1  Esize = 1
  Number of Variables = 1
     val  29  t,z,x,y  IWC (mg/m3)

The ctl file:

dset MLS-Aura_L3DB-IWC_v04-24-c01_2022d365.nc
title MLS IWC
dtype hdf5_grid
undef -999.99
xdef 72 linear -177.5   5.0
ydef 45 linear -88.0    4.0
zdef 29 levels 1000, 825.4042, 681.2921, 562.3413, 464.1589, 383.1187, 316.2278, 261.0157, 215.4435, 177.8279, 146.7799, 121.1528, 100, 82.54042, 68.1292, 56.23413, 46.41589, 38.31187, 31.62278, 26.10157, 21.54435, 17.78279, 14.67799, 12.11528, 10, 1, 0.1, 0.01, 0.001
tdef 1 linear 31DEC2022 1dy
vars 1
/IWC/PressureGrid/value=>val 29 t,z,x,y IWC (mg/m3)
endvars

With Regards and Thanks,
Mahakur

________________________________
From: "Adams, Jennifer M. (GSFC-619.0)[ADNET SYSTEMS INC]" <jennifer.m.adams at nasa.gov>
To: "GrADS Users Forum" <gradsusr at gradsusr.org>
Sent: Tuesday, November 21, 2023 1:22:58 AM
Subject: [Possibly Spoofed] Re: [gradsusr] [EXTERNAL] [BULK]  netcdf group variable

It should work if you change:
dtype netcdf
to
dtype hdf5_grid

The HDF5 interface does support groups, and netcdf4 is really an hdf5 file under the hood.
--Jennifer
--
Jennifer Miletta Adams
Senior Scientific Software Developer
Goddard Earth Sciences Data and Information Services Center (GES DISC)
NASA/GSFC, Code 619
Building 32, Room S159

From: gradsusr <gradsusr-bounces at gradsusr.org> on behalf of M.Mahakur <mmahakur at tropmet.res.in>
Date: Monday, November 20, 2023 at 2:43 PM
To: GrADS Users Forum <gradsusr at gradsusr.org>
Subject: [EXTERNAL] [BULK] [gradsusr] netcdf group variable
CAUTION: This email originated from outside of NASA.  Please take care when clicking links or opening attachments.  Use the "Report Message" button to report suspicious messages to the NASA SOC.

Dear All,

I like to display the variable "value" from the attached small netcdf file. I feel the pressure level variable is in the group 'IWC/PressureGrid'. For this I created the ctl :

dset MLS-Aura_L3DB-IWC_v04-24-c01_2022d365.nc
title MLS IWC
dtype netcdf
undef -999.99
xdef 72 linear -177.5   5.0
ydef 45 linear -88.0    4.0
zdef 29 levels 1000, 825.4042, 681.2921, 562.3413, 464.1589, 383.1187, 316.2278, 261.0157, 215.4435, 177.8279, 146.7799, 121.1528, 100, 82.54042, 68.1292, 56.23413, 46.41589, 38.31187, 31.62278, 26.10157, 21.54435, 17.78279, 14.67799, 12.11528, 10, 1, 0.1, 0.01, 0.001
tdef 1 linear 1DEC2022 1dy
vars 1
/IWC/PressureGrid/value=>val 29 t,z,x,y IWC (mg/m3)
endvars
But, it does not works. It seems it does not matches with the variable!

ga-> d val
Error: Variable /IWC/PressureGrid/value not in netcdf file
Data Request Error:  Error for variable 'val'
  Error ocurred at column 1
DISPLAY error:  Invalid expression
  Expression = val

i am using Grads 2.0.a9. A portion from the ncdump is:

=========================

group: IWC\ PressureGrid {
  dimensions:
        lat = 45 ;
        lon = 72 ;
        lev = 29 ;
        time = 1 ;
        nv = 2 ;
  variables:

....
...

float value(time, lev, lon, lat) ;
                string value:long_name = "Average Value of Cloud Ice" ;
                string value:units = "mg/m!U3!N" ;
                value:_FillValue = -999.99f ;
                string value:comments = "The average value in a bin" ;
============================
Please suggest, where to modify in the ctl.

Thanks and Regards,

Mahakur

_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org
http://gradsusr.org/mailman/listinfo/gradsusr

--
M. Mahakur
Scientist
Radar and Satellite Meteorology Project (PDTC)
Room No. S127, Main Bldg
INDIAN INSTITUTE OF TROPICAL  METEOROLOGY
(Ministry of Earth Sciences, Govt. of India)
Dr. Homi Bhabha Road, NCL Post, Pashan
PUNE - 411 008 (INDIA)

Mob: +91-8806861895
Tel: (O) +91-20-259 04 225
FAX: +91-20-25865142

E-mail: mmahakur at tropmet.res.in
         mahakur.iitm at gmail.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gradsusr.org/pipermail/gradsusr/attachments/20231121/50d635c1/attachment-0001.html>