Problem to open grib2 Nomad/CFSR data with opengrads

Fri Feb 12 11:52:30 EST 2010

Hello,

I need to take back a few statements I made in
my previous post.

I have gotten several CFSR/CFSRMON files to plot successfully.
Here are some samples:

http://nomads.ncdc.noaa.gov/data/cfsr/200001/wnd10m.gdas.200001.grb2
http://nomads.ncdc.noaa.gov/data/cfsr/200001/wnd10m.gdas.200001.ctl
http://nomads.ncdc.noaa.gov/data/cfsr/200001/pwat.gdas.200001.grb2.idx

http://nomads.ncdc.noaa.gov/data/cfsr/200001/pwat.gdas.200001.grb2
http://nomads.ncdc.noaa.gov/data/cfsr/200001/pwat.gdas.200001.ctl
http://nomads.ncdc.noaa.gov/data/cfsr/200001/pwat.gdas.200001.grb2.idx

http://nomads.ncdc.noaa.gov/data/cfsrmon/200001/ipvh02.gdas.200001.ctl
http://nomads.ncdc.noaa.gov/data/cfsrmon/200001/ipvh02.gdas.200001.grb2.idx
http://nomads.ncdc.noaa.gov/data/cfsrmon/200001/ipvh02.gdas.200001.grb2

http://nomads.ncdc.noaa.gov/data/cfsrmon/200909/ocnf04.gdas.200909.06Z.grb2
http://nomads.ncdc.noaa.gov/data/cfsrmon/200909/ocnf04.gdas.200909.06Z.ctl
http://nomads.ncdc.noaa.gov/data/cfsrmon/200909/ocnf04.gdas.200909.06Z.grb2.idx

The -b should be used when invoking GrADS2 gribmap.
The -0 opt should be set when working with monthly means
  (cfsrmon) and diurnal fraction monthlies.

Here are the known gotchas:
wgrib2 v.1.8.2 or earlier may produce control files
with incorrect TDEF's when used with g2ctl.pl .

The original CFSR was produced in GRIB1 at NCEP.
NCEP converted this data to GRIB2 for transmission
and storage purposes.  There are possible issues
in the translation between GRIB1 and GRIB2 [indicator
of statistical process] header.  This affects monthly
means (which are all statistically processed in some way).
gribmap's -0 option nulls this *potential* issue by matching
using the reference time.  Again, this issue is unproven.

The GRIB2 headers contain local extensions
used by Table 4.10 (-fix_ncep wgrib2 option).
I had thought this might cause issues with gribmap
  but am now not certain.

-Dan

Henrique Barbosa wrote the following on 2/10/2010 1:59 PM:
> Dear Dan Swank,
> 
> Thank you for clarifying the problem. The difference between GDS and TDS
> servers was not clear to me. I have not mentioned in my previous email,
> but I already tried to download the data and write a ctl...
> ... and I got the time coordinate error you mentioned!
> 
>> I hope the GrADS dev-team will take some time to get the
>> CFSR working in GrADS.
> 
> I can only enforce your request. +1 here.
> 
> []'s
> Henrique
> 
> On Tue, Feb 9, 2010 at 6:34 PM, Dan Swank <Dan.Swank at noaa.gov> wrote:
>> Hello,
>>
>> GrADS in general is not ready for the CFSR dataset.
>> I was able to use gradsdap alpha5 to sdfopen the source
>>  you listed ~ but this is a THREDDS server, which is
>>  presenting variable names which are too long for
>>  grads to handle.  [(q file) snipit]:
>>
>>     relative_humidi  0  t,y,x  Relative humidity @ sigma
>>     relative_humidi  0  t,y,x  Relative humidity @ sigma_layer
>>     relative_humidi  0  t,y,x  Relative humidity @ zeroDegC_isotherm
>>     specific_humidi  0  ty,x  Specific humidity @ pressure
>>     specific_humidi  0  ty,x  Specific humidity @ pressure_difference_layer
>>     storm_relative_  0  ty,x  Storm relative helicity @
>> height_above_ground_layer
>>     stream_function  0  ty,x  Stream function @ pressure
>>     surface_lifted_  0  t,y,x  Surface Lifted Index @ surface
>>     temperature  0  ty,x  Temperature @ pressure
>>     temperature_alt  4  t,z,y,x  Temperature @ altitude_above_msl
>>     temperature_pot  0  ty,x  Temperature @ potential_vorticity_surface
>>     temperature_max  0  t,y,x  Temperature @ maximum_wind
>>
>> So even if you managed to open it, you wouldn't get much further.
>>
>> Connecting a GrADS client to a THREDDS server has many known issues.
>> It will flat out refuse to identify the x/y coordinates when opening
>>  non-lat/lon grids served from THREDDS, for instance.  Although the
>>  source you specified IS a lat/lon, some CFSR sets are on
>>  gaussian coordinates.  The problem there is that THREDDS presents
>>  the native grid x/y's without projecting it to a lat/lon
>>  like a GDS does.
>>
>> Attempting to download a GRIB2 file and creating a control
>>  file will result in a gribmap2 not being able to MATCH the
>>  time coordinates because of the new local GRIB table the headers
>>  use.  This is especially evident in the monthly means.  This
>>  pretty much stops us from putting the CFSR behind a GDS2.
>>
>> I hope the GrADS dev-team will take some time to get the
>> CFSR working in GrADS.
>>
>> -Dan Swank
>> NCDC-NOMADS Support
>>
>>
>>
>>
>> Henrique Barbosa wrote the following on 2/9/2010 12:12 PM:
>>> Dear All,
>>>
>>> I need help to access CFSR (NCEP reanalysis 3) over opendap with opengrads.
>>> I get an horrendous error message (see below) when I try to:
>>>
>>> ga-> sdfopen http://nomads.ncdc.noaa.gov/thredds/dodsC/cfsrmon/200912/pgblnl.gdas.200912.grb2
>>>
>>> I am using v2.0.a7.oga.3 downloaded from opengrads website. I do
>>> have opendap enabled (see output of 'q config' below).
>>>
>>> The strange thing is that I can open other sources of data, for example
>>>
>>> ga-> sdfopen http://nomads.ncep.noaa.gov:9090/dods/gfs/gfs20100111/gfs_00z
>>>
>>> works perfectly and I can plot, subset, etc...
>>>
>>> Any ideas?
>>>
>>> []s
>>> Henrique
>>>
>>>
>>> --------------------------------------------------------------------------------------------------------------------------------------------------
>>> ga-> q config
>>> Config: v2.0.a7.oga.3 little-endian readline printim grib2 netcdf
>>> hdf4-sds hdf5 opendap-grids,stn athena geotiff
>>> Grid Analysis and Display System (GrADS) Version 2.0.a7.oga.3
>>> Copyright (c) 1988-2009 by Brian Doty and the
>>> Institute for Global Environment and Society (IGES)
>>> This program is distributed WITHOUT ANY WARRANTY
>>> See file COPYRIGHT for more information.
>>>
>>> Built Thu Oct 29 17:57:08 EDT 2009 for x86_64-unknown-linux-gnu
>>>
>>> This version of GrADS has been configured with the following options:
>>>   o Built on a LITTLE ENDIAN machine
>>>   o Command line editing ENABLED
>>>       http://tiswww.case.edu/php/chet/readline/rltop.html
>>>   o printim command for image output ENABLED
>>>       http://www.zlib.net
>>>       http://www.libpng.org/pub/png/libpng.html
>>>       http://www.libgd.org/Main_Page
>>>   o GRIB2 interface ENABLED
>>>       http://www.ijg.org
>>>       http://www.ece.uvic.ca/~mdadams/jasper
>>>       http://www.nco.ncep.noaa.gov/pmb/codes/GRIB2
>>>       g2clib-1.0.5
>>>   o NetCDF interface ENABLED
>>>       http://www.opendap.org
>>>       libnc-dap 4.0.1-beta3-snapshot2009021712 of Mar  3 2009 14:13:33 $
>>>   o HDF interface ENABLED
>>>       http://hdfgroup.org
>>>       HDF 4.2r3
>>>       HDF5 1.8.2
>>>   o Athena Widget GUI ENABLED
>>>   o OPeNDAP gridded data interface ENABLED
>>>       http://www.opendap.org
>>>       libdap 3.7.10
>>>   o OPeNDAP station data interface ENABLED
>>>       http://iges.org/grads/gadoc/supplibs.html
>>>       libgadap 2.0.oga.1
>>>   o GeoTIFF and KML output ENABLED
>>>       http://www.libtiff.org
>>>       http://geotiff.osgeo.org
>>>
>>> For additional information please consult http://iges.org/grads
>>>
>>> --------------------------------------------------------------------------------------------------------------------------------------------------
>>> ga-> sdfopen http://nomads.ncdc.noaa.gov/thredds/dodsC/cfsrmon/2001912/pgblnl.gdas.200912.grb2
>>> Scanning self-describing file:
>>> http://nomads.ncdc.noaa.gov/thredds/dodsC/cfsrmon/200912/pgblnl.gdas.200912.grb2
>>> *** glibc detected *** grads: free(): invalid pointer: 0x00000000018ff354 ***
>>> ======= Backtrace: =========
>>> /lib/libc.so.6[0x2b6884e9fcb8]
>>> /lib/libc.so.6(cfree+0x76)[0x2b6884ea2276]
>>> grads(_Z15convert_nc_type7nc_typeiiPvS0_+0x2ca)[0x8011ea]
>>> grads(_ZN7NCArray14extract_valuesEPviiRK12ClientParams+0x2e9)[0x833a05]
>>> grads(_Z8DODvarioiiPKmS0_PKlPvi+0xc6e)[0x803aea]
>>> grads(nc3_get_vara_double+0x9e)[0x80e3b8]
>>> grads(nc_get_vara_double+0x1a)[0x60bcca]
>>> grads(read_one_dimension+0x89)[0x541ac1]
>>> grads(gadsdf+0x17d6)[0x540080]
>>> grads(gasdfopen+0x1d4)[0x53e700]
>>> grads(gacmd+0xfb2)[0x524504]
>>> grads(Main+0x4e1)[0x496f01]
>>> grads(main+0x2e)[0x496a16]
>>> /lib/libc.so.6(__libc_start_main+0xe6)[0x2b6884e465a6]
>>> grads(__gxx_personality_v0+0x1a9)[0x496959]
>>> ======= Memory map: ========
>>> 00400000-00cca000 r-xp 00000000 08:01 604869
>>>   /usr/local/grads-2.0.a7.oga.3/bin/grads
>>> 00dc9000-00dea000 rwxp 008c9000 08:01 604869
>>>   /usr/local/grads-2.0.a7.oga.3/bin/grads
>>> 00dea000-010aa000 rwxp 00dea000 00:00 0
>>> 0180f000-01a04000 rwxp 0180f000 00:00 0                                  [heap]
>>> 2b6882c7a000-2b6882c9a000 r-xp 00000000 08:01 83056
>>>   /lib/ld-2.9.so
>>> 2b6882c9a000-2b6882c9d000 rwxp 2b6882c9a000 00:00 0
>> --
>> Dan Swank
>> STG, Incorporated - Government Contractor
>> NCDC-NOMADS Project:  Software & Data Management
>> Data Access Branch
>> National Climatic Data Center
>> Veach-Baley Federal Building
>> 151 Patton Avenue
>> Asheville, NC 28801-5001
>> Email: Dan.Swank at noaa.gov
>> Phone: 828-271-4007
>>

-- 
Dan Swank
STG, Incorporated - Government Contractor
NCDC-NOMADS Project:  Software & Data Management
Data Access Branch
National Climatic Data Center
Veach-Baley Federal Building
151 Patton Avenue
Asheville, NC 28801-5001
Email: Dan.Swank at noaa.gov
Phone: 828-271-4007