Problem to open grib2 Nomad/CFSR data with opengrads

Jennifer Adams jma at COLA.IGES.ORG
Fri Feb 12 13:48:13 EST 2010


Dear All,
Everyone at COLA is interested in looking at the new CFS Reanalysis,  
so you can be sure that GrADS will be able handle it. The -b option  
for gribmap has not been implemented for grib2 yet, so that will have  
to be patched quickly. We're working on some other changes to gribmap  
to handle large (>2GB) grib files, so the timing will be just about  
right for a new release with these features. Please don't ask when, I  
don't know.

As for the CFSR data behind TDS at NCDC, GrADS works fine as a client,  
but you will likely have to create a descriptor file, especially if  
the data are not on a regular lat/lon grid and need a PDEF statement.  
The variable names are very long and need to be aliased. The time axis  
metadata from TDS may also be incompatible with 'sdfopen'. I was able  
to open the URL posted earlier in this thread and display several  
variables.

I have not had a chance to grab CFSR samples and see where the  
problems are (as far as GrADS is concerned). It sounds like it will  
depend a lot on which file/product you are looking at. I know the data  
wranglers at NCEP and NCDC are working hard to make this data  
available and usable and it's not an easy job. If you are eager to  
look at this data, make sure you are using the most recent versions of  
all the available tools. And if you post questions to the list, be  
specific about the version you're using and the operating system  
you're running on as well as the data file (or URL) you're trying to  
read.

--Jennifer

On Feb 12, 2010, at 11:52 AM, Dan Swank wrote:

> Hello,
>
> I need to take back a few statements I made in
> my previous post.
>
> I have gotten several CFSR/CFSRMON files to plot successfully.
> Here are some samples:
>
> http://nomads.ncdc.noaa.gov/data/cfsr/200001/wnd10m.gdas.200001.grb2
> http://nomads.ncdc.noaa.gov/data/cfsr/200001/wnd10m.gdas.200001.ctl
> http://nomads.ncdc.noaa.gov/data/cfsr/200001/pwat.gdas.200001.grb2.idx
>
> http://nomads.ncdc.noaa.gov/data/cfsr/200001/pwat.gdas.200001.grb2
> http://nomads.ncdc.noaa.gov/data/cfsr/200001/pwat.gdas.200001.ctl
> http://nomads.ncdc.noaa.gov/data/cfsr/200001/pwat.gdas.200001.grb2.idx
>
> http://nomads.ncdc.noaa.gov/data/cfsrmon/200001/ipvh02.gdas.200001.ctl
> http://nomads.ncdc.noaa.gov/data/cfsrmon/200001/ipvh02.gdas.200001.grb2.idx
> http://nomads.ncdc.noaa.gov/data/cfsrmon/200001/ipvh02.gdas. 
> 200001.grb2
>
> http://nomads.ncdc.noaa.gov/data/cfsrmon/200909/ocnf04.gdas.200909.06Z.grb2
> http://nomads.ncdc.noaa.gov/data/cfsrmon/200909/ocnf04.gdas.200909.06Z.ctl
> http://nomads.ncdc.noaa.gov/data/cfsrmon/200909/ocnf04.gdas.200909.06Z.grb2.idx
>
> The -b should be used when invoking GrADS2 gribmap.
> The -0 opt should be set when working with monthly means
>  (cfsrmon) and diurnal fraction monthlies.
>
> Here are the known gotchas:
> wgrib2 v.1.8.2 or earlier may produce control files
> with incorrect TDEF's when used with g2ctl.pl .
>
> The original CFSR was produced in GRIB1 at NCEP.
> NCEP converted this data to GRIB2 for transmission
> and storage purposes.  There are possible issues
> in the translation between GRIB1 and GRIB2 [indicator
> of statistical process] header.  This affects monthly
> means (which are all statistically processed in some way).
> gribmap's -0 option nulls this *potential* issue by matching
> using the reference time.  Again, this issue is unproven.
>
> The GRIB2 headers contain local extensions
> used by Table 4.10 (-fix_ncep wgrib2 option).
> I had thought this might cause issues with gribmap
>  but am now not certain.
>
>
> -Dan
>
>
>
>
> Henrique Barbosa wrote the following on 2/10/2010 1:59 PM:
>> Dear Dan Swank,
>>
>> Thank you for clarifying the problem. The difference between GDS  
>> and TDS
>> servers was not clear to me. I have not mentioned in my previous  
>> email,
>> but I already tried to download the data and write a ctl...
>> ... and I got the time coordinate error you mentioned!
>>
>>> I hope the GrADS dev-team will take some time to get the
>>> CFSR working in GrADS.
>>
>> I can only enforce your request. +1 here.
>>
>> []'s
>> Henrique
>>
>> On Tue, Feb 9, 2010 at 6:34 PM, Dan Swank <Dan.Swank at noaa.gov> wrote:
>>> Hello,
>>>
>>> GrADS in general is not ready for the CFSR dataset.
>>> I was able to use gradsdap alpha5 to sdfopen the source
>>> you listed ~ but this is a THREDDS server, which is
>>> presenting variable names which are too long for
>>> grads to handle.  [(q file) snipit]:
>>>
>>>    relative_humidi  0  t,y,x  Relative humidity @ sigma
>>>    relative_humidi  0  t,y,x  Relative humidity @ sigma_layer
>>>    relative_humidi  0  t,y,x  Relative humidity @ zeroDegC_isotherm
>>>    specific_humidi  0  ty,x  Specific humidity @ pressure
>>>    specific_humidi  0  ty,x  Specific humidity @  
>>> pressure_difference_layer
>>>    storm_relative_  0  ty,x  Storm relative helicity @
>>> height_above_ground_layer
>>>    stream_function  0  ty,x  Stream function @ pressure
>>>    surface_lifted_  0  t,y,x  Surface Lifted Index @ surface
>>>    temperature  0  ty,x  Temperature @ pressure
>>>    temperature_alt  4  t,z,y,x  Temperature @ altitude_above_msl
>>>    temperature_pot  0  ty,x  Temperature @  
>>> potential_vorticity_surface
>>>    temperature_max  0  t,y,x  Temperature @ maximum_wind
>>>
>>> So even if you managed to open it, you wouldn't get much further.
>>>
>>> Connecting a GrADS client to a THREDDS server has many known issues.
>>> It will flat out refuse to identify the x/y coordinates when opening
>>> non-lat/lon grids served from THREDDS, for instance.  Although the
>>> source you specified IS a lat/lon, some CFSR sets are on
>>> gaussian coordinates.  The problem there is that THREDDS presents
>>> the native grid x/y's without projecting it to a lat/lon
>>> like a GDS does.
>>>
>>> Attempting to download a GRIB2 file and creating a control
>>> file will result in a gribmap2 not being able to MATCH the
>>> time coordinates because of the new local GRIB table the headers
>>> use.  This is especially evident in the monthly means.  This
>>> pretty much stops us from putting the CFSR behind a GDS2.
>>>
>>> I hope the GrADS dev-team will take some time to get the
>>> CFSR working in GrADS.
>>>
>>> -Dan Swank
>>> NCDC-NOMADS Support
>>>
>>>
>>>
>>>
>>> Henrique Barbosa wrote the following on 2/9/2010 12:12 PM:
>>>> Dear All,
>>>>
>>>> I need help to access CFSR (NCEP reanalysis 3) over opendap with  
>>>> opengrads.
>>>> I get an horrendous error message (see below) when I try to:
>>>>
>>>> ga-> sdfopen http://nomads.ncdc.noaa.gov/thredds/dodsC/cfsrmon/200912/pgblnl.gdas.200912.grb2
>>>>
>>>> I am using v2.0.a7.oga.3 downloaded from opengrads website. I do
>>>> have opendap enabled (see output of 'q config' below).
>>>>
>>>> The strange thing is that I can open other sources of data, for  
>>>> example
>>>>
>>>> ga-> sdfopen http://nomads.ncep.noaa.gov:9090/dods/gfs/gfs20100111/gfs_00z
>>>>
>>>> works perfectly and I can plot, subset, etc...
>>>>
>>>> Any ideas?
>>>>
>>>> []s
>>>> Henrique
>>>>
>>>>
>>>> --------------------------------------------------------------------------------------------------------------------------------------------------
>>>> ga-> q config
>>>> Config: v2.0.a7.oga.3 little-endian readline printim grib2 netcdf
>>>> hdf4-sds hdf5 opendap-grids,stn athena geotiff
>>>> Grid Analysis and Display System (GrADS) Version 2.0.a7.oga.3
>>>> Copyright (c) 1988-2009 by Brian Doty and the
>>>> Institute for Global Environment and Society (IGES)
>>>> This program is distributed WITHOUT ANY WARRANTY
>>>> See file COPYRIGHT for more information.
>>>>
>>>> Built Thu Oct 29 17:57:08 EDT 2009 for x86_64-unknown-linux-gnu
>>>>
>>>> This version of GrADS has been configured with the following  
>>>> options:
>>>>  o Built on a LITTLE ENDIAN machine
>>>>  o Command line editing ENABLED
>>>>      http://tiswww.case.edu/php/chet/readline/rltop.html
>>>>  o printim command for image output ENABLED
>>>>      http://www.zlib.net
>>>>      http://www.libpng.org/pub/png/libpng.html
>>>>      http://www.libgd.org/Main_Page
>>>>  o GRIB2 interface ENABLED
>>>>      http://www.ijg.org
>>>>      http://www.ece.uvic.ca/~mdadams/jasper
>>>>      http://www.nco.ncep.noaa.gov/pmb/codes/GRIB2
>>>>      g2clib-1.0.5
>>>>  o NetCDF interface ENABLED
>>>>      http://www.opendap.org
>>>>      libnc-dap 4.0.1-beta3-snapshot2009021712 of Mar  3 2009  
>>>> 14:13:33 $
>>>>  o HDF interface ENABLED
>>>>      http://hdfgroup.org
>>>>      HDF 4.2r3
>>>>      HDF5 1.8.2
>>>>  o Athena Widget GUI ENABLED
>>>>  o OPeNDAP gridded data interface ENABLED
>>>>      http://www.opendap.org
>>>>      libdap 3.7.10
>>>>  o OPeNDAP station data interface ENABLED
>>>>      http://iges.org/grads/gadoc/supplibs.html
>>>>      libgadap 2.0.oga.1
>>>>  o GeoTIFF and KML output ENABLED
>>>>      http://www.libtiff.org
>>>>      http://geotiff.osgeo.org
>>>>
>>>> For additional information please consult http://iges.org/grads
>>>>
>>>> --------------------------------------------------------------------------------------------------------------------------------------------------
>>>> ga-> sdfopen http://nomads.ncdc.noaa.gov/thredds/dodsC/cfsrmon/2001912/pgblnl.gdas.200912.grb2
>>>> Scanning self-describing file:
>>>> http://nomads.ncdc.noaa.gov/thredds/dodsC/cfsrmon/200912/pgblnl.gdas.200912.grb2
>>>> *** glibc detected *** grads: free(): invalid pointer:  
>>>> 0x00000000018ff354 ***
>>>> ======= Backtrace: =========
>>>> /lib/libc.so.6[0x2b6884e9fcb8]
>>>> /lib/libc.so.6(cfree+0x76)[0x2b6884ea2276]
>>>> grads(_Z15convert_nc_type7nc_typeiiPvS0_+0x2ca)[0x8011ea]
>>>> grads(_ZN7NCArray14extract_valuesEPviiRK12ClientParams+0x2e9) 
>>>> [0x833a05]
>>>> grads(_Z8DODvarioiiPKmS0_PKlPvi+0xc6e)[0x803aea]
>>>> grads(nc3_get_vara_double+0x9e)[0x80e3b8]
>>>> grads(nc_get_vara_double+0x1a)[0x60bcca]
>>>> grads(read_one_dimension+0x89)[0x541ac1]
>>>> grads(gadsdf+0x17d6)[0x540080]
>>>> grads(gasdfopen+0x1d4)[0x53e700]
>>>> grads(gacmd+0xfb2)[0x524504]
>>>> grads(Main+0x4e1)[0x496f01]
>>>> grads(main+0x2e)[0x496a16]
>>>> /lib/libc.so.6(__libc_start_main+0xe6)[0x2b6884e465a6]
>>>> grads(__gxx_personality_v0+0x1a9)[0x496959]
>>>> ======= Memory map: ========
>>>> 00400000-00cca000 r-xp 00000000 08:01 604869
>>>>  /usr/local/grads-2.0.a7.oga.3/bin/grads
>>>> 00dc9000-00dea000 rwxp 008c9000 08:01 604869
>>>>  /usr/local/grads-2.0.a7.oga.3/bin/grads
>>>> 00dea000-010aa000 rwxp 00dea000 00:00 0
>>>> 0180f000-01a04000 rwxp 0180f000 00:00  
>>>> 0                                  [heap]
>>>> 2b6882c7a000-2b6882c9a000 r-xp 00000000 08:01 83056
>>>>  /lib/ld-2.9.so
>>>> 2b6882c9a000-2b6882c9d000 rwxp 2b6882c9a000 00:00 0
>>> --
>>> Dan Swank
>>> STG, Incorporated - Government Contractor
>>> NCDC-NOMADS Project:  Software & Data Management
>>> Data Access Branch
>>> National Climatic Data Center
>>> Veach-Baley Federal Building
>>> 151 Patton Avenue
>>> Asheville, NC 28801-5001
>>> Email: Dan.Swank at noaa.gov
>>> Phone: 828-271-4007
>>>
>
> -- 
> Dan Swank
> STG, Incorporated - Government Contractor
> NCDC-NOMADS Project:  Software & Data Management
> Data Access Branch
> National Climatic Data Center
> Veach-Baley Federal Building
> 151 Patton Avenue
> Asheville, NC 28801-5001
> Email: Dan.Swank at noaa.gov
> Phone: 828-271-4007

--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jma at cola.iges.org



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20100212/e79f1334/attachment.html 


More information about the gradsusr mailing list