NetCDF issue?

[Jennifer Adams]

On Mar 17, 2008, at 1:58 PM, Matt Alonso wrote:

> Hi Jennifer,
>
>
> Did you build GrADS from source on this machine?
>
> Yes
>
> Looks like you have converted grib2 into netcdf -- one file per
> forecast time. Is that right?
>
> Yes, I created one netcdf file per grib2 forecast run (ie 01.nc
> would contain all 61 time periods from the 12z WW3 run); see more
> thorough description below.
>
>
> There are some bugs in the gancgrid() routine in 2.0.a1 that I
> believe have been fixed for the next release. The gancgrid()
> routine is designed for speeding up the I/O from OPeNDAP servers,
> but it can work for non-pdef non-templated local netcdf files too.
> The non-templated condition for calling gancgrid() is what's
> missing in 2.0.a1 plus some other bug fixes in the subroutine itself.
>
> You can bypass the gancgrid() routine by changing one line of code
> in gaio.c (lines 212-216) and re-compiling:
>
>   /*   if (!strncmp(pgrid->pfile->name,"http://",7)) { */
>   if ((!strncmp(pgrid->pfile->name,"http://",7)) || ((pfi-
> >ncflg==1) && (pfi->ppflag==0))) {
>     rc = gancgrid(gr,gru,id,jd,d,dx);
>     return (rc);
>   }
>
> If you un-comment out the first line and comment out the second --
> i.e., call gancgrid only for opendap data sets -- then the I/O
> should work.
>
> However, it sounds like you are also making a mistake in the way
> you are templating your netcdf files. If you have one file per
> forecast time, you can aggregate them into one data set (with
> XDFOPEN and OPTIONS TEMPLATE) so there's no need to have each time
> step be in a separate data set. Also, when you call the smth9()
> function, which dimensions are varying?
>
> I will modify the src accordingly and see if that helps.
It should do the trick, since you'll be using the 'dtype netcdf' code
path from 1.9, which is more robust.

> I think I may have misspoke initially when describing the setup of
> the various files or perhaps I am just misunderstanding.  As I
> attempted to clarify above each netcdf file has the exact same
> contents as the original grib2 file on NCEP's FTP site (ftp://
> ftpprd.ncep.noaa.gov/pub/data/nccf/com/wave/prod/wave.20080317).
>
> I want to show how the model is trending by using an "ensemble" of
> the previous 7 runs.  When my script is executed it inserts the
> newest runs data into 00.nc and "bumps" the previous runs down.  If
> the newest run is the 12Z run for today (20080317) the netcdf files
> would contain the data for the various runs as follows:
>
> 00.nc -> 12Z 20080317
> 01.nc -> 06Z 20080317
> 02.nc -> 00Z 20080317
> 03.nc -> 18Z 20080316
> ...

This is a GREAT example for using the E dimension to simplify your
analysis! Here are excerpts from your 5D descriptor file:

dset  ^%e.nc
options template
tdef 67 linear 00z16mar2008 6hr
edef 7
00 61 12z17mar2008
01 61 06z17mar2008
02 61 00z17mar2008
03 61 18z16mar2008
04 61 12z16mar2008
05 61 06z16mar2008
06 61 00z16mar2008
endedef

The tdef statement describes a single time axis that spans all 7
ensembles (which are reallly forecasts with different initial times).

An additional simplification might be to skip the conversion to
NetCDF -- you can read the grib2 files directly, templating over T
and E. Once you've done the conversion, though, I/O will be faster
with NetCDF, especially if X and Y don't vary. But definitely use
EDEF instead of 'set dfile 'n. There's a lot more analysis you can do
when your ensembles are not separate data sets.

Jennifer


>
>
> And so on until there are 8 netcdf files.  Would your templating
> example still apply and help me display data in the manner I
> describe above?  I attached a sample image which is being generated
> by this same script but is running in GrADS 1.8S11.
>
> Thanks as always.
>
> Cheers,
> Matt
>
>

--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jma at cola.iges.org



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20080317/9e16d8d9/attachment.html