[gradsusr] grib2map and skipping fields (time steps) inside of single record

[Jennifer Adams]

Sergey --
That is a pathological use of GRIB2 format, and I cannot imagine WHY  
the Japan Meteorological Agency would set up their files that way.  
Handling it in GrADS requires more than switching a BREAK to a  
CONTINUE in gribmap. The record length will likely be ultra-long  
(requiring 8 octets instead of 4, which is not currently supported)  
and the memory use will be high and I/O will be slow. A file like this  
be unusable if the grids are very high resolution. If you have a small  
sample of this kind of grib2 file, I will take a look (please send it  
to me offline), but I make no promises that we will support this abuse  
of the GRIB2 format.

I suggest you use wgrib2 to break up the record into its proper  
pieces, writing out one one field per record, saving only the fields  
you are interested in. Then you can read it in GrADS in the normal way.
--Jennifer


On Dec 7, 2010, at 11:26 PM, Sergey Varlamov wrote:

> Dear GrADS developers and experts,
>
> My problem is that with the new version of gridmap for
> grib2 files I can not skip record.fields in grib2 file that do not
> fit to requested time stepping.
>
> Working with old versions of grads (1.x) I often used
> grads description files (ctl) to skip some time steps
> from analysis by making custom description file
> where was specified larger time step value (and smaller total
> number of time steps) compared with these in the grib file(s).
> Example: select only 0,3,6 etc. hours forecasts from the file
> that includes 0,1,2,3,4,.. etc. hours forecasts.
> Gribmap utility for grib1 files perfectly scanned such files (grib 
> +ctl) and
> generated index file where only matched records were
> included and other skipped, as I expected.
>
> But trying to do the same with the Japan Meteorological Agency (JMA)
> grib2 files and gribmap for grads v 2.0.8, I met the problem that new
> gribmap utility
> for grib2 files stops scanning JMA files after the first non-matching
> field is found:
>
> 1.11: lev1=101 var=0,3,1 valid time 2010110113:00 (t=1.33333) has
> non-integer grid index
>
> I was expecting that this field must be skipped as  NON-MATCHING
> and next fields analyzed, but really all following field of analyzed  
> record
> were skipped.
>
> As I found, this resulted from both specifics of JMA grib2 files
> and design of gribmap utility:
>
> 1) JMA grib2 file has only one record where all subsequent
> forecasts for all parameters are sub-records (fields in terms of  
> gribmap).

>
> 2) Design of gribmap (gagmap.c file) is such that:
> g2time_check functions return -99 (an error) and gribmap
> function uses this error information to BREAK sub-records (fields)  
> analysis
> cycle, leaving all other VALID fields inside of the same grib2
> record do not tested/indexed.
>
> I would like to change BREAK statement to CONTINUE
> in the fields analysis cycles of gribmap function.
>
> Is it possible or I do not see some hidden problems of
> such modification?
> Could it be done in the official gribmap
> utility distributed with grads (possibly with some new
> option key added as it could increase analysis time and
> verbose output size)?
>
> Sincerely yours,
> Sergey Varlamov
>
> Senior Scientist
> Research Institute for Global Change
> JAMSTEC, JAPAN
> E-mail: vsm at jamstec.go.jp
>
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr

--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jma at cola.iges.org



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20101208/b1466c39/attachment.html