[gradsusr] pdef file works at v1.9b4 but not at v2.0.a9

Fri Jul 22 06:28:45 EDT 2011

Douglas,

The zero values for the mask in pdef file look to be normal.
But reading your netcdf require to know each dimension size
"as it is" in your file. When you provide description file (ctl)
grads ignore internal netcdf information for dimension
using yours instead. Then, for the input arrays allocation
"67420 1" is OK, but the real netcdf lat-lon grid size is necessary for
correct data input before the pdef mask  application.
Please run ncdump on you file and try "nx ny" instead.
Just it must help. Or, if you are ready to rebuild an interpolation
arrays, please create PDEF of GENERAL type that is better documented.

Once more, the problem is in reading not the pdef mask file, but
in reading from your netsdf file all 67420 values along the first (x or lon)
dimension with all other dimensions fixed! It crashes netcdf library
and then grads, I guess.

In Grads prior version 1.9 the pdef do not worked for netcdf files at all
due to this problem, but worked for binary files.
>From 1.9 it worked, but already required to use nx and ny
instead of "nx*ny 1" for FILE type pdefs in case of netcdf files,
except that your netsdf file really has
an internal structure with "67420*1" spatial grid.

Good luck to you!

Sergey

Clark, Douglas B. wrote:
> Sergey,
>
> Thanks for your answer. In fact I don't think your suggestion was the answer I needed, but it did cause me to look at the documentation again, at which point I got my answer!
>
> My data are essentially a "vector" of land points and I use the pdef to scatter these across a lat-lon grid (very like the final example on the pdef documentation page). So the "native grid" really is 67420 x 1. The problem was that in the pdef data file (WFD_0p5deg_pdefData.gra in my example below), I had values of zero for the index (or offset) variable over the sea - i.e. the locations that I didn't have data for. These would have been better set to the missing data value. At v1.9.b4, the zeroes worked fine (and I was blissfully unaware that GrADS was struggling with different offset values for GRIB and non-GRIB files). At v2.0.a9 presumably GrADS notes that I don't have a GRIB file and tests the offsets; when it finds a zero it fails (rather dramatically with a core dump!).
>
> So the answer was to make sure that my offset is set to the missing data value where I don't have data. Which sounds (and is) pretty obvious, but had worked for years with zero instead!
>
> I have been using this style of pdef file for years and hadn't noticed that the online documentation for pdef now has various "WARNING" and other messages documenting issues with the offset (and that pdef FILE is now deprecated in favour of pdef GENERAL).
>
> Thanks again,
> Doug
>
>
> -----Original Message-----
> From: gradsusr-bounces at gradsusr.org [mailto:gradsusr-bounces at gradsusr.org] On Behalf Of Sergey Varlamov
> Sent: 22 July 2011 03:33
> To: GrADS Users Forum
> Subject: Re: [gradsusr] pdef file works at v1.9b4 but not at v2.0.a9
>
>
> Dear Douglas,
>
> Please try to change
>
> pdef 67420 1 file ...
>
> to you real grid size
>
> pdef ix jy file ...
>
> where ix and jy are dimensions of your original curvilinear grid,
> such that ix*jy=67420.
> It must still work in version 1.9 and could work in versions 2 of GrADS.
>
> Sergey Varlamov,
>
> JAMSTEC, Yokohama
>
>
> Clark, Douglas B. wrote:
>   
>> Dear All,
>>
>> I have a dataset that uses a pdef file and that I can use in GrADS v1.9b4 (in both SunOS and RH executables downloaded from the GrADS website), but which results in a segmentation fault at v2.0.a9 (and, from memory, at v2.0.a8) - again in various downloads. Looks like it might be a bug in GrADS...but of course most things turn out to be user error! Any ideas?
>>
>> At v2.0.a9:
>>
>> ga-> q config
>> Config: v2.0.a9 little-endian readline printim grib2 netcdf hdf4-sds hdf5 opendap-grids geotiff shapefile
>> Grid Analysis and Display System (GrADS) Version 2.0.a9
>> Copyright (c) 1988-2010 by Brian Doty and the
>> Institute for Global Environment and Society (IGES) 
>> This program is distributed WITHOUT ANY WARRANTY 
>> See file COPYRIGHT for more information. 
>>
>> Built Fri Sep  3 17:21:21 UTC 2010 for x86_64-redhat-linux-gnu
>>
>> This version of GrADS has been configured with the following options:
>>   o Built on a LITTLE ENDIAN machine
>>   o Athena Widget GUI DISABLED
>>   o Command line editing ENABLED 
>>       http://tiswww.case.edu/php/chet/readline/rltop.html 
>>   o printim command for image output ENABLED 
>>       http://www.zlib.net 
>>       http://www.libpng.org/pub/png/libpng.html 
>>       http://www.libgd.org/Main_Page 
>>   o GRIB2 interface ENABLED 
>>       http://www.ijg.org 
>>       http://www.ece.uvic.ca/~mdadams/jasper 
>>       http://www.nco.ncep.noaa.gov/pmb/codes/GRIB2 
>>       g2clib-1.2.0  
>>   o NetCDF interface ENABLED 
>>       http://www.unidata.ucar.edu/software/netcdf  
>>       netcdf 4.1.1 of Dec  6 2010 08:20:12 $  
>>   o OPeNDAP gridded data interface ENABLED
>>   o OPeNDAP station data interface DISABLED
>>   o HDF4 and HDF5 interfaces ENABLED 
>>       http://hdfgroup.org 
>>       HDF 4.2r5 
>>       HDF5 1.8.5 
>>   o GeoTIFF and KML/TIFF output ENABLED
>>       http://www.libtiff.org 
>>       http://geotiff.osgeo.org 
>>   o KML contour output ENABLED
>>   o Shapefile interface ENABLED
>>       http://shapelib.maptools.org 
>>
>> For additional information please consult http://iges.org/grads
>>
>> ga-> open Rainf_WFD_GPCC.ctl 
>> Scanning description file:  Rainf_WFD_GPCC.ctl
>> PDEF FILE Error: The offsets in the pdef file 
>>   must be 1-based (i.e., > 0 and <= isize*jsize). 
>>   The data file was not opened. 
>> Segmentation fault (core dumped)
>>
>> -------------------------------------------------------------
>>
>> The first few lines of the ctl file are:
>>
>> dset ^../data/Rainf_WFD_GPCC/Rainf_WFD_GPCC_%y4%m2.nc
>> options template
>> dtype netcdf
>> undef 1.0e20
>> pdef 67420  1 file 1 stream binary-big ^../../../ancil/data/WFD_0p5deg_pdefData.gra
>> xdef    720 linear -179.75 0.5
>> ydef    280 linear  -55.75 0.5
>>
>>
>> -------------------------------------------------------------
>>
>> Works fine at v1.9.b4:
>>
>> ga-> open Rainf_WFD_GPCC.ctl 
>> Scanning description file:  Rainf_WFD_GPCC.ctl
>> Data file ../data/Rainf_WFD_GPCC/Rainf_WFD_GPCC_%y4%m2.nc is open as file 1
>> LON set to 0 360 
>> LAT set to -55.75 83.75 
>> LEV set to 1 1 
>> Time values set: 2001:1:1:0 2001:1:1:0 
>> Notice: Implied interpolation for file Rainf_WFD_GPCC.ctl
>>  Interpolation will be performed on any data displayed from this file
>> ga-> d rainf
>> Notice:  Automatic Grid Interpolation Taking Place
>> Contouring: 0 to 0.0033 interval 0.0003
>>
>>
>> -------------------------------------------------------------
>>
>> Vague notion: The pdef data file is big-endian (as declared in the pdef line of the ctl file) but
>> I'm using a little-endian machine. I wonder if v2.0.a9 can't cope with this mix?
>>
>>
>> Cheers,
>> Doug
>>
>>
>>
>>
>>   
>>     
>
>
>   

-- 

Sergey Varlamov

Senior Scientist
Ocean Downscaled Prediction Research Team
Climate Variation Predictability and Applicability Research Program
Research Institute for Global Change
JAMSTEC, 3173-25 Showa-machi, Kanazawa-ku,
Yokohama, Kanagawa-ken, 236-0001 JAPAN
Tel: +81-45-778-5516  Fax: +81-45-778-5707
E-mail: vsm at jamstec.go.jp

〒236-0001 横浜市金沢区昭和町3173-25
独立行政法人海洋研究開発機構 地球環境変動領域
短期気候変動応用予測研究プログラム
ダウンスケール沿海変動予測研究チーム
Varlamov Sergey