[gradsusr] Maximum data file size
pmarkowski at psu.edu
Wed May 31 12:29:31 EDT 2017
This same error was the subject of a discussion on April 13. I've
pasted Jennifer Adams' reply below. It appears to be a subtle Grads bug
(maybe "bug" isn't the right word--perhaps "limitation" is better?).
Jennifer, any updates on where things stand regarding a patch? Thanks!!
On Apr 13, 2017, at 7:25 PM, Jennifer M Adams <jadams21 at gmu.edu
<mailto:jadams21 at gmu.edu>> wrote:
I figured out what the problem is. For each variable in a binary file,
GrADS calculates an offset --- the number of bytes that you must skip
over before you read that particular variable. The offset is for the
first variable in the file is 0. For each successive variable, it is the
offset from the previous variable plus its nlevs*nlons*nlats. In your
data file, nlevs is very large (127), and the xsize and ysize are also
large enough so that the offset grows very quickly. Unfortunately, it is
stored internally as a 4-byte integer, and after the 31st variable, it
runs out of bits and the offset values become garbage. I added a
debugging print statement which shows the offset for each variable, note
it goes bad for the last 6 variables:
So, I will have to patch this bug and use an 8-byte integer to store the
offset. Until then, your workaround is to write out fewer Z levels, or
separate the variables into two distinct files. The output from 'q file'
was very helpful, and so was the clue about the final six variables
showing the error.
On 5/31/17, 11:20 AM, Chris Nowotarski wrote:
> I'm running grads 2.0.2 and I appear to be running into file size
> limitations. Is there a maximum file size (or record number) for
> binary files that are interpretable by Grads? I had heard there was
> previously a 2 GB limit, but that this limit is alleviated in newer
> versions of Grads. I seem to have an issue once files exceed 8 GB.
> Is there any way around this besides breaking up files?
> Chris Nowotarski
> Assistant Professor
> Department of Atmospheric Sciences
> Texas A&M University
> MS 3150
> College Station, TX 77843
> gradsusr mailing list
> gradsusr at gradsusr.org
Professor of Meteorology
Department of Meteorology and Atmospheric Science
Pennsylvania State University
520 Walker Building
University Park, PA 16802
email: pmarkowski at psu.edu
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the gradsusr