Memory issue in GrADS

[Jennifer Adams]

Hi, Bob --
I have cc'd gradsusr on this answer because it is of general
interest ...


GrADS caches grib2 grids in memory. This facilitates the I/O if you
make a 2nd display request for a variable -- it will be read from the
cache instead of the grib2 file (and therefore won't need to be
uncompressed again). The limit of the cache size is (for now) hard-
coded in gaio around line 76:
   #define MAXG2CACHE 500100100
You can change this and recompile if you want to limit or expand your
memory usage. I intend to make grib2 cache size a command line
argument at some point. When the cache reaches its max size, it
releases the oldest grid in the cache (i.e., the one you read in
first).  You can also release all the memory in the grib2 cache with
the 'flush' command.

Using 'define' with grib2 data is redundant, and it doubles your
memory usage -- retrieving a cached grib2 variable is just as fast as
retrieving a defined variable. Use 'define' if you think you'll need
a variable for longer than it will linger in the cache, or if you are
flushing the cache regularly.

Your script uses 'define' at almost every line, but this is not
really necessary. For example,
     "define A=2.53e9"
     "define B=5.42e3"
     "define Td=B/log(A*0.622/(mix*lev(z+1)))"
  could be
     A='2.53e9'
     B='5.42e3'
     "define Td="B"/log("A"*0.622/(mix*lev(z+1)))"

Depending on the complexity of the expressions "A" and "B" you will
save a lot of memory if you don't define them at every step but let
the GrADS recursive expression handling do the job for you.

Jennifer


On Mar 4, 2008, at 2:05 PM, Robert Hart wrote:

> I'm noticing that scripts that worked fine now cause memory
> allocation errors with GrADS v2a.  Even when I have the correct
> undefines, the memory usage continues to grow.
>
> As an example, please see:
>
> http://moe.met.fsu.edu/~rhart/grads-mem
>
> This uses the NAM 12km GRIB2 data from, e.g.
>
> ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/nam/prod/nam.20080304/
> nam.t00z.awphys00.grb2.tm00
> through 84hr
>
> By time step 14 or 15, the script has accumulated 1GB of memory and
> then crashes due to memory allocation errors.
>
> This worked fine in 1.94 and earlier.
>
> Bob
>

--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jma at cola.iges.org



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20080304/259b4b22/attachment.html