Memory issue in GrADS

[Eric Altshuler]

Hi Jennifer,

I was under the impression that the following commands had exactly the same effect in grads:

'define var = expr'
'var = expr'

i.e. the 'define' command was optional and it didn't matter whether it was there or not. Is this no longer true in general, or is this caching behavior only for grib2 data?

Eric

----- Original Message -----
From: "Jennifer Adams" <jma at COLA.IGES.ORG>
To: GRADSUSR at LIST.CINECA.IT
Sent: Tuesday, March 4, 2008 4:25:37 PM GMT -05:00 US/Canada Eastern
Subject: Memory issue in GrADS

Hi, Bob --Â
I have cc'd gradsusr on this answer because it is of general interest ...Â




GrADS caches grib2 grids in memory. This facilitates the I/O if you make a 2nd display request for a variable -- it will be read from the cache instead of the grib2 file (and therefore won't need to be uncompressed again). The limit of the cache size is (for now) hard-coded in gaio around line 76:
  #define MAXG2CACHE 500100100
You can change this and recompile if you want to limit or expand your memory usage. I intend to make grib2 cache size a command line argument at some point. When the cache reaches its max size, it releases the oldest grid in the cache (i.e., the one you read in first). Â You can also release all the memory in the grib2 cache with the 'flush' command.Â


Using 'define' with grib2 data is redundant, and it doubles your memory usage -- retrieving a cached grib2 variable is just as fast as retrieving a defined variable. Use 'define' if you think you'll need a variable for longer than it will linger in the cache, or if you are flushing the cache regularly.Â


Your script uses 'define' at almost every line, but this is not really necessary. For example,Â
    "define A=2.53e9"
    "define B=5.42e3"

    "define Td=B/log(A*0.622/(mix*lev(z+1)))"
 could beÂ
    A='2.53e9'
    B='5.42e3'

    "define Td="B"/log("A"*0.622/(mix*lev(z+1)))"



Depending on the complexity of the expressions "A" and "B" you will save a lot of memory if you don't define them at every step but let the GrADS recursive expression handling do the job for you.Â


Jennifer






On Mar 4, 2008, at 2:05 PM, Robert Hart wrote:



I'm noticing that scripts that worked fine now cause memory allocation errors with GrADS v2a.  Even when I have the correct undefines, the memory usage continues to grow.


As an example, please see:


http://moe.met.fsu.edu/~rhart/grads-mem


This uses the NAM 12km GRIB2 data from, e.g.


ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/nam/prod/nam.20080304/nam.t00z.awphys00.grb2.tm00
through 84hr


By time step 14 or 15, the script has accumulated 1GB of memory and then crashes due to memory allocation errors.


This worked fine in 1.94 and earlier.


Bob




--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jma at cola.iges.org