Memory issue in GrADS

[Jennifer Adams]

On Mar 4, 2008, at 5:12 PM, Eric Altshuler wrote:

> Hi Jennifer,
>
> I was under the impression that the following commands had exactly  
> the same effect in grads:
>
> 'define var = expr'
> 'var = expr'
>
> i.e. the 'define' command was optional and it didn't matter whether  
> it was there or not.
That's right. The 2nd expression is an implied define command.

> Is this no longer true in general, or is this caching behavior only  
> for grib2 data?
Define hasn't changed. The cacheing is only for grib2 -- it was  
implemented to alleviate the problems of reading compressed data (see  
earlier post regarding meteograms).

Jennifer

>
> Eric
>
> ----- Original Message -----
> From: "Jennifer Adams" <jma at COLA.IGES.ORG>
> To: GRADSUSR at LIST.CINECA.IT
> Sent: Tuesday, March 4, 2008 4:25:37 PM GMT -05:00 US/Canada Eastern
> Subject: Memory issue in GrADS
>
> Hi, Bob --Â
> I have cc'd gradsusr on this answer because it is of general  
> interest ...Â
>
>
>
>
> GrADS caches grib2 grids in memory. This facilitates the I/O if you  
> make a 2nd display request for a variable -- it will be read from  
> the cache instead of the grib2 file (and therefore won't need to be  
> uncompressed again). The limit of the cache size is (for now) hard- 
> coded in gaio around line 76:
> Â Â #define MAXG2CACHE 500100100
> You can change this and recompile if you want to limit or expand  
> your memory usage. I intend to make grib2 cache size a command line  
> argument at some point. When the cache reaches its max size, it  
> releases the oldest grid in the cache (i.e., the one you read in  
> first). Â You can also release all the memory in the grib2 cache  
> with the 'flush' command.Â
>
>
> Using 'define' with grib2 data is redundant, and it doubles your  
> memory usage -- retrieving a cached grib2 variable is just as fast  
> as retrieving a defined variable. Use 'define' if you think you'll  
> need a variable for longer than it will linger in the cache, or if  
> you are flushing the cache regularly.Â
>
>
> Your script uses 'define' at almost every line, but this is not  
> really necessary. For example,Â
> Â Â  Â "define A=2.53e9"
> Â Â  Â "define B=5.42e3"
>
> Â Â  Â "define Td=B/log(A*0.622/(mix*lev(z+1)))"
> Â could beÂ
> Â Â  Â A='2.53e9'
> Â Â  Â B='5.42e3'
>
> Â Â  Â "define Td="B"/log("A"*0.622/(mix*lev(z+1)))"
>
>
>
> Depending on the complexity of the expressions "A" and "B" you will  
> save a lot of memory if you don't define them at every step but let  
> the GrADS recursive expression handling do the job for you.Â
>
>
> Jennifer
>
>
>
>
>
>
> On Mar 4, 2008, at 2:05 PM, Robert Hart wrote:
>
>
>
> I'm noticing that scripts that worked fine now cause memory  
> allocation errors with GrADS v2a.  Even when I have the correct  
> undefines, the memory usage continues to grow.
>
>
> As an example, please see:
>
>
> http://moe.met.fsu.edu/~rhart/grads-mem
>
>
> This uses the NAM 12km GRIB2 data from, e.g.
>
>
> ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/nam/prod/nam.20080304/ 
> nam.t00z.awphys00.grb2.tm00
> through 84hr
>
>
> By time step 14 or 15, the script has accumulated 1GB of memory and  
> then crashes due to memory allocation errors.
>
>
> This worked fine in 1.94 and earlier.
>
>
> Bob
>
>
>
>
> --
> Jennifer M. Adams
> IGES/COLA
> 4041 Powder Mill Road, Suite 302
> Calverton, MD 20705
> jma at cola.iges.org

--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jma at cola.iges.org



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20080305/64eb8157/attachment.html