[gradsusr] double free or corruption (!prev)

[Stefan Gofferje]

Hi Jennifer,

I'm indeed making pretty big defines. At the moment, My process is to 
download the grib files for all time points, concat them, gribmap them 
and load the resulting 1.9GB file into grads. Then I start a script 
which makes a lot of defines and then runs subscripts which all make use 
of the defines. The reason behind this process is to save time, because 
on that machine, the defines take well 4 minutes so I want to run them 
only once. I'm graphing the resources of the machine and it always had 
at least half a gig memory "free" and >5GB "available", so in theory, 
grads should be good. Now the recent crashes happened while rendering 
shape files, though.

How does the memory management work? Are the caches sizes hardcoded or 
is it possible to increase them?
On 4/8/20 10:12 PM, Jennifer M Adams wrote:
> Hi, Stefan --
> The error message says it was unable to allocate 1778821*8 bytes in 
> memory. Try checking the output from 'q define' and 'q define <name>' to 
> see whether you're storing too much in memory. Make good use of the 
> 'undefine' command when you're finished working with a defined variable.
> 
> Are you drawing some mulit-layered graphic that requires a huge 
> metabuffer? Maybe your grib2 cache is also maxed out. You can track the 
> memory use of your grads process using the 'ps' command in another 
> terminal window.
> 
> If you really want to see every bit of memory that GrADS has allocated, 
> check the 'q mem' command -- an undocumented developer's tool that will 
> show you the list of allocated items, their length, and the type, which 
> provides a hint for where in the src code that chunk was allocated. 
> Allocations of type 'defn*buf' are from the 'define' command.
> 
> Maybe those things will help you track down where you're unintentionally 
> gobbling up memory. 32GB isn't that much, given the size of GFS output 
> files these days.
> --Jennifer
> ------------------------------------------------------------------------
> *From:* gradsusr <gradsusr-bounces at gradsusr.org> on behalf of Stefan 
> Gofferje <stefan at saakeskus.fi>
> *Sent:* Tuesday, April 7, 2020 2:56 AM
> *To:* gradsusr at gradsusr.org <gradsusr at gradsusr.org>
> *Subject:* [gradsusr] double free or corruption (!prev)
> Hi,
> 
> I recently experience grads crashes relatively often on GFS renderings.
> Here is an example output:
> 
> *** Error in `/usr/local/bin/grads221/grads': double free or corruption
> (!prev): 0x00000000010b6330 ***
> 
> Grid Analysis and Display System (GrADS) Version 2.2.1
> Copyright (C) 1988-2018 by George Mason University
> GrADS comes with ABSOLUTELY NO WARRANTY
> See file COPYRIGHT for more information
> 
> Config: v2.2.1 little-endian readline grib2 netcdf hdf4-sds hdf5
> opendap-grids geotiff shapefile
> Issue 'q config' and 'q gxconfig' commands for more detailed
> configuration information
> GX Package Initialization: Size = 11 8.5
> Running in Batch mode
> Making defines
> Define Error:  Unable to allocate data memory
>     Size of request was 1778821 grid elements
> /storage/webserver/weather/grads/rendermaps.sh: line 34: 24217 Aborted
>                 (core dumped) ${GRADS} -blc "run sg_master_landscape.gs
> ${GRIB2FILE}.ctl"
> 
> The machine has 32GB memory and usually well over 50% of it available
> (although "cached"). Some debugging help and a fix would be appreciated :).
> 
> 
> -Stefan
> 
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://secure-web.cisco.com/1o04x8ZaCcokRSN2H0wrt9yN9fCUp-8e87ViUWtM8pToPNoAccjpEuwN1U5g83LlT41wUqNNYDz4q0uWA04v34jxFKhaoGjvsgon3KRw3oW-GDIfmMmn5mm8aRbLbK3nGbcgbKTVFxHHOlb0Ix7wXGA4tLTqrX6KICDwomeA2oiWpKOe3Oxl7OL4y0ifgLe1u2vhFh8ObUE0j3xMLFZmtCQiZvds9OuX_xQyQlLx0ra5Uy4vIm55rFzPQgspBWMw63w6ZeJbJFRG4JmaQzwhz9HXTuysCWSu2Z-A9E6OWf5ap56dSFOQHgspTS3LtLfee-SzEZQa-7ET8yKtkIzZ9epEBOquCgDcGP0cl806C-_vIK9Y-w0IRVgmH96c_y4F2OKayOc12YmcJfkzMnm9NPaYEj2Gb7uqLgVtqEKI9aModRqJBx1y_MUlA9r_VGeKC/http%3A%2F%2Fgradsusr.org%2Fmailman%2Flistinfo%2Fgradsusr
> 
> 
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
>