[gradsusr] 'flush' command does not work
Brandon Schmidt
admin at wilmingtonwx.com
Thu Mar 19 13:56:05 EDT 2015
Hi Jennifer,
Thank you for the 'q mem' command...that has helped in seeing what you've
described. I did some more testing with the 'flush' command and it does
seem to work. However, I already 'clear' after each display yet memory
continues to rise. Should all "mbufbuff" entries in the memory query
disappear after a 'clear' command? If so, they aren't. I trimmed mbufbuff
entries in the middle of each result below but they were identical in each
case.
-Brandon
ga-> q mem
pos=0 ptr=0x5ef8580 len=7 type=setmpds
pos=1 ptr=0x5ef8840 len=24 type=gxmbuf
pos=2 ptr=0x7f3ca256c010 len=1000000 type=mbufbuff
pos=3 ptr=0x7582a60 len=7 type=cmd
pos=5 ptr=0x5f11090 len=23416 type=pfi
pos=6 ptr=0x5f10020 len=57 type=mnam
pos=7 ptr=0x5f10360 len=48 type=vals1
pos=8 ptr=0x5f103a0 len=48 type=vals1
pos=9 ptr=0x5f103e0 len=64 type=tvals5
pos=10 ptr=0x5f10430 len=80 type=vals2
pos=11 ptr=0x5f16c20 len=50505 type=pvar2
pos=12 ptr=0x5f0fbd0 len=48 type=evals3
pos=13 ptr=0x5f0fc10 len=60 type=ens5
pos=14 ptr=0x5f0fc60 len=40 type=g2indx
pos=15 ptr=0x5f106d0 len=568 type=g2intpnt
pos=16 ptr=0x5f23180 len=20024 type=rbuf1
pos=17 ptr=0x5f27fd0 len=20024 type=pbuf1
pos=18 ptr=0x5f2ce20 len=20024 type=ubuf1
pos=19 ptr=0x7f3c99c61010 len=11563860 type=ppi0
pos=20 ptr=0x7f3c98652010 len=23127720 type=ppf0
pos=21 ptr=0x7f3c97043010 len=23127720 type=ppf1
pos=22 ptr=0x7f3c95a34010 len=23127720 type=ppw
pos=24 ptr=0x5f32d20 len=24 type=gxmbuf
pos=25 ptr=0x69f5c80 len=1000000 type=mbufbuff
pos=26 ptr=0x5f32590 len=24 type=gxmbuf
pos=27 ptr=0x6e38ed0 len=1000000 type=mbufbuff
...
pos=181 ptr=0x7b3ca90 len=24 type=gxmbuf
pos=182 ptr=0xd2f2bb0 len=1000000 type=mbufbuff
pos=183 ptr=0x7b3c240 len=24 type=gxmbuf
pos=184 ptr=0xd3e6e00 len=1000000 type=mbufbuff
ga-> clear
ga-> q mem
pos=0 ptr=0x5ef8580 len=7 type=setmpds
pos=1 ptr=0x5ef8840 len=24 type=gxmbuf
pos=2 ptr=0x7f3ca256c010 len=1000000 type=mbufbuff
pos=3 ptr=0x7b3c330 len=7 type=cmd
pos=5 ptr=0x5f11090 len=23416 type=pfi
pos=6 ptr=0x5f10020 len=57 type=mnam
pos=7 ptr=0x5f10360 len=48 type=vals1
pos=8 ptr=0x5f103a0 len=48 type=vals1
pos=9 ptr=0x5f103e0 len=64 type=tvals5
pos=10 ptr=0x5f10430 len=80 type=vals2
pos=11 ptr=0x5f16c20 len=50505 type=pvar2
pos=12 ptr=0x5f0fbd0 len=48 type=evals3
pos=13 ptr=0x5f0fc10 len=60 type=ens5
pos=14 ptr=0x5f0fc60 len=40 type=g2indx
pos=15 ptr=0x5f106d0 len=568 type=g2intpnt
pos=16 ptr=0x5f23180 len=20024 type=rbuf1
pos=17 ptr=0x5f27fd0 len=20024 type=pbuf1
pos=18 ptr=0x5f2ce20 len=20024 type=ubuf1
pos=19 ptr=0x7f3c99c61010 len=11563860 type=ppi0
pos=20 ptr=0x7f3c98652010 len=23127720 type=ppf0
pos=21 ptr=0x7f3c97043010 len=23127720 type=ppf1
pos=22 ptr=0x7f3c95a34010 len=23127720 type=ppw
pos=24 ptr=0x5f32d20 len=24 type=gxmbuf
pos=25 ptr=0x69f5c80 len=1000000 type=mbufbuff
pos=26 ptr=0x5f32590 len=24 type=gxmbuf
pos=27 ptr=0x6e38ed0 len=1000000 type=mbufbuff
.........
pos=181 ptr=0x7b3ca90 len=24 type=gxmbuf
pos=182 ptr=0xd2f2bb0 len=1000000 type=mbufbuff
pos=183 ptr=0x7b3c240 len=24 type=gxmbuf
pos=184 ptr=0xd3e6e00 len=1000000 type=mbufbuff
On Thu, Mar 19, 2015 at 12:00 PM, <gradsusr-request at gradsusr.org> wrote:
> Send gradsusr mailing list submissions to
> gradsusr at gradsusr.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://gradsusr.org/mailman/listinfo/gradsusr
> or, via email, send a message with subject or body 'help' to
> gradsusr-request at gradsusr.org
>
> You can reach the person managing the list at
> gradsusr-owner at gradsusr.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of gradsusr digest..."
>
>
> Today's Topics:
>
> 1. Re: 'flush' command does not work (Jennifer Adams)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 19 Mar 2015 10:40:22 -0400
> From: Jennifer Adams <jma at cola.iges.org>
> Subject: Re: [gradsusr] 'flush' command does not work
> To: GrADS Users Forum <gradsusr at gradsusr.org>
> Message-ID: <DCBDC890-E502-4901-8579-949C454BE3DF at cola.iges.org>
> Content-Type: text/plain; charset="windows-1252"
>
> Hi, Brandon ?
> There is some code built into GrADS to help us developers track memory
> use. It is accessible with the undocumented ?q mem? command. We keep a list
> of memory blocks that have been allocated (not all, but most), and ?q mem?
> prints the list. The output from ?q mem? includes the address, the size,
> and the type of the memory block ? the type is really a name that makes it
> convenient for finding the place in the code where that block was
> allocated, and also for knowing what that block of memory is for. If you
> compare the output from ?q mem? on startup and then after opening a file,
> and then again after displaying a variable, you will get a sense for what
> is responsible for the size of your executable as it is running. Items
> associated with the grib2 cache have names g2buff1, anchor, g2fld1, and
> g2mask1. If you invoke ?flush? and then check ?q mem? you can see
> immediately whether or not the flush command has done its job.
>
> I tried a sequence of commands similar to what you showed using the
> 0.25-degree GFS output, and could not see any problems with ?flush?. (Aside
> ? thanks for teaching me about ?pgrep?!) I suspect that the memory use in
> your case is not due to an ever-increasing grib2 cache, but to increasing
> allocations of metabuffer blocks (named mbufbuff in the ?q mem? output).
> The metabuffer contains all the instructions for drawing a graphic, and if
> your graphic is very detailed with shaded plots and contour overlays, the
> use of shapefiles or the basemap script, or if it contains many panels,
> then the metabuffer expands accordingly and additional blocks are
> allocated, and you will see lots of ?mbufbuff? entries when checking ?q
> mem?. The metabuffer blocks are not released on ?reinit? ? we assume if you
> needed them once, you may need them again, so we leave them chained up,
> ready to be re-used. A ?clear? command will flush the metabuffer and reset
> the pointer to the beginning of the first !
> block; it may be that all your script needs is a ?clear? at the beginning
> of each loop.
>
> ?Jennifer
>
> On Mar 19, 2015, at 2:01 AM, Brandon Schmidt <admin at wilmingtonwx.com>
> wrote:
>
> > Through extensive debugging, I've found that the GrADS flush function
> does not work properly. This first came to my realization when I began to
> automate my GrADS scripts. Each script processes a single hour by
> downloading a GRIB file, creating a control file, opening the control file,
> and displaying/outputting the needed data. The scripts initialize with
> right around 100 MB of memory but through execution, they can rack up as
> much as 1.4 GB. Of course when processing multiple hours, this slowed my
> server to a crawl and in some cases caused crashes (especially with
> high-res HRRR data for example). Anyway, I traced the excessive memory
> usage to the display commands. I am aware that with each display command,
> the displayed variable is stored to the GrADS GRIB2 cache to eliminate
> further decompression overhead. This is all well given the flush command
> but it seems to only work half of the time. One can easily see this by
> noting the initial memory, displaying a few variables, f!
> lushing, and then noting the final memory. Seemingly randomly, the memory
> following a flush will slowly creep up. The behavior memory-wise that I
> expected from flush can only be obtained with reinit but calling that
> function thousands of times is not an option. Any ideas as to a work-around
> or fix? I am using the latest 2.1.a3 version. The first flush below works
> but the second does not.
> >
> > -Brandon
> >
> >
> ---------------------------------------------------------------------------------
> > ga-> !pgrep grads
> >
> > 7879
> >
> > ga-> ! ps -p 7879 -o %mem,size
> >
> > %MEM SZ
> >
> > 1.3 87276
> >
> > ga-> d tmp2m
> >
> > Notice: Automatic Grid Interpolation Taking Place
> >
> > Contouring: 255 to 305 interval 5
> >
> > ga-> ! ps -p 7879 -o %mem,size
> >
> > %MEM SZ
> >
> > 1.7 108024
> >
> > ga-> d capesfc
> >
> > Notice: Automatic Grid Interpolation Taking Place
> >
> > Contouring: 0 to 3500 interval 500
> >
> > ga-> set lat 50 70
> >
> > LAT set to 50 70
> >
> > ga-> d gustsfc
> >
> > Notice: Automatic Grid Interpolation Taking Place
> >
> > Contouring: 1 to 13 interval 1
> >
> > ga-> set lat 20 90
> >
> > LAT set to 20 90
> >
> > ga-> d tmp2m
> >
> > Notice: Automatic Grid Interpolation Taking Place
> >
> > Contouring: 255 to 305 interval 5
> >
> > ga-> flush
> >
> > grib2 cache cleared
> >
> > ga-> ! ps -p 7879 -o %mem,size
> >
> > %MEM SZ
> >
> > 1.7 108024
> >
> > ga-> d tmp2m
> >
> > Notice: Automatic Grid Interpolation Taking Place
> >
> > Contouring: 255 to 305 interval 5
> >
> > ga-> ! ps -p 7879 -o %mem,size
> >
> > %MEM SZ
> >
> > 1.9 122908
> >
> > ga-> flush
> >
> > grib2 cache cleared
> >
> > ga-> ! ps -p 7879 -o %mem,size
> >
> > %MEM SZ
> >
> > 1.9 122908
> >
> > ga->
> >
> > _______________________________________________
> > gradsusr mailing list
> > gradsusr at gradsusr.org
> > http://gradsusr.org/mailman/listinfo/gradsusr
>
> --
> Jennifer M. Adams
> Center for Ocean-Land-Atmosphere Studies (COLA)
> 111 Research Hall, Mail Stop 2B3
> George Mason University
> 4400 University Drive
> Fairfax, VA 22030
>
>
>
>
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://gradsusr.org/pipermail/gradsusr/attachments/20150319/82b935a5/attachment-0001.html
>
> ------------------------------
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
>
>
> End of gradsusr Digest, Vol 61, Issue 23
> ****************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20150319/56b69995/attachment-0001.html
More information about the gradsusr
mailing list