[gradsusr] Performance issue when working with multiple files on GrADS 2.1

[Brian Gaze]

Jennifer,

Thanks - but that pattern won't work if I want to draw both of the
variables on the same image will it?

Brian


On 26 March 2015 at 17:49, Jennifer Adams <jma at cola.iges.org> wrote:

> Hi, Brian —
> The grib2 cache is a linked list of 2-D grids that are described by the
> file number, the coordinates of the non-varying dimensions (Z, T, and E),
> the size of the grid, and the name of the variable. If you are doing more
> than one display from that grid, the data are read from the cache instead
> of re-doing the I/O. Each new grid that is read is added to the list, most
> recent first. But because of the size limit of the grib2 cache, if you
> exceed the size, the grid at the end of the chain is released to make room
> for the new grid. So, if you are displaying data from 22 different grids,
> that is probably enough to exceed the size of the cache and when you go
> back to grid #1 the I/O has to be done again because that grid is no longer
> in the cache. It therefore makes a lot of sense for you to re-arrange the
> order of your nesting loops and do all your displays of a single cached
> grid before moving to the next variable.
>
> loop over file names
>   open file
>   loop over variables
>     loop over lat/lon domains
>        draw plot
>        print image
>     end
>   end
>   close file
> end
>
> The metafile buffer size option is parsed but not looked at in version
> 2.1. I should probably add a warning to alert the user, or remove it
> altogether.
> --Jennifer
>
>
>
> On Mar 26, 2015, at 12:26 PM, Brian Gaze <brian.gaze at NTLWORLD.COM> wrote:
>
> Hi,
>
> When working concurrently with multiple grib2 files I'm encountering a
> performance issue. I initially thought this was caused by the grib2 cache
> but now wonder if it is related to the metafile buffer.
>
> The sample script below helps to explain. Scenario is....
>
> 1) Open all files
> 2) Outer loop where latitude and longitude for plot are set
> 3) Inner loop through all open files plotting 2 variables from each one
> for the latitude and longitude currently set
> 4) Write output to an image file
> 5) Iterate
>
> If I reduce the filenumber count to be lower than 13 OR only plot 1
> variable the image files are generated quickly, approx 1.5 per second.
>
> If I leave the filenumber count as it is and plot the 2 variables the plot
> plot is very slow taking approx 50 seconds to be written to the file system.
>
>
>
> *Open files 1 to 22
> 'open file00.ctl'
> 'open file01.ctl'
> 'open filexx.ctl'
>
>
> while(b<60)
>
> filenumber=1
> while (filenumber<23)
> 'set dfile ' %filenumber
> 'd var1'
> 'd var2'
> filenumber=filenumber+1
> endwhile
>
> op = ' C:\'%imagefilename% '.png png x800 y600'
> 'gxprint ' %op ' white'
>
> B=B+1
> endwhile
>
>
> When running the .gs I've tried using the -m NNN option to increase the
> metafile buffer size (e.g. to 3000000) but this doesn't seem to make any
> difference.
>
> Any ideas??
>
> Thanks
>
> Brian
>
>
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
>
>
> --
> Jennifer M. Adams
> Center for Ocean-Land-Atmosphere Studies (COLA)
> 111 Research Hall, Mail Stop 2B3
> George Mason University
> 4400 University Drive
> Fairfax, VA 22030
>
>
>
>
>
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20150326/707e35e3/attachment.html