[gradsusr] GrADS performance vs alternatives?

Mon Apr 11 21:22:20 EDT 2022

The original question didn't mention which version of GrADS is in use. The more recent ones have a metafile buffer that gets dynamically allocated as you start drawing stuff. The -m option on startup allows you to set a maximum buffer size. As you use 'draw' or 'display' commands, each graphical element gets added to the metabuffer. Check the memory usage of the grads executable while the drawing is going on to see if that memory footprint is growing steadily to a point where you're reaching system limits. Use 'reinit' or 'undefine' to keep memory use in check. When rendering an image, GrADS goes through the metabuffer and draws each line, dot, polygon, or letter. If you're drawing a grfill plot of a very high resolution grid, that's a lot of little tiny squares to draw and it can take a while. Then if you add basemap to mask out land or ocean, that's a whole lot of polygons to draw on top of that, and the more you layer on with contour line overlays or labels and other cosmetics... you get the idea. A rendering time of 1.5 seconds doesn't seem like a lot, but I can see where it would scale up. Use maskout() or negative numbers in your 'set ccols' command to avoid drawing filler you don't need in your graphic. Maybe there's some things you can pre-draw and use as a background or foreground image (merged with the -b or -f option with gxprint) so they don't have to be rendered multiple times. Or check if you are needlessly drawing stuff that will get masked out by another layer on top.

If you're testing without using batch mode, does it take 1.5 seconds to draw the graphic in the display window? Watch and see if you can tell what takes the longest to draw for each of your plot styles, maybe that will shed some light on what's slowing down your image production.

Of course, it may just be a basic truth that your beautiful GrADS plots are going to need 1.5 seconds to get ready for prime time. 😄
--Jennifer

________________________________
From: gradsusr <gradsusr-bounces at gradsusr.org> on behalf of Mark Sponsler <msponsler at comcast.net>
Sent: Monday, April 11, 2022 12:21 PM
To: GrADS Users Forum <gradsusr at gradsusr.org>
Subject: Re: [gradsusr] GrADS performance vs alternatives?

One other consideration is HOW your data is stored. Reading grib files versus reading binary files I believe is inherently quicker.

Try coverting your grib2 files to binary using wgrib2 before you start running scrpits to produce images.

Just a thought.
Mark

On April 11, 2022 1:33:15 AM PDT, Davide Sacchetti <davide.sacchetti at arpal.liguria.it> wrote:
I see
there are different ways to produce png outputs, depending also on grads version
eg: gxprint, printim, print: you could investigate which method is faster.
when I have to produce lots of png I divide the number of png processing among the available cpus, but I immagine you already did this

On my knowledge it is difficult to read and render data from grib or netcdf faster than grads does, the only chance I imagine is to take advantage of parallelization

Davide

Dott. Davide Sacchetti
Dipartimento Stato dell’Ambiente e Tutela dai Rischi Naturali
U.O. Clima Meteo Idro
Viale Brigate Partigiane 2, 16129 Genova (I)
tel: +39 0106437535
mail: davide.sacchetti at arpal.liguria.it<mailto:mauro.damonte at arpal.liguria.it>  web: https://www.arpal.liguria.it/homepage/meteo.html<http://secure-web.cisco.com/1gZr8j65sWn3pQixShqA64yx2-ElCh3P2HieHemBe2Gk4wfXBgHys44Avjcj6ZTK1C3CIq764eOP1ntiV6A9ZXZfXzVtBROgZx5LAf017ewplrcVXyW3NeWn_CNFF24alGBreWMBMPcab0Nh5SHId-7FZl5X3rQx6_R8pCAUymoAUfX4SrlMbg-2dycf8dphl8V3m7SWWAQugf1JjfIar0Ol_xc--xJnJs7L624ihdh4IPkxQoaZvPBPKqzL2bSTAjPqSu4fCAKW_eol8TODIjRNNkaxLeNdazFqEy5wta4aHKK_ypzH3FYJcqkRtM5IBkYOXnUSPuvWLUaV_BUI9fbACyDbOZnVABcrLoFXC6X6G4p2aXxf05DkHEMJFnGJkBMxAMqOYtYx5Mn0TTmcQd1K-2Kj2AHGulwDZRZasvmCzhD5hE31ZUuDBTG3RjgPu/http%3A%2F%2Fwww.meteoliguria.it%2F>

Da: Brian Gaze <brianw.gaze at googlemail.com>
A: <gradsusr at gradsusr.org>
Inviato: 08/04/2022 19:03
Oggetto: [gradsusr] GrADS performance vs alternatives?

Thanks David.

The data is stored locally on high speed SSDs.

Any tips with scripting? My professional background is software development so I'm reasonably comfortable working with grads script. However, I've not found many tweaks that make a great deal of difference in terms of performance.

Thanks

Brian

On Fri, 8 Apr 2022 at 17:08, <gradsusr-request at gradsusr.org<mailto:gradsusr-request at gradsusr.org>> wrote:
Send gradsusr mailing list submissions to
        gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>

To subscribe or unsubscribe via the World Wide Web, visit
        http://gradsusr.org/mailman/listinfo/gradsusr<https://secure-web.cisco.com/163dD1XGeO-hjnmSvUANLOGWcV4Myz0XcxmmeR5AW1l9fMIYZvifQ1cSy3DXLU9gJg7IRXjSZlFt9uWucCXDBEopejhCeEZySbPKE3VK7QLpGr6pP98nvm_QJA4QTBhMlg_8ndf-6_dqqY8nHCdZn94z-KHMSuu9xfcCHGqtmT73Q-S-huFoJ7Dw2rlXf1vIAMCGXHB5zHshe3D4ZmM-LTViFUQQbg4E8CN8aeZZVC_Y1962Gnh6bONjXd0Semezdg8hKlXZV21mqWKEUT5i6q9kbCEtgQ84DzV9j5qhpwgi_T3SDGlB5j34uI3sjuF9lLa8OBG4yQZDwM4hCAfRu9KXP019jxqG2g87KZ9XuxXoQEVXS5jc_EZuA6gIMZj8pnHgl_NOrXLDno5zBLaA78ib-i8RHTPZLvtDZYBdV2mbqnFgENQ9_DkCYCRNfmP66/https%3A%2F%2Furlsand.esvalabs.com%2F%3Fu%3Dhttp%253A%252F%252Fgradsusr.org%252Fmailman%252Flistinfo%252Fgradsusr%26e%3D7312d17f%26h%3D5ee7e1e3%26f%3Dy%26p%3Dn>
or, via email, send a message with subject or body 'help' to
        gradsusr-request at gradsusr.org<mailto:gradsusr-request at gradsusr.org>

You can reach the person managing the list at
        gradsusr-owner at gradsusr.org<mailto:gradsusr-owner at gradsusr.org>

When replying, please edit your Subject line so it is more specific
than "Re: Contents of gradsusr digest..."

Today's Topics:

   1. Re: GrADS performance vs alternatives? (Davide Sacchetti)

----------------------------------------------------------------------

Message: 1
Date: Thu, 7 Apr 2022 18:06:10 +0200
From: Davide Sacchetti <davide.sacchetti at arpal.liguria.it<mailto:davide.sacchetti at arpal.liguria.it>>
To: GrADS Users Forum <gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>>
Subject: Re: [gradsusr] GrADS performance vs alternatives?
Message-ID: <51862359-4676 at mbox.arpal.liguria.it<mailto:51862359-4676 at mbox.arpal.liguria.it>>
Content-Type: text/plain; charset="utf-8"

the speed may depend on cpu but also on where your data is stored (are your datasets available locally?) and how your script is written ...
Upon my knowloedge GrADS IS fast, it may be the fastest sw you can find everywhere ...
Davide

Dott. Davide Sacchetti
Dipartimento Stato dell?Ambiente e Tutela dai Rischi Naturali
U.O. Clima Meteo Idro
Viale Brigate Partigiane 2, 16129 Genova (I)
tel: +39 0106437535
mail:?davide.sacchetti at arpal.liguria.it<mailto:davide.sacchetti at arpal.liguria.it>? web:?https://www.arpal.liguria.it/homepage/meteo.html<https://secure-web.cisco.com/1LIZPEjfzGBuSGwKcHKEOukwPXRwTCBIU_CPX8ZtsY9O0P34nXLlUGBlSD7PjeNRIWdDE_51LCqDVo9FNQERjSjwEk8dXp1e_fN00HUbJpWWgTOyFw5HWXD8hX_j_FI0Ue43-KKvchAHGg0qo_1kmJ3YZ1zobRPH2Op9HZISH1mAIHbZxtfyFv-bNXiWQGzXIX5YAzYeF-pT09ko1ZVRIUiXYOQwXQkxxRpdjK8D6DPfYEbT84hIGKwyiEFOlh2CTvcnlosOScgdcB9Jj43YFABpmRqgIwy6BqGZz_XzN-DgLvQQMu7Ch5FsRCI2GqIVZWywxj-33xVHUIcw_ftdOp09IHmFMqUq8bYA9RQvEtdVLPolAwwrANerDWFjkRUwhW_hOdkR9JOyRgYbyHLs_kVJNa-BKiHvQeLqtnmxEHfrca06ypJGZ3b0U1O9RLBYV/https%3A%2F%2Furlsand.esvalabs.com%2F%3Fu%3Dhttps%253A%252F%252Fwww.arpal.liguria.it%252Fhomepage%252Fmeteo.html%26e%3D7312d17f%26h%3D07ef49f6%26f%3Dy%26p%3Dn>

 Da:   Brian Gaze <brianw.gaze at googlemail.com<mailto:brianw.gaze at googlemail.com>>
 A:   <gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>>
 Inviato:   07/04/2022 17:07
 Oggetto:   [gradsusr] GrADS performance vs alternatives?

                 Questa ? la prima volta che ricevi un'email da questo mittente.         Assicurati che sia qualcuno di cui ti fidi.
Hi,

I've recently started generating ECMWF 50 member ensemble plots using a GrADS script for a large latitude - longitude range. Technically it all works fine, but the issue I'm having is performance. It is dependent of course on CPU speed, but even with a very fast processor each individual graph can take approx 1.5 seconds to render and save.?

To an extent running instances on multiple cores helps, but generating a range of graphs (e.g. 2m temps, 850hPa temps, MSLP etc...) can take a long time. Perhaps 40 mins or so for a country the size of the United Kingdom.?

Does anyone have tips for how to optimize performance in the script? Alternatively, is there software available which runs much faster than GrADS script to do the same thing??

Thanks

BWG

_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
 https://urlsand.esvalabs.com/?u=http%3A%2F%2Fgradsusr.org%2Fmailman%2Flistinfo%2Fgradsusr&e=7312d17f&h=5ee7e1e3&f=y&p=n<https://secure-web.cisco.com/163dD1XGeO-hjnmSvUANLOGWcV4Myz0XcxmmeR5AW1l9fMIYZvifQ1cSy3DXLU9gJg7IRXjSZlFt9uWucCXDBEopejhCeEZySbPKE3VK7QLpGr6pP98nvm_QJA4QTBhMlg_8ndf-6_dqqY8nHCdZn94z-KHMSuu9xfcCHGqtmT73Q-S-huFoJ7Dw2rlXf1vIAMCGXHB5zHshe3D4ZmM-LTViFUQQbg4E8CN8aeZZVC_Y1962Gnh6bONjXd0Semezdg8hKlXZV21mqWKEUT5i6q9kbCEtgQ84DzV9j5qhpwgi_T3SDGlB5j34uI3sjuF9lLa8OBG4yQZDwM4hCAfRu9KXP019jxqG2g87KZ9XuxXoQEVXS5jc_EZuA6gIMZj8pnHgl_NOrXLDno5zBLaA78ib-i8RHTPZLvtDZYBdV2mbqnFgENQ9_DkCYCRNfmP66/https%3A%2F%2Furlsand.esvalabs.com%2F%3Fu%3Dhttp%253A%252F%252Fgradsusr.org%252Fmailman%252Flistinfo%252Fgradsusr%26e%3D7312d17f%26h%3D5ee7e1e3%26f%3Dy%26p%3Dn>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gradsusr.org/pipermail/gradsusr/attachments/20220407/a20b8590/attachment-0001.html<https://secure-web.cisco.com/12AezH2TlXxg4q3edBAH9cssl9qlPgHU928uAsy6oUBjLPuNOMN9gr45aNKl0DC0PObuVL93D0WrlBQND_SO9R6wS5czhAMD4g_3gdlCjduoTxlKUDKdB-v-zk7y26auowee86bl3EVKIeopIFWeTgnFd8ztf4Q1nb0xX-OXh9ZGf3mf04WJdPbDHXFBPVfme9xe9SKeJ0PhzB0eS5_45kY0Jes_FGhmkkZv7m_Ktug6jgPiRXrHhKxj6-9tHjWlCIPtAshlJxPuLmXR9eWJL0TU75jWOWEjCfuBAUwUUPbt-AY-hP-T92X0CpojKoD9vmuYUv42R_wa2dnkrfu6eFqdRGkQfrhbHraasbYbipub0lwyqQJXEQOZ2Sh0QvNYNPIfWak5_nZGQbQWx9PHR0q5xe-e3sehg6tw3AZdnyqjuXLZ8P267gDaTxNtLM0kF/https%3A%2F%2Furlsand.esvalabs.com%2F%3Fu%3Dhttp%253A%252F%252Fgradsusr.org%252Fpipermail%252Fgradsusr%252Fattachments%252F20220407%252Fa20b8590%252Fattachment-0001.html%26e%3D7312d17f%26h%3D296be31e%26f%3Dy%26p%3Dn>>

------------------------------

Subject: Digest Footer

_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr<https://secure-web.cisco.com/163dD1XGeO-hjnmSvUANLOGWcV4Myz0XcxmmeR5AW1l9fMIYZvifQ1cSy3DXLU9gJg7IRXjSZlFt9uWucCXDBEopejhCeEZySbPKE3VK7QLpGr6pP98nvm_QJA4QTBhMlg_8ndf-6_dqqY8nHCdZn94z-KHMSuu9xfcCHGqtmT73Q-S-huFoJ7Dw2rlXf1vIAMCGXHB5zHshe3D4ZmM-LTViFUQQbg4E8CN8aeZZVC_Y1962Gnh6bONjXd0Semezdg8hKlXZV21mqWKEUT5i6q9kbCEtgQ84DzV9j5qhpwgi_T3SDGlB5j34uI3sjuF9lLa8OBG4yQZDwM4hCAfRu9KXP019jxqG2g87KZ9XuxXoQEVXS5jc_EZuA6gIMZj8pnHgl_NOrXLDno5zBLaA78ib-i8RHTPZLvtDZYBdV2mbqnFgENQ9_DkCYCRNfmP66/https%3A%2F%2Furlsand.esvalabs.com%2F%3Fu%3Dhttp%253A%252F%252Fgradsusr.org%252Fmailman%252Flistinfo%252Fgradsusr%26e%3D7312d17f%26h%3D5ee7e1e3%26f%3Dy%26p%3Dn>

------------------------------

End of gradsusr Digest, Vol 145, Issue 2
****************************************

_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org
https://urlsand.esvalabs.com/?u=http%3A%2F%2Fgradsusr.org%2Fmailman%2Flistinfo%2Fgradsusr&e=7312d17f&h=5ee7e1e3&f=y&p=n
--
Thanks
Mark Sponsler
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gradsusr.org/pipermail/gradsusr/attachments/20220412/8eff85c5/attachment-0001.html>