Memory optimization (was Re: Performance tuning)

[Kevin M Levey]

WED 17SEP08: 1030PDT

Hi Arlindo

Below is a snippet you contributed and I'd like to know in a little
more detail what you mean by

"Grib-2 is a good format for data transmission but is far less less
optimal for day to day use. For example, the performance of your
meteograms scripts will go up by a lot with grib-1"

Are you saying that GRADS processes GRIB1 data much better/faster than
GRIB2 data? Also more specifically, does GRADS 2x take longer to
process GRIB2 data than say GRADS 1.9RC1 takes to process GRIB1 using
the same grads script?

Right now I've run into a time issue using the GFS 0.5x0.5 GRIB2 data
using your GRADS 1.9RC1 + GEX  - I thought it was merely the result of
using higher res data that caused the time issue (i.e. taking very
long to process all the necessary scripts).

Would it be more efficient to use GRIB1 data instead of GRIB2 data? As
you said, the conversion factor is often the problem point here as
does take some time to convert the 0.5x0.5 GFS GRIB2 data to GRIB1 and
is not possible to do in our case as it takes too long to convert 61
time steps between 0 and 180 hours, even with high end Linux servers.

A little more clarification would probably help us all.

Many thanks

Regards

Kevin M Levey, MSc in Oceans and Atmospheric Sciences (University of
Cape Town)
Director of Meteorological Operations
CustomWeather, Inc.
San Francisco, California, USA

"Taking the World by Storm!"

http://www.customweather.com
http://www.myforecast.com
http://www.1stweather.com

cell: 415-794-0411
work: 415-777-3566
email: klevey at customweather.com




On Sep 17, 2008, at 09/17/08 - 4:43 AM, Arlindo da Silva wrote:

> One very simple solution is to add a GRIB-2 to GRIB-1 filter along
> the way, to convert the GRIB-2 to GRIB-1 format. Grib-2 is a good
> format for data transmission but is far less less optimal for day to
> day use. For example, the performance of your meteograms scripts
> will go up by a lot with grib-1. Of course, you have to factor in
> the cost of conversion.
>
>   Arlindo