[gradsusr] Grads computation speed
Wesley Ebisuzaki
Wesley.Ebisuzaki at noaa.gov
Tue May 25 13:06:45 EDT 2010
NCEP distributes the grib2 forecasts using jpeg2000 packing which is pretty
good at compression but very slow. Grib1 one uses simple packing which
is fast. Complex packing combines speed with compression.
> 45x91 grid, 5 fields
>
> simple complex1 complex2 complex3 jpeg
> size 21884 16030 13623 13917 11514 bytes
> read 0.001 0.001 0.001 0.001 0.012 seconds user time
> write 0.002 0.006 0.006 0.006 0.017 seconds user time
>
>
> 4320x2160 high resolution SST field
>
> simple complex1 complex2 complex3 jpeg
> size 11054 4076 2365 2280 3252 K bytes
> read 0.38 0.41 0.42 0.42 2.66 seconds user time
> write 0.78 2.06 1.76 1.68 4.20 seconds user time
>
Rather than convert a grib2 file into binary, I would convert the file
to complex3 packing by
wgrib2 in.grb -set_grib_type complex3 -grib_out out.grb
Wesley
Mark Sponsler wrote:
> Not sure if this is of use, but I beleive if you dump your gribs to
> binary (using wgrib2) before hand, and then run your scripts againt
> the binary files, it will be even faster (than grib1). Of course, it
> might take a few minutes to do the conversion to binary.
>
> Thanks,
> Mark
>
>
> ----- Original Message -----
> From: "JPB" <info at a-conseil.com>
> To: "GrADS Users Forum" <gradsusr at gradsusr.org>
> Sent: Monday, May 24, 2010 11:59:33 PM GMT -08:00 US/Canada Pacific
> Subject: Re: [gradsusr] Grads computation speed
>
> Hello,
>
> grib1 is faster because it does not need decompression process
> compared to
> grib2
>
> regards,
>
> JPB
>
> Jonas Kaufmann writes:
>
> > Hello,
> >
> > which one should be faster then, grib1 or grib2? Does the projection
> > of the file have an influence on the computation speed?
> >
> > Best regards,
> > Jonas Kaufmann
> >
> > On Fri, May 21, 2010 at 15:20, JPB <info at a-conseil.com> wrote:
> >> Hello,
> >>
> >> there is a big difference in computation time between grib1 and grib2
> >> when using grads. Perhaps you have to look at the grib file types...
> >>
> >> regards,
> >> JPB
> >>
> >> Le 21 mai 10 à 15:16, Jonas Kaufmann a écrit :
> >>
> >>> Hello Michael,
> >>>
> >>> I tried on two machines with the same results. One is running Grads
> >>> Version 2.0.a1.oga.1, the other one is OpenGrads Version 2.0.a7.oga.3.
> >>> In both cases using the new version of my model was significantly
> >>> slower than the old one.
> >>>
> >>> The old model used ARWpost as WRF postprocessor, the new one uses
> >>> wrfpost. Both outputs are only customized in terms of available fields
> >>> in the Grib file, all other options should be the default ones.
> >>>
> >>> Thank you for your help!
> >>>
> >>> Best regards,
> >>> Jonas Kaufmann
> >>>
> >>> On Fri, May 21, 2010 at 15:09, Michael Fiorino
> <michael.fiorino at noaa.gov
> >>> > wrote:
> >>>> Jonas, you need to tell us the version of grads... so the old data
> >>>> is
> >>>> binary and the new data grib? ciao mike
> >>>>
> >>>> On 5/21/10 11:42 AM, Jonas Kaufmann wrote:
> >>>>> Hello,
> >>>>>
> >>>>> I just ran into a problem regarding Grads computation speed. I have
> >>>>> set up a WRF model over Europe and extract surface parameters for
> >>>>> about 4-5000 points in the model for 72 timesteps. Now I tried
> >>>>> another
> >>>>> WRF version which produces different output files, and the
> >>>>> computation
> >>>>> speed for my Grads tasks for the new model is much much slower than
> >>>>> before. You can find the first lines of my control files below.
> >>>>>
> >>>>> Using my old model, I could run my complete Grads tasks in about 90
> >>>>> minutes. Using the new setup, this takes forever to complete (had to
> >>>>> skip this as it took too long, think it should be around 180 minutes
> >>>>> or so), of course on the same machine. The main difference must be
> >>>>> in
> >>>>> the Gribfiles I am using.
> >>>>>
> >>>>> Can anyone explain why this might be happening? Which Gribfile
> >>>>> settings (version, projection etc.) are the best usable ones for
> >>>>> Grads? And finally: how can I improve computation speed for my new
> >>>>> model?
> >>>>>
> >>>>> Thank you for your answer!
> >>>>>
> >>>>> Best regards,
> >>>>> Jonas Kaufmann
> >>>>>
> >>>>>
> >>>>> Old model:
> >>>>> options byteswapped
> >>>>> pdef 161 146 lcc 47.300 9.800 81.000 73.500 47.30000
> >>>>> 47.30000 9.80000 21000.000 21000.000
> >>>>> xdef 626 linear -19.84533 0.09459461
> >>>>> ydef 324 linear 31.13766 0.09459461
> >>>>> zdef 27 linear 1 1
> >>>>> tdef 88 linear 21Z26MAY2009 60MN
> >>>>> VARS 14
> >>>>>
> >>>>> New model:
> >>>>> dtype grib 255
> >>>>> pdef 265 211 lccr 31.202000 -10.532000 1 1 47.515000 47.515000
> >>>>> 10.025000 14684 14768
> >>>>> xdef 265 linear -10.532000 0.156066529170106
> >>>>> ydef 211 linear 31.202000 0.134254545454545
> >>>>> tdef 1 linear 00Z24may2010 1mo
> >>>>> zdef 1 linear 1 1
> >>>>> vars 16
> >>>>> _______________________________________________
> >>>>> gradsusr mailing list
> >>>>> gradsusr at gradsusr.org
> >>>>> http://gradsusr.org/mailman/listinfo/gradsusr
> >>>>>
> >>>>
> >>>
> >>> _______________________________________________
> >>> gradsusr mailing list
> >>> gradsusr at gradsusr.org
> >>> http://gradsusr.org/mailman/listinfo/gradsusr
> >>>
> >>
> >>
> >> _______________________________________________
> >> gradsusr mailing list
> >> gradsusr at gradsusr.org
> >> http://gradsusr.org/mailman/listinfo/gradsusr
> >>
> >
> > _______________________________________________
> > gradsusr mailing list
> > gradsusr at gradsusr.org
> > http://gradsusr.org/mailman/listinfo/gradsusr
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
> ------------------------------------------------------------------------
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
>
More information about the gradsusr
mailing list