<div dir="ltr"><div><div><div>Sam,<br><br></div> I guess that you didn't see much of a speed up because the data had been decoded and<br>then cached. You can regrid the data using wgrib2.<br><br></div> wgrib2 IN.grb -set_grib_type c3 -new_grid_winds earth -new_grid latlon LON0:NLON:DLON LAT0:NLAT:DLAT OUT.grb<br>
<br></div> Wesley<br><br><div><br><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Apr 25, 2013 at 5:46 PM, Sam Wilson <span dir="ltr"><<a href="mailto:sam@surfline.com" target="_blank">sam@surfline.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="font-size:14px;word-wrap:break-word">
<div>
<div>
<div style="font-family:Calibri,sans-serif">Hi Wesley,</div>
<div style="font-family:Calibri,sans-serif"><br>
</div>
<div style="font-family:Calibri,sans-serif">Thank you for your reply and sorry for the slow response. </div>
<div style="font-family:Calibri,sans-serif"><br>
</div>
<div style="font-family:Calibri,sans-serif">I checked the packing like you suggested and the NAM model files I'm using are compressed with jpeg2000, hence the slowness.</div>
<div style="font-family:Calibri,sans-serif"><br>
</div>
<div style="font-family:Calibri,sans-serif">I did what you suggested by converting the grb2 files to complex packing, but I'm still having problems with slowness – it didn't seem to buy me much.</div>
<div style="font-family:Calibri,sans-serif"><br>
</div>
<div style="font-family:Calibri,sans-serif">I'm thinking that if there was a way I could change the grid used from Lambert Conformal to just a regular lat/lon grid, which would eliminate the need for a PDEF entry in the ctl file, it would speed things up
a lot. I've checked around online, but haven't had any luck.</div>
<div style="font-family:Calibri,sans-serif"><br>
</div>
<div style="font-family:Calibri,sans-serif">Do you know if there is a way to do that using lats4d or the re() function? Or anything else? </div>
<div style="font-family:Calibri,sans-serif"><br>
</div>
<div style="font-family:Calibri,sans-serif">Thanks again for your help.</div>
<div style="font-family:Calibri,sans-serif"><br>
</div>
<div style="font-family:Calibri,sans-serif">Best,</div>
<div style="font-family:Calibri,sans-serif">Sam</div>
<div>
<div style="font-family:Calibri,sans-serif"><br>
</div>
</div>
</div>
</div>
<div style="font-family:Calibri,sans-serif"><br>
</div>
<span style="font-family:Calibri,sans-serif">
<div style="border-right:medium none;padding-right:0in;padding-left:0in;padding-top:3pt;text-align:left;font-size:11pt;border-bottom:medium none;font-family:Calibri;border-top:#b5c4df 1pt solid;padding-bottom:0in;border-left:medium none">
<span style="font-weight:bold">From: </span>Wesley Ebisuzaki - NOAA Federal <<a href="mailto:wesley.ebisuzaki@noaa.gov" target="_blank">wesley.ebisuzaki@noaa.gov</a>><br>
<span style="font-weight:bold">Reply-To: </span>GrADS Users Forum <<a href="mailto:gradsusr@gradsusr.org" target="_blank">gradsusr@gradsusr.org</a>><br>
<span style="font-weight:bold">Date: </span>Wed, 17 Apr 2013 07:54:29 -0400<br>
<span style="font-weight:bold">To: </span>GrADS Users Forum <<a href="mailto:gradsusr@gradsusr.org" target="_blank">gradsusr@gradsusr.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [gradsusr] Faster point output from grb data files<br>
</div>
<div><br>
</div>
<div dir="ltr">
<div>
<div>
<div>Sam,<br>
<br>
</div>
Grib2 files compressed with jpeg2000 are slow to read. You can determine<br>
the type of compressing by using <br>
<br>
</div>
wgrib2 IN.grb -packing<br>
<br>
</div>
<div>You can speed up the reading of jpeg compressed grib2 files by converting<br>
the file to complex packing.<br>
<br>
</div>
<div> wgrib2 IN.grb -set_grib_type c3 -grib_out OUT.grb<br>
<br>
</div>
<div>c3 is good for smooth fields and c1 is good for noisy fields. Many people<br>
use jpeg compression because it often makes the smallest files.<br>
</div>
<div class="gmail_extra"><br>
</div>
<div class="gmail_extra">After converting the file, the index files will have to be remade.<br>
<br>
</div>
<div class="gmail_extra"> Wesley<br>
<br>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Tue, Apr 16, 2013 at 7:16 PM, Sam Wilson <span dir="ltr">
<<a href="mailto:sam@surfline.com" target="_blank">sam@surfline.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="font-size:14px;word-wrap:break-word">
<div>
<div>
<div style="font-family:Calibri,sans-serif">Hi,</div>
<div style="font-family:Calibri,sans-serif"><br>
</div>
<div style="font-family:Calibri,sans-serif">I have a set of grads scripts that generate point output of various variables (wind, pressure, etc.) for many locations using different model datasets in grb format. I'm using the gr2stn function to get the output
at the station locations. Most of my scripts run very quickly and generate data for thousands of locations. However, some of them are very slow and will take many hours to get output at all the points I want. The only difference between the scripts are
the dataset that they are accessing and using to generate the output. </div>
<div style="font-family:Calibri,sans-serif"><br>
</div>
<div style="font-family:Calibri,sans-serif">I'm trying to determine why some of them run very slow and others do not. I assumed the reason was the size of the grb files being used – if the grb files were very large then it would be slower and vice versa. As
a test, I decreased the number of variables being included in the grb file set that was very large to reduce the size of those grb files. I then ran the script to generate the point output from the smaller grb files and it didn't seem to buy me much run time.</div>
<div style="font-family:Calibri,sans-serif"><br>
</div>
<div style="font-family:Calibri,sans-serif">I did notice that the grb files that are associated with the slow processing have a PDEF entry in the .ctl file, whereas the grb files that process very fast have no PDEF entry. Could this be the cause? Does it
take a lot longer to get point output from data that has to be mapped to a rectilinear lat/lon grid (has a PDEF entry) versus data that is already on a rectilinear lat/lon grid (no PDEF entry)? If this is the case, does anyone know of a way around this to
get faster point output from grb files?</div>
<div style="font-family:Calibri,sans-serif"><br>
</div>
<div style="font-family:Calibri,sans-serif">Thanks for your time and any help you can provide.</div>
<div style="font-family:Calibri,sans-serif"><br>
</div>
<div style="font-family:Calibri,sans-serif">Best,</div>
</div>
</div>
<div style="font-family:Calibri,sans-serif">Sam </div>
</div>
<br>
_______________________________________________<br>
gradsusr mailing list<br>
<a href="mailto:gradsusr@gradsusr.org" target="_blank">gradsusr@gradsusr.org</a><br>
<a href="http://gradsusr.org/mailman/listinfo/gradsusr" target="_blank">http://gradsusr.org/mailman/listinfo/gradsusr</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
_______________________________________________ gradsusr mailing list <a href="mailto:gradsusr@gradsusr.org" target="_blank">
gradsusr@gradsusr.org</a> <a href="http://gradsusr.org/mailman/listinfo/gradsusr" target="_blank">
http://gradsusr.org/mailman/listinfo/gradsusr</a> </span>
</div>
<br>_______________________________________________<br>
gradsusr mailing list<br>
<a href="mailto:gradsusr@gradsusr.org">gradsusr@gradsusr.org</a><br>
<a href="http://gradsusr.org/mailman/listinfo/gradsusr" target="_blank">http://gradsusr.org/mailman/listinfo/gradsusr</a><br>
<br></blockquote></div><br></div>