Turns out the subsetting <b>is</b> the culprit... changed it from the CONUS to US/Canada/Alaska and it actually downloads (insignificantly) faster... I'll try testing raw data downloads sometime too (especially since I can use parallel connections without bogging down the NOMADS server). Oh, and I implemented the staggered downloads and now have a lag of about 2-4 seconds for the first map and 12-20 seconds for the last map (GrADS is really fast at generating maps... at least when the right settings are used on printim).<br>
<br>Thanks,<br>Joey<br><a href="http://www.instantweathermaps.com">http://www.instantweathermaps.com</a><br><br><div class="gmail_quote">On Tue, Jul 12, 2011 at 2:42 PM, Wesley Ebisuzaki <span dir="ltr"><<a href="mailto:Wesley.Ebisuzaki@noaa.gov">Wesley.Ebisuzaki@noaa.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">Joey,<br>
<br>
My impression of the nomads server is that it is very fast.<br>
However, I have a very fast link to the internet and<br>
the site is only 10 miles away.<br>
<br>
You are using the GFS which has 0.5 degree resolution (260K<br>
points) and you are using the filter program<br>
which can create a regional subset. If you are asking for a 5x5 grid,<br>
then your download speeds will be<br>
slow. After all you have to process 10K points for every point sent to<br>
you.<br>
<br>
Another possibility is that you may have run afoul of some<br>
fair-usage filter. Every site needs some protection<br>
against denial-of-service attacks and poorly programmed web scripts.<br>
Anyways the nomads help desk is a more<br>
appropriate source of information than the GrADS user forum.<br>
<br>
Wesley Ebisuzaki<br>
<div><div></div><div class="h5"><br>
<br>
Joey Woodson wrote:<br>
> Is there any way to speed up downloads from the NOMADS server<br>
> (<a href="http://nomads.ncep.noaa.gov/cgi-bin/filter_gfs_hd.pl" target="_blank">http://nomads.ncep.noaa.gov/cgi-bin/filter_gfs_hd.pl</a>)? I currently<br>
> run a site that spends around 6-8 seconds per forecast hour generating<br>
> maps. Having said that, it is quite frustrating to see download times<br>
> for 1 MB of GRIB data exceed 10 seconds at times. I currently use 4<br>
> wget sessions in parallel to handle the downloading, and I only sleep<br>
> for 2 seconds instead of 10 when polling the NOMADS server to check<br>
> when the next file is available, so as you can see I am very<br>
> speed-oriented. Is there any way to download faster, or is<br>
> partial-downloading (downloading part of the GRIB file, generating the<br>
> relevant maps for the first part while the rest downloads, then<br>
> generating the maps for the rest) the fastest way of doing this?<br>
><br>
> Joey<br>
</div></div>> <a href="http://www.instantweathermaps.com" target="_blank">http://www.instantweathermaps.com</a> <<a href="http://www.instantweathermaps.com/" target="_blank">http://www.instantweathermaps.com/</a>><br>
> ------------------------------------------------------------------------<br>
><br>
> _______________________________________________<br>
> gradsusr mailing list<br>
> <a href="mailto:gradsusr@gradsusr.org">gradsusr@gradsusr.org</a><br>
> <a href="http://gradsusr.org/mailman/listinfo/gradsusr" target="_blank">http://gradsusr.org/mailman/listinfo/gradsusr</a><br>
><br>
<br>
_______________________________________________<br>
gradsusr mailing list<br>
<a href="mailto:gradsusr@gradsusr.org">gradsusr@gradsusr.org</a><br>
<a href="http://gradsusr.org/mailman/listinfo/gradsusr" target="_blank">http://gradsusr.org/mailman/listinfo/gradsusr</a><br>
</blockquote></div><br>