<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi Jennifer-<div><br></div><div>I updated to opengrads 2.0.a8.oga.1. The problem I reported seems to have been solved here. I don't understand why: whether I sdfopen open a file directly or xdfopen a templated DDF the same data file must be open and uncompressed. Presumably xdfopen-ing the DDF is doing something a little more than just that.</div><div><br></div><div>Thank you,</div><div>Pete</div><div><br></div><div><br></div><div><br></div><div><br><div><div>On Jul 13, 2010, at 8:11 AM, Jennifer Adams wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Pete, <div>The short answer is that you should upgrade to 2.0.a8. A whole lot of critical changes were implemented in the newer versions (of GrADS and the netCDF library) to improve performance when handling compressed netcdf files. Please read the doc page on compression (<a href="http://iges.org/grads/gadoc/compression.html">http://iges.org/grads/gadoc/compression.html</a>) for a full explanation of the factors at play. When the cache settings are appropriate (which they are almost certainly not in your build), you will find that the I/O for compressed netcdf can be a whole lot faster than classic netcdf and flat binary data sets. </div><div>--Jennifer</div><div><br></div><div><div><br><div><div>On Jul 12, 2010, at 9:26 PM, Colarco, Peter R. (GSFC-6133) wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi-<div><br></div><div>Apologies, here is the requested information.</div><div><br></div><div>Thanks,</div><div>Pete</div><div><br></div><div><br></div><div>OS:</div><div><div>% uname -a</div><div>Linux <a href="http://redacted.gsfc.nasa.gov/">redacted.gsfc.nasa.gov</a> 2.6.32-gentoo-r7 #2 SMP Thu Jul 8 11:39:03 EDT 2010 x86_64 Intel(R) Xeon(R) CPU X5460 @ 3.16GHz GenuineIntel GNU/Linux</div></div><div><br></div><div>File Information:</div><div><div>% ncdump -hs /path/to/file/Y2003/M01/MYD04_L2_ocn.aero_tc8_051.qawt.20030101.nc4</div><div>netcdf MYD04_L2_ocn.aero_tc8_051.qawt.20030101 {</div><div>dimensions:</div><div> time = UNLIMITED ; // (8 currently)</div><div> levels = 7 ;</div><div> longitude = 1152 ;</div><div> latitude = 721 ;</div><div>variables:</div><div> double time(time) ;</div><div> time:units = "hours since 2003-1-1 1" ;</div><div> time:_Storage = "chunked" ;</div><div> time:_ChunkSizes = 1 ;</div><div> double levels(levels) ;</div><div> levels:units = "hPa" ;</div><div> levels:description = "Pressure level" ;</div><div> levels:type = "plev" ;</div><div> levels:long_name = "Level" ;</div><div> levels:positive = "down" ;</div><div> levels:_Storage = "contiguous" ;</div><div> double longitude(longitude) ;</div><div> longitude:units = "degrees_east" ;</div><div> longitude:long_name = "Longitude" ;</div><div> longitude:_Storage = "contiguous" ;</div><div> double latitude(latitude) ;</div><div> latitude:units = "degrees_north" ;</div><div> latitude:long_name = "Latitude" ;</div><div> latitude:_Storage = "contiguous" ;</div><div> float aodtau(time, levels, latitude, longitude) ;</div><div> aodtau:comments = "Unknown1 variable comment" ;</div><div> aodtau:long_name = "aodtau" ;</div><div> aodtau:units = "" ;</div><div> aodtau:grid_name = "grid01" ;</div><div> aodtau:grid_type = "linear" ;</div><div> aodtau:time_statistic = "instantaneous" ;</div><div> aodtau:missing_value = 1.e+15f ;</div><div> aodtau:_Storage = "chunked" ;</div><div> aodtau:_ChunkSizes = 1, 1, 721, 1152 ;</div><div> aodtau:_DeflateLevel = 2 ;</div><div><br></div><div>// global attributes:</div><div> :Conventions = "COARDS" ;</div><div> :calendar = "standard" ;</div><div> :comments = "File" ;</div><div> :model = "geos/das" ;</div><div> :center = "gsfc" ;</div><div> :_Format = "netCDF-4" ;</div><div>}</div></div><div><br></div><div><br></div><div><br></div><div><div>ga-> q config</div><div>Config: v2.0.a7.oga.3 little-endian readline printim grib2 netcdf hdf4-sds hdf5 opendap-grids,stn athena geotiff</div><div>Grid Analysis and Display System (GrADS) Version 2.0.a7.oga.3</div><div>Copyright (c) 1988-2009 by Brian Doty and the</div><div>Institute for Global Environment and Society (IGES) </div><div>This program is distributed WITHOUT ANY WARRANTY </div><div>See file COPYRIGHT for more information. </div><div><br></div><div>Built Thu Oct 29 17:57:08 EDT 2009 for x86_64-unknown-linux-gnu</div><div><br></div><div>This version of GrADS has been configured with the following options:</div><div> o Built on a LITTLE ENDIAN machine</div><div> o Command line editing ENABLED </div><div> <a href="http://tiswww.case.edu/php/chet/readline/rltop.html">http://tiswww.case.edu/php/chet/readline/rltop.html</a> </div><div> o printim command for image output ENABLED </div><div> <a href="http://www.zlib.net/">http://www.zlib.net</a> </div><div> <a href="http://www.libpng.org/pub/png/libpng.html">http://www.libpng.org/pub/png/libpng.html</a> </div><div> <a href="http://www.libgd.org/Main_Page">http://www.libgd.org/Main_Page</a> </div><div> o GRIB2 interface ENABLED </div><div> <a href="http://www.ijg.org/">http://www.ijg.org</a> </div><div> <a href="http://www.ece.uvic.ca/~mdadams/jasper">http://www.ece.uvic.ca/~mdadams/jasper</a> </div><div> <a href="http://www.nco.ncep.noaa.gov/pmb/codes/GRIB2">http://www.nco.ncep.noaa.gov/pmb/codes/GRIB2</a> </div><div> g2clib-1.0.5 </div><div> o NetCDF interface ENABLED </div><div> <a href="http://www.opendap.org/">http://www.opendap.org</a> </div><div> libnc-dap 4.0.1-beta3-snapshot2009021712 of Mar 3 2009 14:13:33 $ </div><div> o HDF interface ENABLED </div><div> <a href="http://hdfgroup.org/">http://hdfgroup.org</a> </div><div> HDF 4.2r3 </div><div> HDF5 1.8.2 </div><div> o Athena Widget GUI ENABLED</div><div> o OPeNDAP gridded data interface ENABLED</div><div> <a href="http://www.opendap.org/">http://www.opendap.org</a> </div><div> libdap 3.7.10 </div><div> o OPeNDAP station data interface ENABLED</div><div> <a href="http://iges.org/grads/gadoc/supplibs.html">http://iges.org/grads/gadoc/supplibs.html</a> </div><div> libgadap 2.0.oga.1 </div><div> o GeoTIFF and KML output ENABLED</div><div> <a href="http://www.libtiff.org/">http://www.libtiff.org</a> </div><div> <a href="http://geotiff.osgeo.org/">http://geotiff.osgeo.org</a> </div><div><br></div><div>For additional information please consult <a href="http://iges.org/grads">http://iges.org/grads</a></div><div><br></div></div><div><div><div>On Jul 12, 2010, at 4:51 PM, Jennifer Adams wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Could you please send the output from ncdump -hs (the s is for compression info, since you have netcdf-4 files). Also send the output from 'q config' and the specs of the OS you are running on. These things are important for solving every problem. <div><br><div>--Jennifer</div><div><div><br><div><br><div><div>On Jul 12, 2010, at 4:42 PM, Colarco, Peter R. (GSFC-6133) wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi-<div><br></div><div>I am running opengrads 2.0.a7.oga.3/x86_64.</div><div><br></div><div>I have a very simple data descriptor file (below) to template a set of large-ish files (gridded 1152 x 721 x 7 points). When I simply "sdfopen" the first file and plot the first variable/first time/first level, I get my result essentially immediately. When I instead "xdfopen" the data descriptor file and try to plot the same, it takes some time (~10 sec) to plot. When I repeat the same sort of thing but operate on several times the difference in performance between the two methods is of course very noticeable.</div><div><br></div><div>The issue seems to be in the "options template" line below; taking that out and explicitly putting the first file in I can "xdfopen" the data descriptor file and plot the first variable essentially instantly. Additional metadata in the data descriptor file (e.g., from make_ctl.sh) does not help or matter as far as I can tell.</div><div><br></div><div>Can anyone suggest a solution to this problem? Am I doing something weird or wrong here?</div><div><br></div><div>Data Descriptor File:</div><div><br></div><div>dset /path/to/my/data/Y2003/M01/MYD04_L2_ocn.aero_tc8_051.qawt.%y4%m2%d2.nc4</div><div><div>options template</div><div>tdef time 248 linear 0z01jan2003 3hr</div><div><br></div><div>Thank you,</div><div>Pete Colarco</div><div><br></div><div><br></div><div><br></div><div> <div><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; font-size: 12px; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">--------------------------------------------------------------------</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">Peter Colarco</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">NASA GSFC</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">Code 613.3</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">NASA Goddard Space Flight Center</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">Greenbelt, MD 20771</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">301.614.6382 (ph)</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">301.614.5903 (fax)</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; "><a href="mailto:peter.r.colarco@nasa.gov">peter.r.colarco@nasa.gov</a></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; "><a href="http://hyperion.gsfc.nasa.gov/People/Colarco">http://hyperion.gsfc.nasa.gov/People/Colarco</a></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">--------------------------------------------------------------------</font></div><div><br class="webkit-block-placeholder"></div></div></div><br class="Apple-interchange-newline"></div><br class="Apple-interchange-newline"> </div> <br></div></div>_______________________________________________<br>gradsusr mailing list<br><a href="mailto:gradsusr@gradsusr.org">gradsusr@gradsusr.org</a><br><a href="http://gradsusr.org/mailman/listinfo/gradsusr">http://gradsusr.org/mailman/listinfo/gradsusr</a><br></blockquote></div><br><div apple-content-edited="true"> <div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; font-size: 12px; "><div>--</div><div>Jennifer M. Adams</div><div>IGES/COLA</div><div>4041 Powder Mill Road, Suite 302</div><div>Calverton, MD 20705</div><div><a href="mailto:jma@cola.iges.org">jma@cola.iges.org</a></div><div><br class="khtml-block-placeholder"></div><br class="Apple-interchange-newline"></div> </div><br></div></div></div></div></div><span><ATT00001..txt></span></blockquote></div><br><div> <span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div><span class="Apple-style-span" style="font-size: 12px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">--------------------------------------------------------------------</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">Peter Colarco</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">NASA GSFC</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">Code 613.3</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">NASA Goddard Space Flight Center</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">Greenbelt, MD 20771</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">301.614.6382 (ph)</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">301.614.5903 (fax)</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; "><a href="mailto:peter.r.colarco@nasa.gov">peter.r.colarco@nasa.gov</a></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; "><a href="http://hyperion.gsfc.nasa.gov/People/Colarco">http://hyperion.gsfc.nasa.gov/People/Colarco</a></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">--------------------------------------------------------------------</font></div><div><br class="webkit-block-placeholder"></div></div></div></span><br class="Apple-interchange-newline"></div></span><br class="Apple-interchange-newline"> </div> <br></div></div>_______________________________________________<br>gradsusr mailing list<br><a href="mailto:gradsusr@gradsusr.org">gradsusr@gradsusr.org</a><br><a href="http://gradsusr.org/mailman/listinfo/gradsusr">http://gradsusr.org/mailman/listinfo/gradsusr</a><br></blockquote></div><br><div apple-content-edited="true"> <span class="Apple-style-span" style="border-collapse: separate; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; -webkit-text-decorations-in-effect: none; text-indent: 0px; -webkit-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><div style="word-wrap: break-word; -khtml-nbsp-mode: space; -khtml-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; -webkit-text-decorations-in-effect: none; text-indent: 0px; -webkit-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><span class="Apple-style-span" style="border-collapse: separate; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; -webkit-text-decorations-in-effect: none; text-indent: 0px; -webkit-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><div>--</div><div>Jennifer M. Adams</div><div>IGES/COLA</div><div>4041 Powder Mill Road, Suite 302</div><div>Calverton, MD 20705</div><div><a href="mailto:jma@cola.iges.org">jma@cola.iges.org</a></div><div><br class="khtml-block-placeholder"></div><br class="Apple-interchange-newline"></span></span></div></span> </div><br></div></div></div><span><ATT00001..txt></span></blockquote></div><br><div>
<span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div><span class="Apple-style-span" style="font-size: 12px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">--------------------------------------------------------------------</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">Peter Colarco</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">NASA GSFC</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">Code 613.3</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">NASA Goddard Space Flight Center</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">Greenbelt, MD 20771</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">301.614.6382 (ph)</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">301.614.5903 (fax)</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; "><a href="mailto:peter.r.colarco@nasa.gov">peter.r.colarco@nasa.gov</a></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; "><a href="http://hyperion.gsfc.nasa.gov/People/Colarco">http://hyperion.gsfc.nasa.gov/People/Colarco</a></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; ">--------------------------------------------------------------------</font></div><div><br class="webkit-block-placeholder"></div></div></div></span><br class="Apple-interchange-newline"></div></span><br class="Apple-interchange-newline">
</div>
<br></div></body></html>