[gradsusr] Performance hit with data descriptor files?

[Jennifer Adams]

Could you please send the output from ncdump -hs (the s is for  
compression info, since you have netcdf-4 files). Also send the output  
from 'q config' and the specs of the OS you are running on. These  
things are important for solving every problem.

--Jennifer


On Jul 12, 2010, at 4:42 PM, Colarco, Peter R. (GSFC-6133) wrote:

> Hi-
>
> I am running opengrads 2.0.a7.oga.3/x86_64.
>
> I have a very simple data descriptor file (below) to template a set  
> of large-ish files (gridded 1152 x 721 x 7 points).  When I simply  
> "sdfopen" the first file and plot the first variable/first time/ 
> first level, I get my result essentially immediately.  When I  
> instead "xdfopen" the data descriptor file and try to plot the same,  
> it takes some time (~10 sec) to plot.  When I repeat the same sort  
> of thing but operate on several times the difference in performance  
> between the two methods is of course very noticeable.
>
> The issue seems to be in the "options template" line below; taking  
> that out and explicitly putting the first file in I can "xdfopen"  
> the data descriptor file and plot the first variable essentially  
> instantly.  Additional metadata in the data descriptor file (e.g.,  
> from make_ctl.sh) does not help or matter as far as I can tell.
>
> Can anyone suggest a solution to this problem?  Am I doing something  
> weird or wrong here?
>
> Data Descriptor File:
>
> dset /path/to/my/data/Y2003/M01/MYD04_L2_ocn.aero_tc8_051.qawt. 
> %y4%m2%d2.nc4
> options template
> tdef time 248 linear 0z01jan2003 3hr
>
> Thank you,
> Pete Colarco
>
>
>
> --------------------------------------------------------------------
> Peter Colarco
> NASA GSFC
> Code 613.3
> NASA Goddard Space Flight Center
> Greenbelt, MD 20771
> 301.614.6382 (ph)
> 301.614.5903 (fax)
>
> peter.r.colarco at nasa.gov
> http://hyperion.gsfc.nasa.gov/People/Colarco
> --------------------------------------------------------------------
>
>
>
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr

--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jma at cola.iges.org



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20100712/9a26c27b/attachment-0003.html