[gradsusr] template for multiple netcdf files

Fri Mar 11 19:39:36 EST 2011

Hang in there, Joyce. You're paving the way for all the future CMIP5 users out there. Unfortunately, you are going to have to use chsub for the template string substitutions. The file naming convention, although handy because you can know exactly which time steps are contained in each file, is not convenient for GrADS, and the only way you can do the templating is to give GrADS explicit instructions as to which time steps belong in each file (that's what CHSUB is good for). This was true for some CMIP3 data as well... 
DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_%ch.nc
CHSUB 1 360 200512010600-200603010000
etc. 
(I think you have already worked out how chsub entries should look)

Another thing I noticed about your descriptor file -- your variable "orog" does not vary in Z, so you should change the 38 to 0 in the variable declaration.  
--Jennifer

----- Original Message -----
From: "Joyce Meyerson" <hobo at atmos.ucla.edu>
To: "GrADS Users Forum" <gradsusr at gradsusr.org>
Sent: Friday, March 11, 2011 7:15:43 PM
Subject: Re: [gradsusr] template for multiple netcdf files

Hi Jennifer, 

Thank you for your patient help. 
I read the SDFdesciptorfile and saw what you were talking about wrt the variable declarations and dtype which I implemented. 
When I used only a single file it opened fine so I'm assuming the problem now lies with the template time substitution. 

I'm attaching the ncdump if that is helpful. 

Thanks again. 
Joyce 

A subset of the files to be opened (3 month chunks of 6hr data) 

-rw-r--r-- 1 csi csi 1523538404 2010-12-18 08:28 ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_200512010600-200603010000.nc 
-rw-r--r-- 1 csi csi 1523538404 2010-12-18 08:29 ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_200603010600-200606010000.nc 
-rw-r--r-- 1 csi csi 1523538404 2010-12-18 08:29 ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_200606010600-200609010000.nc 
-rw-r--r-- 1 csi csi 1523538404 2010-12-18 08:29 ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_200609010600-200612010000.nc 
-rw-r--r-- 1 csi csi 1523538404 2010-12-18 08:30 ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_200612010600-200703010000.nc 

The new .ctl file is 

DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_%y4%m2%d2%h200-%y4%m2%d2%h200.nc 
TITLE Air Temperature 
DTYPE netcdf 
UNDEF 1.e+20f 
OPTIONS template 
xdef 192 linear 0 1.875 
ydef 145 linear -90 1.25 
zdef 38 levels 20.000337600708 80.001350402832 179.999114990234 320.00146484375 500.000579833984 720.000366210938 980.000854492188 1279.998046875 1619.99987792969 1999.99841308594 2420.00170898438 2880.00146484375 3379.99829101562 3919.99951171875 4500.00146484375 5120 5779.99951171875 6479.99951171875 7220 8000.00146484375 8820 9679.9990234375 10579.998046875 11519.998046875 12499.9990234375 13520.0009765625 14580.7998046875 15694.6396484375 16875.310546875 18138.626953125 19503.009765625 20990.1875 22626.08203125 24458.28515625 26583.640625 29219.080078125 32908.69140625 39254.83203125 
tdef 1800 linear 6Z1dec2005 6hr 
VARS 3 
b 38 z vertical coordinate formula term (k) 
orog 38 y,x Surface Altitude (m) 
ta 38 t,z,y,x Air Temperature (K) 
ENDVARS 

On Mar 11, 2011, at 4:03 AM, Jennifer Adams wrote: 

Joyce, 
Please have a look at http://iges.org/grads/gadoc/SDFdescriptorfile.html 
You need 'dtype netcdf' and you need something like "t,z,y,x" instead of 0 in your variable declarations. If it is still not working, please send the output from 'ncdump -h'. 
--Jennifer 

On Mar 11, 2011, at 12:04 AM, Joyce Meyerson wrote: 

Thanks Jennifer. Looks like I'm getting close. 
After I removed "time" in TDEF it complained and wanted xdef, ydef, etc. I input that info and can now "open" the file with no errors but cannot display anything. "Entire Grid Undefined". 

Control file now looks like 

DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_%y4%m2%d2%h200-%y4%m2%d2%h200.nc 
TITLE Air Temperature 
UNDEF 1.e+20f 
OPTIONS template 
xdef 192 linear 0 1.875 
ydef 145 linear -90 1.25 
zdef 38 levels 20.000337600708 80.001350402832 179.999114990234 320.00146484375 500.000579833984 720.000366210938 980.000854492188 1279.998046875 1619.99987792969 1999.99841308594 2420.00170898438 2880.00146484375 3379.99829101562 3919.99951171875 4500.00146484375 5120 5779.99951171875 6479.99951171875 7220 8000.00146484375 8820 9679.9990234375 10579.998046875 11519.998046875 12499.9990234375 13520.0009765625 14580.7998046875 15694.6396484375 16875.310546875 18138.626953125 19503.009765625 20990.1875 22626.08203125 24458.28515625 26583.640625 29219.080078125 32908.69140625 39254.83203125 
TDEF 1800 linear 6Z1dec2005 6hr 
VARS 3 
b 38 0 vertical coordinate formula term: b(k) 
orog 38 0 Surface Altitude m 
ta 38 0 Air Temperature K 
ENDVARS 

??? 
Joyce 

On Mar 10, 2011, at 6:30 PM, Jennifer Adams wrote: 

Joyce, 
Take the 'time' keyword out of your TDEF statement -- that is for 'xdfopen' style descriptors, and you are using 'open'. 
--Jennifer 

On Mar 10, 2011, at 8:57 PM, Joyce Meyerson wrote: 

Thanks Jeff, 

Unfortunately, when I tried 

DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_%y4%m2%d2%h200-%y4%m2%d2%h200.nc 
OPTIONS template 
TDEF time 1800 linear 6Z1dec2005 6hr 

I get the message 

ga-> open test2.ctl 
Scanning description file: test2.ctl 
Open Error: Missing or invalid dimension size. 
--> The invalid description file record is: 
--> TDEF time 1800 linear 6Z1dec2005 6hr 
The data file was not opened. 

Joyce 

On Mar 10, 2011, at 5:19 PM, Jeffrey Duda wrote: 

Joyce, 
You have the general idea of templating down right, but you do have a few template strings that are not right. If the last four digits in the dates of your filenames are just four-digit UTC times, use %h200. If they're forecast hours, use %f30. Also, I think you should only have one file name listed in the DSET entry (i.e., don't double up on the file names, just leave the templated file name string there instead). 

Unfortunately if that change doesn't work, that's as far as I can go to help you, as I don't use sdfopen and it could be an issue related to that. 

Hope this helps. 

Jeff Duda 

On Thu, Mar 10, 2011 at 6:58 PM, Joyce Meyerson < hobo at atmos.ucla.edu > wrote: 

Hi All, 

I am trying to write a control file to open multiple CMIP5 6hr .nc files. 

I am running grads-2.0.a8 on Linux version 2.6.27.45-server-1mnb. 

My files are 6h temperature in 3 month bundles and look like 
ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_200512010600-200603010000.nc 
ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_200603010600-200606010000.nc 
ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_200606010600-200609010000.nc 
... 

They open fine individually with sdfopen. 
All the examples I've seen on the user list and elsewhere more or less have names that are simple sequential (e.g., pr.1998.nc , pr.1999.nc , etc. or month, day, year, that kind of thing). 

I'm not sure of the nomenclature with a time interval in the name like the above (xxx_%y4%m2%d2%t4-%y4%m2%d2% t4.nc ) 

I tried %ch 

DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_% ch.nc 
CHSUB 1 360 200512010600-200603010000 
CHSUB 361 720 200603010600-200606010000 
OPTIONS template 
DTYPE netcdf 
TDEF time 720 linear 6Z1dec2005 6hr 
with the error message: 

ga-> open test.ctl 
Scanning description file: test.ctl 
Open Error: Missing or invalid dimension size. 
--> The invalid description file record is: 
--> TDEF time 720 linear 6Z1dec2005 6hr 
The data file was not opened. 

I tried 

DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_200512010600-200603010000.nc ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_%y4%m2%d2%t4-%y4%m2%d2% t4.nc 
OPTIONS template 
TDEF time 1800 linear 6Z1dec2005 6hr 
[opens just the first file] 

and I tried 

DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_%y4%m2%d2%t4-%y4%m2%d2% t4.nc 
OPTIONS template 
TDEF time 1800 linear 6Z1dec2005 6hr 
[gives error message: read_metadata: gaopfn failed (rc=-88888) 
gadsdf: Couldn't ingest SDF metadata. 
SDF Descriptor file test2.ctl was not successfully opened & parsed.] 

I'm not the most adept grads/computer/scripting person. Please help. 

Joyce 

************************** 
Joyce Meyerson 
UCLA 
Dept. of Oceanic & Atmospheric Sciences 
7235 Math Science Bld. 
Los Angeles, CA 90095-1565 
hobo at atmos.ucla.edu 

_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org
http://gradsusr.org/mailman/listinfo/gradsusr