[gradsusr] template for multiple netcdf files

Jennifer Adams jma at cola.iges.org
Sat Mar 12 06:52:32 EST 2011


You need to keep "OPTIONS template" in there when you have %ch in the  
DSET entry.  --Jennifer

On Mar 11, 2011, at 8:46 PM, Joyce Meyerson wrote:

> Well, I'm certainly learning as I go along. Prior to this I've only  
> used .ctl files with binary data or simple time defs with .nc and  
> xdfopen.
>
> I put the lines for DSET and CHSUB and have tried a couple  
> variations including just DSET ^%ch and then listing the full path  
> all without success.
> I double checked the filename just to be sure. I'm running v2.0.a8  
> which seems to be the second to last version.
>
> DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_%ch.nc
> CHSUB 1 360 200603010600-200606010000
> TITLE   Air Temperature
> DTYPE   netcdf
> UNDEF   1.e+20f
> *OPTIONS template
> xdef    192 linear 0 1.875
> ydef    145 linear -90 1.25
> zdef    38  levels 20.000337600708 80.001350402832 179.999114990234  
> 320.00146484375 500.000579833984 720.000366210938 980.000854492188  
> 1279.998046875 1619.99987792969 1999.99841308594 2420.00170898438  
> 2880.00146484375 3379.99829101562 3919.99951171875 4500.00146484375  
> 5120 5779.99951171875 6479.99951171875 7220 8000.00146484375 8820  
> 9679.9990234375 10579.998046875 11519.998046875 12499.9990234375  
> 13520.0009765625 14580.7998046875 15694.6396484375 16875.310546875  
> 18138.626953125 19503.009765625 20990.1875 22626.08203125  
> 24458.28515625 26583.640625 29219.080078125 32908.69140625  
> 39254.83203125
> tdef 1800 linear 6Z1dec2005 6hr
> VARS 3
> b       38    z         vertical coordinate formula term (k)
> orog    0     y,x       Surface Altitude  (m)
> ta      38    t,z,y,x   Air Temperature (K)
> ENDVARS
>
> ga-> open test3.ctl
> Scanning description file:  test3.ctl
> Error: nc_open failed to open file ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_ 
> %ch.nc
> No such file or directory
>
> or with full path
> ga-> open test3.ctl
> Scanning description file:  test3.ctl
> Error: nc_open failed to open file %ch
>
>
> On Mar 11, 2011, at 4:39 PM, Jennifer Adams wrote:
>
>> Hang in there, Joyce. You're paving the way for all the future  
>> CMIP5 users out there. Unfortunately, you are going to have to use  
>> chsub for the template string substitutions. The file naming  
>> convention, although handy because you can know exactly which time  
>> steps are contained in each file, is not convenient for GrADS, and  
>> the only way you can do the templating is to give GrADS explicit  
>> instructions as to which time steps belong in each file (that's  
>> what CHSUB is good for). This was true for some CMIP3 data as well...
>> DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_%ch.nc
>> CHSUB 1 360 200512010600-200603010000
>> etc.
>> (I think you have already worked out how chsub entries should look)
>>
>> Another thing I noticed about your descriptor file -- your variable  
>> "orog" does not vary in Z, so you should change the 38 to 0 in the  
>> variable declaration.
>> --Jennifer
>>
>>
>> ----- Original Message -----
>> From: "Joyce Meyerson" <hobo at atmos.ucla.edu>
>> To: "GrADS Users Forum" <gradsusr at gradsusr.org>
>> Sent: Friday, March 11, 2011 7:15:43 PM
>> Subject: Re: [gradsusr] template for multiple netcdf files
>>
>>
>> Hi Jennifer,
>>
>>
>> Thank you for your patient help.
>> I read the SDFdesciptorfile and saw what you were talking about wrt  
>> the variable declarations and dtype which I implemented.
>> When I used only a single file it opened fine so I'm assuming the  
>> problem now lies with the template time substitution.
>>
>>
>> I'm attaching the ncdump if that is helpful.
>>
>>
>> Thanks again.
>> Joyce
>>
>>
>>
>>
>>
>>
>> A subset of the files to be opened (3 month chunks of 6hr data)
>>
>> -rw-r--r-- 1 csi csi 1523538404 2010-12-18 08:28 ta_6hrLev_HadGEM2- 
>> ES_rcp45_r1i1p1_200512010600-200603010000.nc
>> -rw-r--r-- 1 csi csi 1523538404 2010-12-18 08:29 ta_6hrLev_HadGEM2- 
>> ES_rcp45_r1i1p1_200603010600-200606010000.nc
>> -rw-r--r-- 1 csi csi 1523538404 2010-12-18 08:29 ta_6hrLev_HadGEM2- 
>> ES_rcp45_r1i1p1_200606010600-200609010000.nc
>> -rw-r--r-- 1 csi csi 1523538404 2010-12-18 08:29 ta_6hrLev_HadGEM2- 
>> ES_rcp45_r1i1p1_200609010600-200612010000.nc
>> -rw-r--r-- 1 csi csi 1523538404 2010-12-18 08:30 ta_6hrLev_HadGEM2- 
>> ES_rcp45_r1i1p1_200612010600-200703010000.nc
>>
>>
>> The new .ctl file is
>>
>> DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_%y4%m2%d2%h200- 
>> %y4%m2%d2%h200.nc
>> TITLE Air Temperature
>> DTYPE netcdf
>> UNDEF 1.e+20f
>> OPTIONS template
>> xdef 192 linear 0 1.875
>> ydef 145 linear -90 1.25
>> zdef 38 levels 20.000337600708 80.001350402832 179.999114990234  
>> 320.00146484375 500.000579833984 720.000366210938 980.000854492188  
>> 1279.998046875 1619.99987792969 1999.99841308594 2420.00170898438  
>> 2880.00146484375 3379.99829101562 3919.99951171875 4500.00146484375  
>> 5120 5779.99951171875 6479.99951171875 7220 8000.00146484375 8820  
>> 9679.9990234375 10579.998046875 11519.998046875 12499.9990234375  
>> 13520.0009765625 14580.7998046875 15694.6396484375 16875.310546875  
>> 18138.626953125 19503.009765625 20990.1875 22626.08203125  
>> 24458.28515625 26583.640625 29219.080078125 32908.69140625  
>> 39254.83203125
>> tdef 1800 linear 6Z1dec2005 6hr
>> VARS 3
>> b 38 z vertical coordinate formula term (k)
>> orog 38 y,x Surface Altitude (m)
>> ta 38 t,z,y,x Air Temperature (K)
>> ENDVARS
>>
>>
>>
>>
>>
>> On Mar 11, 2011, at 4:03 AM, Jennifer Adams wrote:
>>
>>
>>
>> Joyce,
>> Please have a look at http://iges.org/grads/gadoc/SDFdescriptorfile.html
>> You need 'dtype netcdf' and you need something like "t,z,y,x"  
>> instead of 0 in your variable declarations. If it is still not  
>> working, please send the output from 'ncdump -h'.
>> --Jennifer
>>
>>
>>
>> On Mar 11, 2011, at 12:04 AM, Joyce Meyerson wrote:
>>
>>
>>
>> Thanks Jennifer. Looks like I'm getting close.
>> After I removed "time" in TDEF it complained and wanted xdef, ydef,  
>> etc. I input that info and can now "open" the file with no errors  
>> but cannot display anything. "Entire Grid Undefined".
>>
>>
>> Control file now looks like
>>
>>
>>
>> DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_%y4%m2%d2%h200- 
>> %y4%m2%d2%h200.nc
>> TITLE Air Temperature
>> UNDEF 1.e+20f
>> OPTIONS template
>> xdef 192 linear 0 1.875
>> ydef 145 linear -90 1.25
>> zdef 38 levels 20.000337600708 80.001350402832 179.999114990234  
>> 320.00146484375 500.000579833984 720.000366210938 980.000854492188  
>> 1279.998046875 1619.99987792969 1999.99841308594 2420.00170898438  
>> 2880.00146484375 3379.99829101562 3919.99951171875 4500.00146484375  
>> 5120 5779.99951171875 6479.99951171875 7220 8000.00146484375 8820  
>> 9679.9990234375 10579.998046875 11519.998046875 12499.9990234375  
>> 13520.0009765625 14580.7998046875 15694.6396484375 16875.310546875  
>> 18138.626953125 19503.009765625 20990.1875 22626.08203125  
>> 24458.28515625 26583.640625 29219.080078125 32908.69140625  
>> 39254.83203125
>> TDEF 1800 linear 6Z1dec2005 6hr
>> VARS 3
>> b 38 0 vertical coordinate formula term: b(k)
>> orog 38 0 Surface Altitude m
>> ta 38 0 Air Temperature K
>> ENDVARS
>>
>>
>> ???
>> Joyce
>>
>>
>>
>> On Mar 10, 2011, at 6:30 PM, Jennifer Adams wrote:
>>
>>
>>
>> Joyce,
>> Take the 'time' keyword out of your TDEF statement -- that is for  
>> 'xdfopen' style descriptors, and you are using 'open'.
>> --Jennifer
>>
>>
>>
>>
>>
>> On Mar 10, 2011, at 8:57 PM, Joyce Meyerson wrote:
>>
>>
>>
>> Thanks Jeff,
>>
>>
>> Unfortunately, when I tried
>>
>> DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_%y4%m2%d2%h200- 
>> %y4%m2%d2%h200.nc
>> OPTIONS template
>> TDEF time 1800 linear 6Z1dec2005 6hr
>>
>>
>> I get the message
>>
>> ga-> open test2.ctl
>> Scanning description file: test2.ctl
>> Open Error: Missing or invalid dimension size.
>> --> The invalid description file record is:
>> --> TDEF time 1800 linear 6Z1dec2005 6hr
>> The data file was not opened.
>>
>>
>> Joyce
>>
>>
>>
>> On Mar 10, 2011, at 5:19 PM, Jeffrey Duda wrote:
>>
>>
>> Joyce,
>> You have the general idea of templating down right, but you do have  
>> a few template strings that are not right. If the last four digits  
>> in the dates of your filenames are just four-digit UTC times, use  
>> %h200. If they're forecast hours, use %f30. Also, I think you  
>> should only have one file name listed in the DSET entry (i.e.,  
>> don't double up on the file names, just leave the templated file  
>> name string there instead).
>>
>> Unfortunately if that change doesn't work, that's as far as I can  
>> go to help you, as I don't use sdfopen and it could be an issue  
>> related to that.
>>
>> Hope this helps.
>>
>> Jeff Duda
>>
>>
>> On Thu, Mar 10, 2011 at 6:58 PM, Joyce Meyerson <  
>> hobo at atmos.ucla.edu > wrote:
>>
>>
>>
>> Hi All,
>>
>>
>> I am trying to write a control file to open multiple CMIP5 6hr .nc  
>> files.
>>
>>
>> I am running grads-2.0.a8 on Linux version 2.6.27.45-server-1mnb.
>>
>>
>> My files are 6h temperature in 3 month bundles and look like
>> ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_200512010600-200603010000.nc
>> ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_200603010600-200606010000.nc
>> ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_200606010600-200609010000.nc
>> ...
>>
>>
>> They open fine individually with sdfopen.
>> All the examples I've seen on the user list and elsewhere more or  
>> less have names that are simple sequential (e.g., pr.1998.nc , pr. 
>> 1999.nc , etc. or month, day, year, that kind of thing).
>>
>>
>> I'm not sure of the nomenclature with a time interval in the name  
>> like the above (xxx_%y4%m2%d2%t4-%y4%m2%d2% t4.nc )
>>
>>
>> I tried %ch
>>
>> DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_% ch.nc
>> CHSUB 1 360 200512010600-200603010000
>> CHSUB 361 720 200603010600-200606010000
>> OPTIONS template
>> DTYPE netcdf
>> TDEF time 720 linear 6Z1dec2005 6hr
>> with the error message:
>>
>> ga-> open test.ctl
>> Scanning description file: test.ctl
>> Open Error: Missing or invalid dimension size.
>> --> The invalid description file record is:
>> --> TDEF time 720 linear 6Z1dec2005 6hr
>> The data file was not opened.
>>
>>
>> I tried
>>
>> DSET ^ta_6hrLev_HadGEM2- 
>> ES_rcp45_r1i1p1_200512010600-200603010000.nc ta_6hrLev_HadGEM2- 
>> ES_rcp45_r1i1p1_%y4%m2%d2%t4-%y4%m2%d2% t4.nc
>> OPTIONS template
>> TDEF time 1800 linear 6Z1dec2005 6hr
>> [opens just the first file]
>>
>>
>> and I tried
>>
>> DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_%y4%m2%d2%t4-%y4%m2%d2% t4.nc
>> OPTIONS template
>> TDEF time 1800 linear 6Z1dec2005 6hr
>> [gives error message: read_metadata: gaopfn failed (rc=-88888)
>> gadsdf: Couldn't ingest SDF metadata.
>> SDF Descriptor file test2.ctl was not successfully opened & parsed.]
>>
>>
>> I'm not the most adept grads/computer/scripting person. Please help.
>>
>>
>> Joyce
>>
>>
>>
>>
>>
>>
>>
>>
>> **************************
>> Joyce Meyerson
>> UCLA
>> Dept. of Oceanic & Atmospheric Sciences
>> 7235 Math Science Bld.
>> Los Angeles, CA 90095-1565
>> hobo at atmos.ucla.edu
>>
>> _______________________________________________
>> gradsusr mailing list
>> gradsusr at gradsusr.org
>> http://gradsusr.org/mailman/listinfo/gradsusr
>> _______________________________________________
>> gradsusr mailing list
>> gradsusr at gradsusr.org
>> http://gradsusr.org/mailman/listinfo/gradsusr
>
> **************************
> Joyce Meyerson
> UCLA
> Dept. of Oceanic & Atmospheric Sciences
> 7235 Math Science Bld.
> Los Angeles, CA 90095-1565
> hobo at atmos.ucla.edu
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr

--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jma at cola.iges.org



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20110312/dc2ff152/attachment-0003.html 


More information about the gradsusr mailing list