[gradsusr] template for multiple netcdf files
Joyce Meyerson
hobo at atmos.ucla.edu
Sat Mar 12 09:18:27 EST 2011
That did it. Thank you again for all your help. : )
Joyce
On Mar 12, 2011, at 3:52 AM, Jennifer Adams wrote:
> You need to keep "OPTIONS template" in there when you have %ch in
> the DSET entry. --Jennifer
>
> On Mar 11, 2011, at 8:46 PM, Joyce Meyerson wrote:
>
>> Well, I'm certainly learning as I go along. Prior to this I've only
>> used .ctl files with binary data or simple time defs with .nc and
>> xdfopen.
>>
>> I put the lines for DSET and CHSUB and have tried a couple
>> variations including just DSET ^%ch and then listing the full path
>> all without success.
>> I double checked the filename just to be sure. I'm running v2.0.a8
>> which seems to be the second to last version.
>>
>> DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_%ch.nc
>> CHSUB 1 360 200603010600-200606010000
>> TITLE Air Temperature
>> DTYPE netcdf
>> UNDEF 1.e+20f
>> *OPTIONS template
>> xdef 192 linear 0 1.875
>> ydef 145 linear -90 1.25
>> zdef 38 levels 20.000337600708 80.001350402832 179.999114990234
>> 320.00146484375 500.000579833984 720.000366210938 980.000854492188
>> 1279.998046875 1619.99987792969 1999.99841308594 2420.00170898438
>> 2880.00146484375 3379.99829101562 3919.99951171875 4500.00146484375
>> 5120 5779.99951171875 6479.99951171875 7220 8000.00146484375 8820
>> 9679.9990234375 10579.998046875 11519.998046875 12499.9990234375
>> 13520.0009765625 14580.7998046875 15694.6396484375 16875.310546875
>> 18138.626953125 19503.009765625 20990.1875 22626.08203125
>> 24458.28515625 26583.640625 29219.080078125 32908.69140625
>> 39254.83203125
>> tdef 1800 linear 6Z1dec2005 6hr
>> VARS 3
>> b 38 z vertical coordinate formula term (k)
>> orog 0 y,x Surface Altitude (m)
>> ta 38 t,z,y,x Air Temperature (K)
>> ENDVARS
>>
>> ga-> open test3.ctl
>> Scanning description file: test3.ctl
>> Error: nc_open failed to open file ta_6hrLev_HadGEM2-
>> ES_rcp45_r1i1p1_%ch.nc
>> No such file or directory
>>
>> or with full path
>> ga-> open test3.ctl
>> Scanning description file: test3.ctl
>> Error: nc_open failed to open file %ch
>>
>>
>> On Mar 11, 2011, at 4:39 PM, Jennifer Adams wrote:
>>
>>> Hang in there, Joyce. You're paving the way for all the future
>>> CMIP5 users out there. Unfortunately, you are going to have to use
>>> chsub for the template string substitutions. The file naming
>>> convention, although handy because you can know exactly which time
>>> steps are contained in each file, is not convenient for GrADS, and
>>> the only way you can do the templating is to give GrADS explicit
>>> instructions as to which time steps belong in each file (that's
>>> what CHSUB is good for). This was true for some CMIP3 data as
>>> well...
>>> DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_%ch.nc
>>> CHSUB 1 360 200512010600-200603010000
>>> etc.
>>> (I think you have already worked out how chsub entries should look)
>>>
>>> Another thing I noticed about your descriptor file -- your
>>> variable "orog" does not vary in Z, so you should change the 38 to
>>> 0 in the variable declaration.
>>> --Jennifer
>>>
>>>
>>> ----- Original Message -----
>>> From: "Joyce Meyerson" <hobo at atmos.ucla.edu>
>>> To: "GrADS Users Forum" <gradsusr at gradsusr.org>
>>> Sent: Friday, March 11, 2011 7:15:43 PM
>>> Subject: Re: [gradsusr] template for multiple netcdf files
>>>
>>>
>>> Hi Jennifer,
>>>
>>>
>>> Thank you for your patient help.
>>> I read the SDFdesciptorfile and saw what you were talking about
>>> wrt the variable declarations and dtype which I implemented.
>>> When I used only a single file it opened fine so I'm assuming the
>>> problem now lies with the template time substitution.
>>>
>>>
>>> I'm attaching the ncdump if that is helpful.
>>>
>>>
>>> Thanks again.
>>> Joyce
>>>
>>>
>>>
>>>
>>>
>>>
>>> A subset of the files to be opened (3 month chunks of 6hr data)
>>>
>>> -rw-r--r-- 1 csi csi 1523538404 2010-12-18 08:28 ta_6hrLev_HadGEM2-
>>> ES_rcp45_r1i1p1_200512010600-200603010000.nc
>>> -rw-r--r-- 1 csi csi 1523538404 2010-12-18 08:29 ta_6hrLev_HadGEM2-
>>> ES_rcp45_r1i1p1_200603010600-200606010000.nc
>>> -rw-r--r-- 1 csi csi 1523538404 2010-12-18 08:29 ta_6hrLev_HadGEM2-
>>> ES_rcp45_r1i1p1_200606010600-200609010000.nc
>>> -rw-r--r-- 1 csi csi 1523538404 2010-12-18 08:29 ta_6hrLev_HadGEM2-
>>> ES_rcp45_r1i1p1_200609010600-200612010000.nc
>>> -rw-r--r-- 1 csi csi 1523538404 2010-12-18 08:30 ta_6hrLev_HadGEM2-
>>> ES_rcp45_r1i1p1_200612010600-200703010000.nc
>>>
>>>
>>> The new .ctl file is
>>>
>>> DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_%y4%m2%d2%h200-
>>> %y4%m2%d2%h200.nc
>>> TITLE Air Temperature
>>> DTYPE netcdf
>>> UNDEF 1.e+20f
>>> OPTIONS template
>>> xdef 192 linear 0 1.875
>>> ydef 145 linear -90 1.25
>>> zdef 38 levels 20.000337600708 80.001350402832 179.999114990234
>>> 320.00146484375 500.000579833984 720.000366210938 980.000854492188
>>> 1279.998046875 1619.99987792969 1999.99841308594 2420.00170898438
>>> 2880.00146484375 3379.99829101562 3919.99951171875
>>> 4500.00146484375 5120 5779.99951171875 6479.99951171875 7220
>>> 8000.00146484375 8820 9679.9990234375 10579.998046875
>>> 11519.998046875 12499.9990234375 13520.0009765625 14580.7998046875
>>> 15694.6396484375 16875.310546875 18138.626953125 19503.009765625
>>> 20990.1875 22626.08203125 24458.28515625 26583.640625
>>> 29219.080078125 32908.69140625 39254.83203125
>>> tdef 1800 linear 6Z1dec2005 6hr
>>> VARS 3
>>> b 38 z vertical coordinate formula term (k)
>>> orog 38 y,x Surface Altitude (m)
>>> ta 38 t,z,y,x Air Temperature (K)
>>> ENDVARS
>>>
>>>
>>>
>>>
>>>
>>> On Mar 11, 2011, at 4:03 AM, Jennifer Adams wrote:
>>>
>>>
>>>
>>> Joyce,
>>> Please have a look at http://iges.org/grads/gadoc/SDFdescriptorfile.html
>>> You need 'dtype netcdf' and you need something like "t,z,y,x"
>>> instead of 0 in your variable declarations. If it is still not
>>> working, please send the output from 'ncdump -h'.
>>> --Jennifer
>>>
>>>
>>>
>>> On Mar 11, 2011, at 12:04 AM, Joyce Meyerson wrote:
>>>
>>>
>>>
>>> Thanks Jennifer. Looks like I'm getting close.
>>> After I removed "time" in TDEF it complained and wanted xdef,
>>> ydef, etc. I input that info and can now "open" the file with no
>>> errors but cannot display anything. "Entire Grid Undefined".
>>>
>>>
>>> Control file now looks like
>>>
>>>
>>>
>>> DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_%y4%m2%d2%h200-
>>> %y4%m2%d2%h200.nc
>>> TITLE Air Temperature
>>> UNDEF 1.e+20f
>>> OPTIONS template
>>> xdef 192 linear 0 1.875
>>> ydef 145 linear -90 1.25
>>> zdef 38 levels 20.000337600708 80.001350402832 179.999114990234
>>> 320.00146484375 500.000579833984 720.000366210938 980.000854492188
>>> 1279.998046875 1619.99987792969 1999.99841308594 2420.00170898438
>>> 2880.00146484375 3379.99829101562 3919.99951171875
>>> 4500.00146484375 5120 5779.99951171875 6479.99951171875 7220
>>> 8000.00146484375 8820 9679.9990234375 10579.998046875
>>> 11519.998046875 12499.9990234375 13520.0009765625 14580.7998046875
>>> 15694.6396484375 16875.310546875 18138.626953125 19503.009765625
>>> 20990.1875 22626.08203125 24458.28515625 26583.640625
>>> 29219.080078125 32908.69140625 39254.83203125
>>> TDEF 1800 linear 6Z1dec2005 6hr
>>> VARS 3
>>> b 38 0 vertical coordinate formula term: b(k)
>>> orog 38 0 Surface Altitude m
>>> ta 38 0 Air Temperature K
>>> ENDVARS
>>>
>>>
>>> ???
>>> Joyce
>>>
>>>
>>>
>>> On Mar 10, 2011, at 6:30 PM, Jennifer Adams wrote:
>>>
>>>
>>>
>>> Joyce,
>>> Take the 'time' keyword out of your TDEF statement -- that is for
>>> 'xdfopen' style descriptors, and you are using 'open'.
>>> --Jennifer
>>>
>>>
>>>
>>>
>>>
>>> On Mar 10, 2011, at 8:57 PM, Joyce Meyerson wrote:
>>>
>>>
>>>
>>> Thanks Jeff,
>>>
>>>
>>> Unfortunately, when I tried
>>>
>>> DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_%y4%m2%d2%h200-
>>> %y4%m2%d2%h200.nc
>>> OPTIONS template
>>> TDEF time 1800 linear 6Z1dec2005 6hr
>>>
>>>
>>> I get the message
>>>
>>> ga-> open test2.ctl
>>> Scanning description file: test2.ctl
>>> Open Error: Missing or invalid dimension size.
>>> --> The invalid description file record is:
>>> --> TDEF time 1800 linear 6Z1dec2005 6hr
>>> The data file was not opened.
>>>
>>>
>>> Joyce
>>>
>>>
>>>
>>> On Mar 10, 2011, at 5:19 PM, Jeffrey Duda wrote:
>>>
>>>
>>> Joyce,
>>> You have the general idea of templating down right, but you do
>>> have a few template strings that are not right. If the last four
>>> digits in the dates of your filenames are just four-digit UTC
>>> times, use %h200. If they're forecast hours, use %f30. Also, I
>>> think you should only have one file name listed in the DSET entry
>>> (i.e., don't double up on the file names, just leave the templated
>>> file name string there instead).
>>>
>>> Unfortunately if that change doesn't work, that's as far as I can
>>> go to help you, as I don't use sdfopen and it could be an issue
>>> related to that.
>>>
>>> Hope this helps.
>>>
>>> Jeff Duda
>>>
>>>
>>> On Thu, Mar 10, 2011 at 6:58 PM, Joyce Meyerson < hobo at atmos.ucla.edu
>>> > wrote:
>>>
>>>
>>>
>>> Hi All,
>>>
>>>
>>> I am trying to write a control file to open multiple CMIP5 6hr .nc
>>> files.
>>>
>>>
>>> I am running grads-2.0.a8 on Linux version 2.6.27.45-server-1mnb.
>>>
>>>
>>> My files are 6h temperature in 3 month bundles and look like
>>> ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_200512010600-200603010000.nc
>>> ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_200603010600-200606010000.nc
>>> ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_200606010600-200609010000.nc
>>> ...
>>>
>>>
>>> They open fine individually with sdfopen.
>>> All the examples I've seen on the user list and elsewhere more or
>>> less have names that are simple sequential (e.g., pr.1998.nc , pr.
>>> 1999.nc , etc. or month, day, year, that kind of thing).
>>>
>>>
>>> I'm not sure of the nomenclature with a time interval in the name
>>> like the above (xxx_%y4%m2%d2%t4-%y4%m2%d2% t4.nc )
>>>
>>>
>>> I tried %ch
>>>
>>> DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_% ch.nc
>>> CHSUB 1 360 200512010600-200603010000
>>> CHSUB 361 720 200603010600-200606010000
>>> OPTIONS template
>>> DTYPE netcdf
>>> TDEF time 720 linear 6Z1dec2005 6hr
>>> with the error message:
>>>
>>> ga-> open test.ctl
>>> Scanning description file: test.ctl
>>> Open Error: Missing or invalid dimension size.
>>> --> The invalid description file record is:
>>> --> TDEF time 720 linear 6Z1dec2005 6hr
>>> The data file was not opened.
>>>
>>>
>>> I tried
>>>
>>> DSET ^ta_6hrLev_HadGEM2-
>>> ES_rcp45_r1i1p1_200512010600-200603010000.nc ta_6hrLev_HadGEM2-
>>> ES_rcp45_r1i1p1_%y4%m2%d2%t4-%y4%m2%d2% t4.nc
>>> OPTIONS template
>>> TDEF time 1800 linear 6Z1dec2005 6hr
>>> [opens just the first file]
>>>
>>>
>>> and I tried
>>>
>>> DSET ^ta_6hrLev_HadGEM2-ES_rcp45_r1i1p1_%y4%m2%d2%t4-%y4%m2%d2%
>>> t4.nc
>>> OPTIONS template
>>> TDEF time 1800 linear 6Z1dec2005 6hr
>>> [gives error message: read_metadata: gaopfn failed (rc=-88888)
>>> gadsdf: Couldn't ingest SDF metadata.
>>> SDF Descriptor file test2.ctl was not successfully opened & parsed.]
>>>
>>>
>>> I'm not the most adept grads/computer/scripting person. Please help.
>>>
>>>
>>> Joyce
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> **************************
>>> Joyce Meyerson
>>> UCLA
>>> Dept. of Oceanic & Atmospheric Sciences
>>> 7235 Math Science Bld.
>>> Los Angeles, CA 90095-1565
>>> hobo at atmos.ucla.edu
>>>
>>> _______________________________________________
>>> gradsusr mailing list
>>> gradsusr at gradsusr.org
>>> http://gradsusr.org/mailman/listinfo/gradsusr
>>> _______________________________________________
>>> gradsusr mailing list
>>> gradsusr at gradsusr.org
>>> http://gradsusr.org/mailman/listinfo/gradsusr
>>
>> **************************
>> Joyce Meyerson
>> UCLA
>> Dept. of Oceanic & Atmospheric Sciences
>> 7235 Math Science Bld.
>> Los Angeles, CA 90095-1565
>> hobo at atmos.ucla.edu
>>
>> _______________________________________________
>> gradsusr mailing list
>> gradsusr at gradsusr.org
>> http://gradsusr.org/mailman/listinfo/gradsusr
>
> --
> Jennifer M. Adams
> IGES/COLA
> 4041 Powder Mill Road, Suite 302
> Calverton, MD 20705
> jma at cola.iges.org
>
>
>
> <ATT00001..txt>
**************************
Joyce Meyerson
UCLA
Dept. of Oceanic & Atmospheric Sciences
7235 Math Science Bld.
Los Angeles, CA 90095-1565
hobo at atmos.ucla.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20110312/29bf9145/attachment-0003.html
More information about the gradsusr
mailing list