[gradsusr] Problem with Ensemble Files

Mon Jan 20 10:00:07 EST 2014

Tom,
I have a couple of suggestions:

1. You have set your TDEF to cover the entire range of times for ensemble e01, but not for the entire span of all the other members. If you were to expand your tdef so that it started at 18z28jan2014 (the start of e20) and ended with the final time step of e01, then you would not see the "valid time outside of file limits" messages. However, if it is your intention to only look at data from the older runs (members e02 - e20) that overlap with the most recent run, then you should change the length of those members from 149 to match the actual number of time steps that fit inside the range of e01. Expanding your TDEF to include the whole length of all members will clean up the gribmap output to make it easier to see if there are other problems. 

2. You have a fixed date string in your file name "/home/mdladmin/CFS/Input/cfspgb.01.20140119%e.grb2" . Are you renaming the older forecast files to match this? Make sure that your naming convention matches your DSET entry. 

3. I can't verify this at the moment, but I have noticed that the CFS real time data files are not the same when they are brand new compared to when they are a couple of days old. I think they add some additional time steps after some time has elapsed since the forecast was initially posted. In my own operational scripts to grab this data I wait for 48 hours before getting the data -- for our purposes, looking at data in real time is not strictly necessary, so it's not big deal to wait. But see if the number of records in each file in your ensemble set really covers the nominal 149 time steps. A careful look at the wgrib2 output should help you see if this may be an issue in your case. 

--Jennifer

On Jan 19, 2014, at 10:32 PM, Tom Hultquist wrote:

> Jennifer,
> 
> No problem, I just didn't want to provide too many details initially. The GRIB files are from the CFS off of NOMADS. I'm grabbing 6-hourly files from the 4x per day runs, and just grabbing member 1 (for a certain window of time, whereby the previous files over the past five days will overlap the same time period for which I'm computing things). The files are at http://nomads.ncep.noaa.gov/pub/data/nccf/com/cfs/prod/cfs/cfs.20140117/00/6hrly_grib_01/ with YYYYMMDD/CC changing depending on date/cycle downloaded.
> 
> The ctl file for the most recent "run" is attached. It works ok except for the issue I described (member #3 always causes the issue). GrADS is version 2.1.a1, although I also had the issue with 2.0.2. As best I can tell g2ctl.pl is 0.0.4o, so perhaps that is an issue (since I do see a more recent version, 0.0.8.8 online). I suppose I'll give that a shot in the meantime.
> 
> tom
> 
> 
> On Sun, Jan 19, 2014 at 8:39 PM, Jennifer Adams <jma at cola.iges.org> wrote:
> Tom,
> Can you please provide the source of your grib files, and the descriptor file that doesn't work? I can't resolve a problem that can't be reproduced. It would also be useful to know the version of GrADS and g2ctl you're using.
> --Jennifer
> 
> On Jan 19, 2014, at 3:12 PM, Tom Hultquist wrote:
> 
>> I have a set of files with different initial times, whose forecast times overlap so that I can use them as an ensemble dataset. I am using g2ctl.pl with the -ens option, and it is creating a proper ctl file. I thought everything was working fine, but then noticed some oddities in a few calculations I was doing. So, I decided to view data from each member interactively in GrADS, using the ctl file which was created by g2ctl.pl. For some reason, it would not display anything from the third ensemble member (entire grid undefined). There was, however, nothing wrong with the file, since it worked fine if viewed individually outside of the ensemble set. In addition, when a new ctl is created later, and that member 3 became member 4, it no longer caused a problem. However, the "new" member 3 (which was previously member 2) had the same issue. When looking at the gribmap feedback (with -v turned on), I can see it runs into issues with valid times being outside of file limits (which indeed is true, since the files overlap, but don't cover the exact same time frame as member 1). When gribmap has this error, it shows it= (e.g. it=10) vs t= for the valid time conflict. I couldn't find any info on what it= references. The same error happens with subsequent files (members 4-20), but gribmap appears to scan those files twice and they display properly for me.
>> 
>> Has anyone experienced anything like this, and/or have any suggestions?
>> _______________________________________________
>> gradsusr mailing list
>> gradsusr at gradsusr.org
>> http://gradsusr.org/mailman/listinfo/gradsusr
> 
> --
> Jennifer M. Adams
> Center for Ocean-Land-Atmosphere Studies (COLA)
> 111 Research Hall, Mail Stop 2B3
> George Mason University
> 4400 University Drive
> Fairfax, VA 22030 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
> 
> 
> <CFS_ensemble1.ctl>_______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr

--
Jennifer M. Adams
Center for Ocean-Land-Atmosphere Studies (COLA)
111 Research Hall, Mail Stop 2B3
George Mason University
4400 University Drive
Fairfax, VA 22030 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20140120/7c9e6e27/attachment.html 