[gradsusr] Problem with Ensemble Files

Mon Jan 20 12:21:13 EST 2014

Jennifer,

Thanks for the reply. With regard to what you mentioned...

1) The TDEF was defined by g2ctl, and seems to match how its documentation
says it would do things for a time-lagged ensemble (whereby it is set to
the 1st/most recent member). I can certainly try adjusting it though, to
see if it helps. It is my intention, however, to only do calculations on
the overlapping time period, essentially from the beginning time of member
1 (most recent) out 4 weeks. The previous runs (members 2-20) all overlap
that time time period, but contain earlier data, since at one time each of
them was member #1 (every six hours that changes). I'm not re-downloading
the files, but am simply using the "realtime" files that were downloaded
earlier. I'll play around, and see if I see any more meaningful info from
gribmap.

2) Yes, I copy/rename the files each run so that the templating is easier,
just the %e is different between each member.

3) I'm not sure of this either, but I am only downloading the realtime
output (basically 12 hours after run time). So, my older members are not
re-downloaded, but are simply the files I previously downloaded.

It is just odd that everything works fine, except for what winds up as
member 3 each time, whether that is from 00, 06, 12, or 18Z run. And, when
that member becomes member 2, it's fine as well.

tom

On Mon, Jan 20, 2014 at 9:00 AM, Jennifer Adams <jma at cola.iges.org> wrote:

> Tom,
> I have a couple of suggestions:
>
> 1. You have set your TDEF to cover the entire range of times for ensemble
> e01, but not for the entire span of all the other members. If you were to
> expand your tdef so that it started at 18z28jan2014 (the start of e20) and
> ended with the final time step of e01, then you would not see the "valid
> time outside of file limits" messages. However, if it is your intention to
> only look at data from the older runs (members e02 - e20) that overlap with
> the most recent run, then you should change the length of those members
> from 149 to match the actual number of time steps that fit inside the range
> of e01. Expanding your TDEF to include the whole length of all members will
> clean up the gribmap output to make it easier to see if there are other
> problems.
>
> 2. You have a fixed date string in your file name
> "/home/mdladmin/CFS/Input/cfspgb.01.20140119%e.grb2" . Are you renaming the
> older forecast files to match this? Make sure that your naming convention
> matches your DSET entry.
>
> 3. I can't verify this at the moment, but I have noticed that the CFS real
> time data files are not the same when they are brand new compared to when
> they are a couple of days old. I think they add some additional time steps
> after some time has elapsed since the forecast was initially posted. In my
> own operational scripts to grab this data I wait for 48 hours before
> getting the data -- for our purposes, looking at data in real time is not
> strictly necessary, so it's not big deal to wait. But see if the number of
> records in each file in your ensemble set really covers the nominal 149
> time steps. A careful look at the wgrib2 output should help you see if this
> may be an issue in your case.
>
> --Jennifer
>
> On Jan 19, 2014, at 10:32 PM, Tom Hultquist wrote:
>
> Jennifer,
>
> No problem, I just didn't want to provide too many details initially. The
> GRIB files are from the CFS off of NOMADS. I'm grabbing 6-hourly files from
> the 4x per day runs, and just grabbing member 1 (for a certain window of
> time, whereby the previous files over the past five days will overlap the
> same time period for which I'm computing things). The files are at
> http://nomads.ncep.noaa.gov/pub/data/nccf/com/cfs/prod/cfs/cfs.20140117/00/6hrly_grib_01/with YYYYMMDD/CC changing depending on date/cycle downloaded.
>
> The ctl file for the most recent "run" is attached. It works ok except for
> the issue I described (member #3 always causes the issue). GrADS is version
> 2.1.a1, although I also had the issue with 2.0.2. As best I can tell
> g2ctl.pl is 0.0.4o, so perhaps that is an issue (since I do see a more
> recent version, 0.0.8.8 online). I suppose I'll give that a shot in the
> meantime.
>
> tom
>
>
> On Sun, Jan 19, 2014 at 8:39 PM, Jennifer Adams <jma at cola.iges.org> wrote:
>
>> Tom,
>> Can you please provide the source of your grib files, and the descriptor
>> file that doesn't work? I can't resolve a problem that can't be reproduced.
>> It would also be useful to know the version of GrADS and g2ctl you're using.
>> --Jennifer
>>
>> On Jan 19, 2014, at 3:12 PM, Tom Hultquist wrote:
>>
>> I have a set of files with different initial times, whose forecast times
>> overlap so that I can use them as an ensemble dataset. I am using
>> g2ctl.pl with the -ens option, and it is creating a proper ctl file. I
>> thought everything was working fine, but then noticed some oddities in a
>> few calculations I was doing. So, I decided to view data from each member
>> interactively in GrADS, using the ctl file which was created by g2ctl.pl.
>> For some reason, it would not display anything from the third ensemble
>> member (entire grid undefined). There was, however, nothing wrong with the
>> file, since it worked fine if viewed individually outside of the ensemble
>> set. In addition, when a new ctl is created later, and that member 3 became
>> member 4, it no longer caused a problem. However, the "new" member 3 (which
>> was previously member 2) had the same issue. When looking at the gribmap
>> feedback (with -v turned on), I can see it runs into issues with valid
>> times being outside of file limits (which indeed is true, since the files
>> overlap, but don't cover the exact same time frame as member 1). When
>> gribmap has this error, it shows it= (e.g. it=10) vs t= for the valid time
>> conflict. I couldn't find any info on what it= references. The same error
>> happens with subsequent files (members 4-20), but gribmap appears to scan
>> those files twice and they display properly for me.
>>
>> Has anyone experienced anything like this, and/or have any suggestions?
>> _______________________________________________
>> gradsusr mailing list
>> gradsusr at gradsusr.org
>> http://gradsusr.org/mailman/listinfo/gradsusr
>>
>>
>>  --
>> Jennifer M. Adams
>> Center for Ocean-Land-Atmosphere Studies (COLA)
>> 111 Research Hall, Mail Stop 2B3
>> George Mason University
>> 4400 University Drive
>> Fairfax, VA 22030
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> gradsusr mailing list
>> gradsusr at gradsusr.org
>> http://gradsusr.org/mailman/listinfo/gradsusr
>>
>>
> <CFS_ensemble1.ctl>_______________________________________________
>
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
>
>
>  --
> Jennifer M. Adams
> Center for Ocean-Land-Atmosphere Studies (COLA)
> 111 Research Hall, Mail Stop 2B3
> George Mason University
> 4400 University Drive
> Fairfax, VA 22030
>
>
>
>
>
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20140120/8265496a/attachment.html 