[gradsusr] GFS total cloud cover
Sam Wilson
sam at surfline.com
Fri Nov 4 13:34:52 EDT 2016
Wesley,
Got it and thanks all for the helpful responses!
I’m going with this method and it appears to work fine +- the errors, which I can deal with!
Thanks for the feedback on wgrib2 as well.
Cheers,
Sam
From: <gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org>> on behalf of Wesley Ebisuzaki - NOAA Federal <wesley.ebisuzaki at noaa.gov<mailto:wesley.ebisuzaki at noaa.gov>>
Reply-To: GrADS Users Forum <gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>>
Date: Friday, November 4, 2016 at 10:08 AM
To: GrADS Users Forum <gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>>
Subject: Re: [gradsusr] GFS total cloud cover
Sam,
TCDC is stored to the nearest percentage in grib2
for hour 1, the mean is C1,
for hour 2, the mean is 2C2-C1
for hour 3, the mean is 3C3-2C2
for hour 4, the mean is 4C4-3C2
for hour 5, the mean is 5C5-4C4
and for hour 6, it is 6C6-5C5
So C1, C2, .., C6 could have an error of +/- 0.5 when stored in grib2.
Therefore the error for hour 6 is +/-6*0.5 + +/-5*0.5 or +/- 5.5%.
By the ways, wgrib2 v0.1.9.7a is 4 year old and some repositories
of precompiled codes had problems with segfaults unless you
turned off g2clib. There are two ways to compile g2clib and
they were not compatible.
Wesley
On Fri, Nov 4, 2016 at 1:27 AM, Sam Wilson <sam at surfline.com<mailto:sam at surfline.com>> wrote:
Hi Stephen,
Thanks for the feedback!
I did try the method Moorthi pointed out initially but was getting values below 0 and above 100 occasionally, so began pursuing Wesley’s wgrib2 route.
Is getting occasional values below 0 and above 100 to be expected with Moorthi's method? Nothing wildly below 0 or above 100 (just a few percent)..but want to make sure I’m doing things correctly..
I did make some graphics using Moorthi’s method and they appear to be correct - so I may just set anything below 0 to 0 and above 100 to 100 and call it a day..
Thanks
Sam
From: <gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org>> on behalf of Stephen McMillan <smcmillan at planalytics.com<mailto:smcmillan at planalytics.com>>
Reply-To: GrADS Users Forum <gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>>
Date: Thursday, November 3, 2016 at 8:08 PM
To: GrADS Users Forum <gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>>
Subject: Re: [gradsusr] GFS total cloud cover
Sam,
I think we're making this too complicated. I agree with Moorthi's solution provided earlier. Here it is again for convenience:
Let us denote the data from 6 hourly output as C1, C2, C3, C4, C5, C6.
To get hourly mean, then
for hour 1, the mean is C1,
for hour 2, the mean is 2C2-C1
for hour 3, the mean is 3C3-2C2
for hour 4, the mean is 4C4-3C2
for hour 5, the mean is 5C5-4C4
and for hour 6, it is 6C6-5C5
This pattern repeats every six ours.
If you haven't already tried what Moorthi suggested, please try it...it works!
Stephen McMillan
On Thu, Nov 3, 2016 at 5:59 PM, Sam Wilson <sam at surfline.com<mailto:sam at surfline.com>> wrote:
Wesley,
I got it to work by passing in multiple grib files in sequence! As you suggested ;)
Sorry for the confusion and thanks again.
Sam
From: <gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org>> on behalf of Sam Wilson <sam at surfline.com<mailto:sam at surfline.com>>
Reply-To: GrADS Users Forum <gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>>
Date: Thursday, November 3, 2016 at 11:59 AM
To: GrADS Users Forum <gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>>
Subject: Re: [gradsusr] GFS total cloud cover
Wesley,
I plotted the output and unfortunately I’m getting the same results. Not sure if I need to upgrade wgrib2 on my machine or not for it to work.
Sam
From: <gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org>> on behalf of Sam Wilson <sam at surfline.com<mailto:sam at surfline.com>>
Reply-To: GrADS Users Forum <gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>>
Date: Thursday, November 3, 2016 at 11:21 AM
To: GrADS Users Forum <gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>>
Subject: Re: [gradsusr] GFS total cloud cover
Wesley,
Here’s the output of wgrib2 -config:
wgrib2 v0.1.9.7a 11/2012 Wesley Ebisuzaki, Reinoud Bokhorst, Jaakko Hyvätti, Dusan Jovic, Kristian Nilssen, Karl Pfeiffer, Pablo Romero, Manfred Schwarb, Arlindo da Silva, Niklas Sondell, Sergey Varlamov
Compiled on 21:56:36 Jan 10 2013
Netcdf package: 4.1.1 of Jan 30 2014 05:31:22 $ is installed
mysql package is installed
regex package is installed
tigge package is installed
interpolation package is not installed
gptpc interface: experimental v0.1
UDF package is not installed
maximum number of arguments on command line: 5000
maximum number of -match,-not,-if, and -not_if arguments: 1000
stdout buffer length: 30000
g2clib is the default decoder
OpenMP: yes
INT_MAX: 2147483647<tel:2147483647>
ULONG_MAX: 18446744073709551615
I ran it using the -g2clib 0 option and it appears to work now.
However, when running with the command you provided, the junk file created still shows the following (an example using the 3 hour forecast data file):
$ cat gfs0p252016110312hr003.grb2 | wgrib2 -g2clib 0 - -match "(:TCDC:entire atmosphere:)" -set_grib_type c3 -if ":TCDC:entire atmosphere:" -ncep_norm junk
$ wgrib2 junk
1:0:d=2016110312<tel:2016110312>:TCDC:entire atmosphere:0-3 hour ave fcst:
Not sure if that’s just the wgrib2 table saying that, when really it’s the 2-3 hour forecast. I was going to plot the results to find out but let me know if you have any additional input.
Thanks!
Sam
From: <gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org>> on behalf of Wesley Ebisuzaki - NOAA Federal <wesley.ebisuzaki at noaa.gov<mailto:wesley.ebisuzaki at noaa.gov>>
Reply-To: GrADS Users Forum <gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>>
Date: Thursday, November 3, 2016 at 11:08 AM
To: GrADS Users Forum <gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>>
Subject: Re: [gradsusr] GFS total cloud cover
Sam,
Which version of wgrib2? (wgrib2 -config)
Try increasing the stack size. (bash: ulimit)
Wesley
On Thu, Nov 3, 2016 at 1:32 PM, Sam Wilson <sam at surfline.com<mailto:sam at surfline.com>> wrote:
Moorthi and Wesley,
Thanks very much for the feedback - I attempted using the ncep_norm option with wgrib2, but got a segmentation fault..argh!
My software installation skills are sub-par, so I will try Moorthi’s option and get back to the group.
Thanks again..
Cheers,
Sam
From: <gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org>> on behalf of Wesley Ebisuzaki - NOAA Federal <wesley.ebisuzaki at noaa.gov<mailto:wesley.ebisuzaki at noaa.gov>>
Reply-To: GrADS Users Forum <gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>>
Date: Thursday, November 3, 2016 at 5:59 AM
To: GrADS Users Forum <gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>>
Subject: Re: [gradsusr] GFS total cloud cover
Sam,
You can get the "0-1 hour ave fcst", "1-2 hour ave fcst", etc
by using the -ncep_norm option of wgrib2.
http://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/ncep_norm.html
Here is a simple script that takes a list of gfs forecast files in order
#!/bin/sh
#
# usage $0 (list of gfs grib2 forecast files in order)
#
A=":TCDC:entire atmosphere:"
B=":TCDC:low cloud layer:"
C=":TCDC:middle cloud layer:"
D=":TCDC:high cloud layer:"
cat $* | wgrib2 - -match "($A|$B|$C|$D)" -set_grib_type c3 \
-if "$A" -ncep_norm $stmp/junk \
-if "$B" -ncep_norm $stmp/junk \
-if "$C" -ncep_norm $stmp/junk \
-if "$D" -ncep_norm $stmp/junk
Wesley
On Thu, Nov 3, 2016 at 6:59 AM, Shrinivas Moorthi <shrinivas.moorthi at noaa.gov<mailto:shrinivas.moorthi at noaa.gov>> wrote:
The grib header is correct.
Let us denote the data from 6 hourly output as C1, C2, C3, C4, C5, C6.
To get hourly mean, then
for hour 1, the mean is C1,
for hour 2, the mean is 2C2-C1
for hour 3, the mean is 3C3-2C2
for hour 4, the mean is 4C4-3C2
for hour 5, the mean is 5C5-4C4
and for hour 6, it is 6C6-5C5
This pattern repeats every six ours.
I hope this helps.
Moorthi
On 11/02/2016 07:41 PM, Sam Wilson wrote:
Hi Jeff,
I plotted a quick and dirty cloud animation using the data - each step is 1 hour from f001 through f015 in the attached.
It appears that they are averages over time - you can see the bigger jumps at steps 8 and 14 (the transition from 0-6hr avg to 6-7hr and from 6-12hr avg to 12-13hr).
Marian - thank you for the feedback. Sounds like using the hourly data for cloud cover may not be a good idea, unless someone else has a solution.
Thanks,
Sam
From: <gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org>> on behalf of Jeff Duda <jeffduda319 at gmail.com<mailto:jeffduda319 at gmail.com>>
Reply-To: GrADS Users Forum <gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>>
Date: Wednesday, November 2, 2016 at 3:21 PM
To: GrADS Users Forum <gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>>
Subject: Re: [gradsusr] GFS total cloud cover
Sam,
That might just be a wgrib table thing. Have you actually plotted the field over those times? If it's truly an averaged field, you should be able to see some indication of that in the texture and smoothness of the fields as you progress through increasing averaged times.
My guess is that this is just some arbitrary setting in wgrib and does not reflect the true nature of the field. But I could be wrong. If there are any NCEP folks who work on the GFS on this forum, perhaps one of them could set the record straight.
Jeff
On Wed, Nov 2, 2016 at 4:54 PM, Sam Wilson <sam at surfline.com<mailto:sam at surfline.com>> wrote:
Thanks for the response Jeff - it’s strange to me as well!
Running wgrib2 on the data files for forecast hours 001 through 007 gives the following for TCDCclm:
f001:
17:7991683:d=2016110212<tel:2016110212>:TCDC:entire atmosphere:0-1 hour ave fcst:
f002:
17:8082626:d=2016110212<tel:2016110212>:TCDC:entire atmosphere:0-2 hour ave fcst:
f003:
17:8059826:d=2016110212<tel:2016110212>:TCDC:entire atmosphere:0-3 hour ave fcst:
f004:
17:8057865:d=2016110212<tel:2016110212>:TCDC:entire atmosphere:0-4 hour ave fcst:
f005:
17:8119289:d=2016110212<tel:2016110212>:TCDC:entire atmosphere:0-5 hour ave fcst:
f006:
17:8172510:d=2016110212<tel:2016110212>:TCDC:entire atmosphere:0-6 hour ave fcst:
f007:
17:8013825:d=2016110212<tel:2016110212>:TCDC:entire atmosphere:6-7 hour ave fcst:
And it continues on..once the files go to 3 hourly (after hour 120), we have:
f123:
17:7687687:d=2016110212<tel:2016110212>:TCDC:entire atmosphere:120-123 hour ave fcst:
f126:
17:7778871:d=2016110212<tel:2016110212>:TCDC:entire atmosphere:120-126 hour ave fcst:
f129:
17:7728042:d=2016110212<tel:2016110212>:TCDC:entire atmosphere:126-129 hour ave fcst:
And finally the 12 hourly files (past hour 240) are all just 12 hour average forecasts:
f252:
17:6845416:d=2016110212<tel:2016110212>:TCDC:entire atmosphere:240-252 hour ave fcst:
I did think about the method you suggested (tcdcclm - tcdcclm(t-1)) but stopped there for the reason you mentioned..
Strange indeed - unless I’m missing something..
Thanks,
Sam
From: <gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org>> on behalf of Jeff Duda <jeffduda319 at gmail.com<mailto:jeffduda319 at gmail.com>>
Reply-To: GrADS Users Forum <gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>>
Date: Wednesday, November 2, 2016 at 2:08 PM
To: GrADS Users Forum <gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>>
Subject: Re: [gradsusr] GFS total cloud cover
It seems a little strange to me that the total cloud cover product would be averaged over time. Is that really the case?
Anyway, if you really do have an average over overlapping and increasing windows, with nothing else to go on and without applying the equations of motion in reverse, you'd have to assume a linear averaging, so subtraction of subsequent slices of the field (i.e., tcdcclm - tcdcclm(t-1)) would give you piecewise temporal averages. However, that method would give you negative cloud cover values which is clearly nonsensical, which again is why I question whether or not that array contains a temporal average. That just doesn't make a lot of sense.
Jeff Duda
On Wed, Nov 2, 2016 at 3:27 PM, Sam Wilson <sam at surfline.com<mailto:sam at surfline.com>> wrote:
Hi,
I’m working with hourly GFS total cloud cover (TCDCclm) and I’m a bit stumped on the following..
Given TCDCclm for hours 0-1, 0-2, 0-3, 0-4, 0-5, and 0-6, what is the proper way to determine TCDCclm for hours 1-2, 2-3, 3-4, 4-5, and 5-6?
I may be making the problem more difficult than it is..but wanted to ping this group to be sure.
Is there an existing grads script that handles this already and if not, does anyone here have any insight?
Thanks so much for your time.
Best,
Sam
_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr
--
Jeff Duda
Post-doctoral research associate
University of Oklahoma School of Meteorology
_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr
--
Jeff Duda
Post-doctoral research associate
University of Oklahoma School of Meteorology
_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>http://gradsusr.org/mailman/listinfo/gradsusr
--
Dr. Shrinivas Moorthi
Research Meteorologist
Global Climate and Weather Modeling Branch
Environmental Modeling Center / National Centers for Environmental Prediction
5830 University Research Court - (W/NP23), College Park MD 20740 USA
Tel:(301)683-3718<tel:%28301%29683-3718>
_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr
_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr
_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr
_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20161104/47706a3b/attachment-0001.html
More information about the gradsusr
mailing list