[gradsusr] GFS total cloud cover

Fri Nov 4 13:08:02 EDT 2016

Sam,

TCDC is stored to the nearest percentage in grib2

for hour 1, the mean is C1,
for hour 2, the mean is 2C2-C1
for hour 3, the mean is 3C3-2C2
for hour 4, the mean is 4C4-3C2
for hour 5, the mean is 5C5-4C4
and for hour 6, it is 6C6-5C5

So C1, C2, .., C6 could have an error of +/- 0.5 when stored in grib2.
Therefore the error for hour 6 is   +/-6*0.5 + +/-5*0.5  or +/- 5.5%.

By the ways, wgrib2 v0.1.9.7a is 4 year old and some repositories
of precompiled codes had problems with segfaults unless you
turned off g2clib.  There are two ways to compile g2clib and
they were not compatible.

Wesley

On Fri, Nov 4, 2016 at 1:27 AM, Sam Wilson <sam at surfline.com> wrote:

> Hi Stephen,
>
> Thanks for the feedback!
>
> I did try the method Moorthi pointed out initially but was getting values
> below 0 and above 100 occasionally, so began pursuing Wesley’s wgrib2
> route.
>
> Is getting occasional values below 0 and above 100 to be expected with
> Moorthi's method? Nothing wildly below 0 or above 100 (just a few
> percent)..but want to make sure I’m doing things correctly..
>
> I did make some graphics using Moorthi’s method and they appear to be
> correct - so I may just set anything below 0 to 0 and above 100 to 100 and
> call it a day..
>
> Thanks
> Sam
>
> From: <gradsusr-bounces at gradsusr.org> on behalf of Stephen McMillan <
> smcmillan at planalytics.com>
> Reply-To: GrADS Users Forum <gradsusr at gradsusr.org>
> Date: Thursday, November 3, 2016 at 8:08 PM
> To: GrADS Users Forum <gradsusr at gradsusr.org>
> Subject: Re: [gradsusr] GFS total cloud cover
>
> Sam,
>
> I think we're making this too complicated.  I agree with Moorthi's
> solution provided earlier.  Here it is again for convenience:
>
> Let us denote the data from 6 hourly output as C1, C2, C3, C4, C5, C6.
>
> To get hourly mean, then
> for hour 1, the mean is C1,
> for hour 2, the mean is 2C2-C1
> for hour 3, the mean is 3C3-2C2
> for hour 4, the mean is 4C4-3C2
> for hour 5, the mean is 5C5-4C4
> and for hour 6, it is 6C6-5C5
>
> This pattern repeats every six ours.
>
> If you haven't already tried what Moorthi suggested, please try it...it
> works!
>
> Stephen McMillan
>
> On Thu, Nov 3, 2016 at 5:59 PM, Sam Wilson <sam at surfline.com> wrote:
>
>> Wesley,
>>
>> I got it to work by passing in multiple grib files in sequence! As you
>> suggested ;)
>>
>> Sorry for the confusion and thanks again.
>>
>> Sam
>>
>> From: <gradsusr-bounces at gradsusr.org> on behalf of Sam Wilson <
>> sam at surfline.com>
>> Reply-To: GrADS Users Forum <gradsusr at gradsusr.org>
>> Date: Thursday, November 3, 2016 at 11:59 AM
>>
>> To: GrADS Users Forum <gradsusr at gradsusr.org>
>> Subject: Re: [gradsusr] GFS total cloud cover
>>
>> Wesley,
>>
>> I plotted the output and unfortunately I’m getting the same results. Not
>> sure if I need to upgrade wgrib2 on my machine or not for it to work.
>>
>> Sam
>>
>> From: <gradsusr-bounces at gradsusr.org> on behalf of Sam Wilson <
>> sam at surfline.com>
>> Reply-To: GrADS Users Forum <gradsusr at gradsusr.org>
>> Date: Thursday, November 3, 2016 at 11:21 AM
>> To: GrADS Users Forum <gradsusr at gradsusr.org>
>> Subject: Re: [gradsusr] GFS total cloud cover
>>
>> Wesley,
>>
>> Here’s the output of wgrib2 -config:
>>
>> wgrib2 v0.1.9.7a 11/2012 Wesley Ebisuzaki, Reinoud Bokhorst, Jaakko
>> Hyvätti, Dusan Jovic, Kristian Nilssen, Karl Pfeiffer, Pablo Romero,
>> Manfred Schwarb, Arlindo da Silva, Niklas Sondell, Sergey Varlamov
>>
>> Compiled on 21:56:36 Jan 10 2013
>>
>> Netcdf package: 4.1.1 of Jan 30 2014 05:31:22 $ is installed
>> mysql package is installed
>> regex package is installed
>> tigge package is installed
>> interpolation package is not installed
>> gptpc interface: experimental v0.1
>> UDF package is not installed
>> maximum number of arguments on command line: 5000
>> maximum number of -match,-not,-if, and -not_if arguments: 1000
>> stdout buffer length: 30000
>> g2clib is the default decoder
>> OpenMP: yes
>> INT_MAX:   2147483647
>> ULONG_MAX: 18446744073709551615
>>
>> I ran it using the -g2clib 0 option and it appears to work now.
>>
>> However, when running with the command you provided, the junk file
>> created still shows the following (an example using the 3 hour forecast
>> data file):
>>
>> $ cat gfs0p252016110312hr003.grb2 | wgrib2 -g2clib 0 - -match
>> "(:TCDC:entire atmosphere:)" -set_grib_type c3 -if ":TCDC:entire
>> atmosphere:" -ncep_norm junk
>>
>> $ wgrib2 junk
>> 1:0:d=2016110312:TCDC:entire atmosphere:*0-3 hour ave fcst*:
>>
>> Not sure if that’s just the wgrib2 table saying that, when really it’s
>> the 2-3 hour forecast.  I was going to plot the results to find out but let
>> me know if you have any additional input.
>>
>> Thanks!
>> Sam
>>
>> From: <gradsusr-bounces at gradsusr.org> on behalf of Wesley Ebisuzaki -
>> NOAA Federal <wesley.ebisuzaki at noaa.gov>
>> Reply-To: GrADS Users Forum <gradsusr at gradsusr.org>
>> Date: Thursday, November 3, 2016 at 11:08 AM
>> To: GrADS Users Forum <gradsusr at gradsusr.org>
>> Subject: Re: [gradsusr] GFS total cloud cover
>>
>> Sam,
>>
>>    Which version of wgrib2?  (wgrib2 -config)
>>
>>    Try increasing the stack size.  (bash: ulimit)
>>
>>
>> Wesley
>>
>>
>> On Thu, Nov 3, 2016 at 1:32 PM, Sam Wilson <sam at surfline.com> wrote:
>>
>>> Moorthi and Wesley,
>>>
>>> Thanks very much for the feedback - I attempted using the ncep_norm
>>> option with wgrib2, but got a segmentation fault..argh!
>>>
>>> My software installation skills are sub-par, so I will try Moorthi’s
>>> option and get back to the group.
>>>
>>> Thanks again..
>>>
>>> Cheers,
>>> Sam
>>>
>>> From: <gradsusr-bounces at gradsusr.org> on behalf of Wesley Ebisuzaki -
>>> NOAA Federal <wesley.ebisuzaki at noaa.gov>
>>> Reply-To: GrADS Users Forum <gradsusr at gradsusr.org>
>>> Date: Thursday, November 3, 2016 at 5:59 AM
>>> To: GrADS Users Forum <gradsusr at gradsusr.org>
>>> Subject: Re: [gradsusr] GFS total cloud cover
>>>
>>> Sam,
>>>
>>>   You can get the "0-1 hour ave fcst", "1-2 hour ave fcst", etc
>>> by using the -ncep_norm option of wgrib2.
>>>
>>>    http://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/ncep_norm.html
>>>
>>> Here is a simple script that takes a list of gfs forecast files in order
>>>
>>>
>>> #!/bin/sh
>>> #
>>> # usage $0 (list of gfs grib2 forecast files in order)
>>> #
>>> A=":TCDC:entire atmosphere:"
>>> B=":TCDC:low cloud layer:"
>>> C=":TCDC:middle cloud layer:"
>>> D=":TCDC:high cloud layer:"
>>>
>>> cat $* | wgrib2 - -match "($A|$B|$C|$D)" -set_grib_type c3 \
>>>    -if "$A" -ncep_norm $stmp/junk \
>>>    -if "$B" -ncep_norm $stmp/junk \
>>>    -if "$C" -ncep_norm $stmp/junk \
>>>    -if "$D" -ncep_norm $stmp/junk
>>>
>>>
>>> Wesley
>>>
>>>
>>> On Thu, Nov 3, 2016 at 6:59 AM, Shrinivas Moorthi <
>>> shrinivas.moorthi at noaa.gov> wrote:
>>>
>>>> The grib header is correct.
>>>> Let us denote the data from 6 hourly output as C1, C2, C3, C4, C5, C6.
>>>>
>>>> To get hourly mean, then
>>>> for hour 1, the mean is C1,
>>>> for hour 2, the mean is 2C2-C1
>>>> for hour 3, the mean is 3C3-2C2
>>>> for hour 4, the mean is 4C4-3C2
>>>> for hour 5, the mean is 5C5-4C4
>>>> and for hour 6, it is 6C6-5C5
>>>>
>>>> This pattern repeats every six ours.
>>>> I hope this helps.
>>>> Moorthi
>>>> On 11/02/2016 07:41 PM, Sam Wilson wrote:
>>>>
>>>> Hi Jeff,
>>>>
>>>> I plotted a quick and dirty cloud animation using the data - each step
>>>> is 1 hour from f001 through f015 in the attached.
>>>>
>>>> It appears that they are averages over time - you can see the bigger
>>>> jumps at steps 8 and 14 (the transition from 0-6hr avg to 6-7hr and from
>>>> 6-12hr avg to 12-13hr).
>>>>
>>>> Marian - thank you for the feedback.  Sounds like using the hourly data
>>>> for cloud cover may not be a good idea, unless someone else has a solution.
>>>>
>>>> Thanks,
>>>> Sam
>>>>
>>>> From: <gradsusr-bounces at gradsusr.org> on behalf of Jeff Duda <
>>>> jeffduda319 at gmail.com>
>>>> Reply-To: GrADS Users Forum <gradsusr at gradsusr.org>
>>>> Date: Wednesday, November 2, 2016 at 3:21 PM
>>>> To: GrADS Users Forum <gradsusr at gradsusr.org>
>>>> Subject: Re: [gradsusr] GFS total cloud cover
>>>>
>>>> Sam,
>>>> That might just be a wgrib table thing. Have you actually plotted the
>>>> field over those times? If it's truly an averaged field, you should be able
>>>> to see some indication of that in the texture and smoothness of the fields
>>>> as you progress through increasing averaged times.
>>>>
>>>> My guess is that this is just some arbitrary setting in wgrib and does
>>>> not reflect the true nature of the field. But I could be wrong. If there
>>>> are any NCEP folks who work on the GFS on this forum, perhaps one of them
>>>> could set the record straight.
>>>>
>>>> Jeff
>>>>
>>>> On Wed, Nov 2, 2016 at 4:54 PM, Sam Wilson <sam at surfline.com> wrote:
>>>>
>>>>> Thanks for the response Jeff - it’s strange to me as well!
>>>>>
>>>>> Running wgrib2 on the data files for forecast hours 001 through 007
>>>>> gives the following for TCDCclm:
>>>>>
>>>>> *f001:*
>>>>> 17:7991683:d=2016110212:TCDC:entire atmosphere:*0-1* hour ave fcst:
>>>>> *f002:*
>>>>> 17:8082626:d=2016110212:TCDC:entire atmosphere:*0-2* hour ave fcst:
>>>>> *f003:*
>>>>> 17:8059826:d=2016110212:TCDC:entire atmosphere:*0-3 *hour ave fcst:
>>>>> *f004:*
>>>>> 17:8057865:d=2016110212:TCDC:entire atmosphere:*0-4* hour ave fcst:
>>>>> *f005:*
>>>>> 17:8119289:d=2016110212:TCDC:entire atmosphere:*0-5* hour ave fcst:
>>>>> *f006:*
>>>>> 17:8172510:d=2016110212:TCDC:entire atmosphere:*0-6* hour ave fcst:
>>>>> *f007:*
>>>>> 17:8013825:d=2016110212:TCDC:entire atmosphere:*6-7* hour ave fcst:
>>>>>
>>>>> And it continues on..once the files go to 3 hourly (after hour 120),
>>>>> we have:
>>>>>
>>>>> *f123:*
>>>>> 17:7687687:d=2016110212:TCDC:entire atmosphere:*120-123* hour ave
>>>>> fcst:
>>>>> *f126:*
>>>>> 17:7778871:d=2016110212:TCDC:entire atmosphere:*120-126* hour ave
>>>>> fcst:
>>>>> *f129:*
>>>>> 17:7728042:d=2016110212:TCDC:entire atmosphere:*126-129* hour ave
>>>>> fcst:
>>>>>
>>>>> And finally the 12 hourly files (past hour 240) are all just 12 hour
>>>>> average forecasts:
>>>>>
>>>>> *f252:*
>>>>> 17:6845416:d=2016110212:TCDC:entire atmosphere:*240-252* hour ave
>>>>> fcst:
>>>>>
>>>>> I did think about the method you suggested (tcdcclm - tcdcclm(t-1))
>>>>> but stopped there for the reason you mentioned..
>>>>>
>>>>> Strange indeed - unless I’m missing something..
>>>>>
>>>>> Thanks,
>>>>> Sam
>>>>>
>>>>> From: <gradsusr-bounces at gradsusr.org> on behalf of Jeff Duda <
>>>>> jeffduda319 at gmail.com>
>>>>> Reply-To: GrADS Users Forum <gradsusr at gradsusr.org>
>>>>> Date: Wednesday, November 2, 2016 at 2:08 PM
>>>>> To: GrADS Users Forum <gradsusr at gradsusr.org>
>>>>> Subject: Re: [gradsusr] GFS total cloud cover
>>>>>
>>>>> It seems a little strange to me that the total cloud cover product
>>>>> would be averaged over time. Is that really the case?
>>>>>
>>>>> Anyway, if you really do have an average over overlapping and
>>>>> increasing windows, with nothing else to go on and without applying the
>>>>> equations of motion in reverse, you'd have to assume a linear averaging, so
>>>>> subtraction of subsequent slices of the field (i.e., tcdcclm -
>>>>> tcdcclm(t-1)) would give you piecewise temporal averages. However, that
>>>>> method would give you negative cloud cover values which is clearly
>>>>> nonsensical, which again is why I question whether or not that array
>>>>> contains a temporal average. That just doesn't make a lot of sense.
>>>>>
>>>>> Jeff Duda
>>>>>
>>>>> On Wed, Nov 2, 2016 at 3:27 PM, Sam Wilson <sam at surfline.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I’m working with hourly GFS total cloud cover (TCDCclm) and I’m a bit
>>>>>> stumped on the following..
>>>>>>
>>>>>> Given TCDCclm for hours 0-1, 0-2, 0-3, 0-4, 0-5, and 0-6, what is the
>>>>>> proper way to determine TCDCclm for hours 1-2, 2-3, 3-4, 4-5, and 5-6?
>>>>>>
>>>>>> I may be making the problem more difficult than it is..but wanted to
>>>>>> ping this group to be sure.
>>>>>>
>>>>>> Is there an existing grads script that handles this already and if
>>>>>> not, does anyone here have any insight?
>>>>>>
>>>>>> Thanks so much for your time.
>>>>>>
>>>>>> Best,
>>>>>> Sam
>>>>>>
>>>>>> _______________________________________________
>>>>>> gradsusr mailing list
>>>>>> gradsusr at gradsusr.org
>>>>>> http://gradsusr.org/mailman/listinfo/gradsusr
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Jeff Duda
>>>>> Post-doctoral research associate
>>>>> University of Oklahoma School of Meteorology
>>>>>
>>>>> _______________________________________________
>>>>> gradsusr mailing list
>>>>> gradsusr at gradsusr.org
>>>>> http://gradsusr.org/mailman/listinfo/gradsusr
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Jeff Duda
>>>> Post-doctoral research associate
>>>> University of Oklahoma School of Meteorology
>>>>
>>>>
>>>> _______________________________________________
>>>> gradsusr mailing listgradsusr at gradsusr.orghttp://gradsusr.org/mailman/listinfo/gradsusr
>>>>
>>>>
>>>>
>>>> --
>>>> Dr. Shrinivas Moorthi
>>>> Research Meteorologist
>>>> Global Climate and Weather Modeling Branch
>>>> Environmental Modeling Center / National Centers for Environmental Prediction
>>>> 5830 University Research Court - (W/NP23), College Park MD 20740 USA
>>>> Tel:(301)683-3718
>>>>
>>>>
>>>> _______________________________________________
>>>> gradsusr mailing list
>>>> gradsusr at gradsusr.org
>>>> http://gradsusr.org/mailman/listinfo/gradsusr
>>>>
>>>>
>>>
>>> _______________________________________________
>>> gradsusr mailing list
>>> gradsusr at gradsusr.org
>>> http://gradsusr.org/mailman/listinfo/gradsusr
>>>
>>>
>>
>> _______________________________________________
>> gradsusr mailing list
>> gradsusr at gradsusr.org
>> http://gradsusr.org/mailman/listinfo/gradsusr
>>
>>
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20161104/8b59c3eb/attachment-0001.html 