[gradsusr] GFS total cloud cover

Thu Nov 3 23:08:35 EDT 2016

Sam,

I think we're making this too complicated.  I agree with Moorthi's solution
provided earlier.  Here it is again for convenience:

Let us denote the data from 6 hourly output as C1, C2, C3, C4, C5, C6.

To get hourly mean, then
for hour 1, the mean is C1,
for hour 2, the mean is 2C2-C1
for hour 3, the mean is 3C3-2C2
for hour 4, the mean is 4C4-3C2
for hour 5, the mean is 5C5-4C4
and for hour 6, it is 6C6-5C5

This pattern repeats every six ours.

If you haven't already tried what Moorthi suggested, please try it...it
works!

Stephen McMillan

On Thu, Nov 3, 2016 at 5:59 PM, Sam Wilson <sam at surfline.com> wrote:

> Wesley,
>
> I got it to work by passing in multiple grib files in sequence! As you
> suggested ;)
>
> Sorry for the confusion and thanks again.
>
> Sam
>
> From: <gradsusr-bounces at gradsusr.org> on behalf of Sam Wilson <
> sam at surfline.com>
> Reply-To: GrADS Users Forum <gradsusr at gradsusr.org>
> Date: Thursday, November 3, 2016 at 11:59 AM
>
> To: GrADS Users Forum <gradsusr at gradsusr.org>
> Subject: Re: [gradsusr] GFS total cloud cover
>
> Wesley,
>
> I plotted the output and unfortunately I’m getting the same results. Not
> sure if I need to upgrade wgrib2 on my machine or not for it to work.
>
> Sam
>
> From: <gradsusr-bounces at gradsusr.org> on behalf of Sam Wilson <
> sam at surfline.com>
> Reply-To: GrADS Users Forum <gradsusr at gradsusr.org>
> Date: Thursday, November 3, 2016 at 11:21 AM
> To: GrADS Users Forum <gradsusr at gradsusr.org>
> Subject: Re: [gradsusr] GFS total cloud cover
>
> Wesley,
>
> Here’s the output of wgrib2 -config:
>
> wgrib2 v0.1.9.7a 11/2012 Wesley Ebisuzaki, Reinoud Bokhorst, Jaakko
> Hyvätti, Dusan Jovic, Kristian Nilssen, Karl Pfeiffer, Pablo Romero,
> Manfred Schwarb, Arlindo da Silva, Niklas Sondell, Sergey Varlamov
>
> Compiled on 21:56:36 Jan 10 2013
>
> Netcdf package: 4.1.1 of Jan 30 2014 05:31:22 $ is installed
> mysql package is installed
> regex package is installed
> tigge package is installed
> interpolation package is not installed
> gptpc interface: experimental v0.1
> UDF package is not installed
> maximum number of arguments on command line: 5000
> maximum number of -match,-not,-if, and -not_if arguments: 1000
> stdout buffer length: 30000
> g2clib is the default decoder
> OpenMP: yes
> INT_MAX:   2147483647
> ULONG_MAX: 18446744073709551615
>
> I ran it using the -g2clib 0 option and it appears to work now.
>
> However, when running with the command you provided, the junk file created
> still shows the following (an example using the 3 hour forecast data file):
>
> $ cat gfs0p252016110312hr003.grb2 | wgrib2 -g2clib 0 - -match
> "(:TCDC:entire atmosphere:)" -set_grib_type c3 -if ":TCDC:entire
> atmosphere:" -ncep_norm junk
>
> $ wgrib2 junk
> 1:0:d=2016110312:TCDC:entire atmosphere:*0-3 hour ave fcst*:
>
> Not sure if that’s just the wgrib2 table saying that, when really it’s the
> 2-3 hour forecast.  I was going to plot the results to find out but let me
> know if you have any additional input.
>
> Thanks!
> Sam
>
> From: <gradsusr-bounces at gradsusr.org> on behalf of Wesley Ebisuzaki -
> NOAA Federal <wesley.ebisuzaki at noaa.gov>
> Reply-To: GrADS Users Forum <gradsusr at gradsusr.org>
> Date: Thursday, November 3, 2016 at 11:08 AM
> To: GrADS Users Forum <gradsusr at gradsusr.org>
> Subject: Re: [gradsusr] GFS total cloud cover
>
> Sam,
>
>    Which version of wgrib2?  (wgrib2 -config)
>
>    Try increasing the stack size.  (bash: ulimit)
>
>
> Wesley
>
>
> On Thu, Nov 3, 2016 at 1:32 PM, Sam Wilson <sam at surfline.com> wrote:
>
>> Moorthi and Wesley,
>>
>> Thanks very much for the feedback - I attempted using the ncep_norm
>> option with wgrib2, but got a segmentation fault..argh!
>>
>> My software installation skills are sub-par, so I will try Moorthi’s
>> option and get back to the group.
>>
>> Thanks again..
>>
>> Cheers,
>> Sam
>>
>> From: <gradsusr-bounces at gradsusr.org> on behalf of Wesley Ebisuzaki -
>> NOAA Federal <wesley.ebisuzaki at noaa.gov>
>> Reply-To: GrADS Users Forum <gradsusr at gradsusr.org>
>> Date: Thursday, November 3, 2016 at 5:59 AM
>> To: GrADS Users Forum <gradsusr at gradsusr.org>
>> Subject: Re: [gradsusr] GFS total cloud cover
>>
>> Sam,
>>
>>   You can get the "0-1 hour ave fcst", "1-2 hour ave fcst", etc
>> by using the -ncep_norm option of wgrib2.
>>
>>    http://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/ncep_norm.html
>>
>> Here is a simple script that takes a list of gfs forecast files in order
>>
>>
>> #!/bin/sh
>> #
>> # usage $0 (list of gfs grib2 forecast files in order)
>> #
>> A=":TCDC:entire atmosphere:"
>> B=":TCDC:low cloud layer:"
>> C=":TCDC:middle cloud layer:"
>> D=":TCDC:high cloud layer:"
>>
>> cat $* | wgrib2 - -match "($A|$B|$C|$D)" -set_grib_type c3 \
>>    -if "$A" -ncep_norm $stmp/junk \
>>    -if "$B" -ncep_norm $stmp/junk \
>>    -if "$C" -ncep_norm $stmp/junk \
>>    -if "$D" -ncep_norm $stmp/junk
>>
>>
>> Wesley
>>
>>
>> On Thu, Nov 3, 2016 at 6:59 AM, Shrinivas Moorthi <
>> shrinivas.moorthi at noaa.gov> wrote:
>>
>>> The grib header is correct.
>>> Let us denote the data from 6 hourly output as C1, C2, C3, C4, C5, C6.
>>>
>>> To get hourly mean, then
>>> for hour 1, the mean is C1,
>>> for hour 2, the mean is 2C2-C1
>>> for hour 3, the mean is 3C3-2C2
>>> for hour 4, the mean is 4C4-3C2
>>> for hour 5, the mean is 5C5-4C4
>>> and for hour 6, it is 6C6-5C5
>>>
>>> This pattern repeats every six ours.
>>> I hope this helps.
>>> Moorthi
>>> On 11/02/2016 07:41 PM, Sam Wilson wrote:
>>>
>>> Hi Jeff,
>>>
>>> I plotted a quick and dirty cloud animation using the data - each step
>>> is 1 hour from f001 through f015 in the attached.
>>>
>>> It appears that they are averages over time - you can see the bigger
>>> jumps at steps 8 and 14 (the transition from 0-6hr avg to 6-7hr and from
>>> 6-12hr avg to 12-13hr).
>>>
>>> Marian - thank you for the feedback.  Sounds like using the hourly data
>>> for cloud cover may not be a good idea, unless someone else has a solution.
>>>
>>> Thanks,
>>> Sam
>>>
>>> From: <gradsusr-bounces at gradsusr.org> on behalf of Jeff Duda <
>>> jeffduda319 at gmail.com>
>>> Reply-To: GrADS Users Forum <gradsusr at gradsusr.org>
>>> Date: Wednesday, November 2, 2016 at 3:21 PM
>>> To: GrADS Users Forum <gradsusr at gradsusr.org>
>>> Subject: Re: [gradsusr] GFS total cloud cover
>>>
>>> Sam,
>>> That might just be a wgrib table thing. Have you actually plotted the
>>> field over those times? If it's truly an averaged field, you should be able
>>> to see some indication of that in the texture and smoothness of the fields
>>> as you progress through increasing averaged times.
>>>
>>> My guess is that this is just some arbitrary setting in wgrib and does
>>> not reflect the true nature of the field. But I could be wrong. If there
>>> are any NCEP folks who work on the GFS on this forum, perhaps one of them
>>> could set the record straight.
>>>
>>> Jeff
>>>
>>> On Wed, Nov 2, 2016 at 4:54 PM, Sam Wilson <sam at surfline.com> wrote:
>>>
>>>> Thanks for the response Jeff - it’s strange to me as well!
>>>>
>>>> Running wgrib2 on the data files for forecast hours 001 through 007
>>>> gives the following for TCDCclm:
>>>>
>>>> *f001:*
>>>> 17:7991683:d=2016110212:TCDC:entire atmosphere:*0-1* hour ave fcst:
>>>> *f002:*
>>>> 17:8082626:d=2016110212:TCDC:entire atmosphere:*0-2* hour ave fcst:
>>>> *f003:*
>>>> 17:8059826:d=2016110212:TCDC:entire atmosphere:*0-3 *hour ave fcst:
>>>> *f004:*
>>>> 17:8057865:d=2016110212:TCDC:entire atmosphere:*0-4* hour ave fcst:
>>>> *f005:*
>>>> 17:8119289:d=2016110212:TCDC:entire atmosphere:*0-5* hour ave fcst:
>>>> *f006:*
>>>> 17:8172510:d=2016110212:TCDC:entire atmosphere:*0-6* hour ave fcst:
>>>> *f007:*
>>>> 17:8013825:d=2016110212:TCDC:entire atmosphere:*6-7* hour ave fcst:
>>>>
>>>> And it continues on..once the files go to 3 hourly (after hour 120), we
>>>> have:
>>>>
>>>> *f123:*
>>>> 17:7687687:d=2016110212:TCDC:entire atmosphere:*120-123* hour ave fcst:
>>>> *f126:*
>>>> 17:7778871:d=2016110212:TCDC:entire atmosphere:*120-126* hour ave fcst:
>>>> *f129:*
>>>> 17:7728042:d=2016110212:TCDC:entire atmosphere:*126-129* hour ave fcst:
>>>>
>>>> And finally the 12 hourly files (past hour 240) are all just 12 hour
>>>> average forecasts:
>>>>
>>>> *f252:*
>>>> 17:6845416:d=2016110212:TCDC:entire atmosphere:*240-252* hour ave fcst:
>>>>
>>>> I did think about the method you suggested (tcdcclm - tcdcclm(t-1)) but
>>>> stopped there for the reason you mentioned..
>>>>
>>>> Strange indeed - unless I’m missing something..
>>>>
>>>> Thanks,
>>>> Sam
>>>>
>>>> From: <gradsusr-bounces at gradsusr.org> on behalf of Jeff Duda <
>>>> jeffduda319 at gmail.com>
>>>> Reply-To: GrADS Users Forum <gradsusr at gradsusr.org>
>>>> Date: Wednesday, November 2, 2016 at 2:08 PM
>>>> To: GrADS Users Forum <gradsusr at gradsusr.org>
>>>> Subject: Re: [gradsusr] GFS total cloud cover
>>>>
>>>> It seems a little strange to me that the total cloud cover product
>>>> would be averaged over time. Is that really the case?
>>>>
>>>> Anyway, if you really do have an average over overlapping and
>>>> increasing windows, with nothing else to go on and without applying the
>>>> equations of motion in reverse, you'd have to assume a linear averaging, so
>>>> subtraction of subsequent slices of the field (i.e., tcdcclm -
>>>> tcdcclm(t-1)) would give you piecewise temporal averages. However, that
>>>> method would give you negative cloud cover values which is clearly
>>>> nonsensical, which again is why I question whether or not that array
>>>> contains a temporal average. That just doesn't make a lot of sense.
>>>>
>>>> Jeff Duda
>>>>
>>>> On Wed, Nov 2, 2016 at 3:27 PM, Sam Wilson <sam at surfline.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I’m working with hourly GFS total cloud cover (TCDCclm) and I’m a bit
>>>>> stumped on the following..
>>>>>
>>>>> Given TCDCclm for hours 0-1, 0-2, 0-3, 0-4, 0-5, and 0-6, what is the
>>>>> proper way to determine TCDCclm for hours 1-2, 2-3, 3-4, 4-5, and 5-6?
>>>>>
>>>>> I may be making the problem more difficult than it is..but wanted to
>>>>> ping this group to be sure.
>>>>>
>>>>> Is there an existing grads script that handles this already and if
>>>>> not, does anyone here have any insight?
>>>>>
>>>>> Thanks so much for your time.
>>>>>
>>>>> Best,
>>>>> Sam
>>>>>
>>>>> _______________________________________________
>>>>> gradsusr mailing list
>>>>> gradsusr at gradsusr.org
>>>>> http://gradsusr.org/mailman/listinfo/gradsusr
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Jeff Duda
>>>> Post-doctoral research associate
>>>> University of Oklahoma School of Meteorology
>>>>
>>>> _______________________________________________
>>>> gradsusr mailing list
>>>> gradsusr at gradsusr.org
>>>> http://gradsusr.org/mailman/listinfo/gradsusr
>>>>
>>>>
>>>
>>>
>>> --
>>> Jeff Duda
>>> Post-doctoral research associate
>>> University of Oklahoma School of Meteorology
>>>
>>>
>>> _______________________________________________
>>> gradsusr mailing listgradsusr at gradsusr.orghttp://gradsusr.org/mailman/listinfo/gradsusr
>>>
>>>
>>>
>>> --
>>> Dr. Shrinivas Moorthi
>>> Research Meteorologist
>>> Global Climate and Weather Modeling Branch
>>> Environmental Modeling Center / National Centers for Environmental Prediction
>>> 5830 University Research Court - (W/NP23), College Park MD 20740 USA
>>> Tel:(301)683-3718
>>>
>>>
>>> _______________________________________________
>>> gradsusr mailing list
>>> gradsusr at gradsusr.org
>>> http://gradsusr.org/mailman/listinfo/gradsusr
>>>
>>>
>>
>> _______________________________________________
>> gradsusr mailing list
>> gradsusr at gradsusr.org
>> http://gradsusr.org/mailman/listinfo/gradsusr
>>
>>
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20161103/436c2a73/attachment-0001.html 