[gradsusr] Problem reading sub-hourly WRF grib files (NCEP code table for 15/30 min records)
Wesley Ebisuzaki - NOAA Federal
wesley.ebisuzaki at noaa.gov
Tue Nov 21 14:42:49 EST 2017
Ivan,
The error message, "add_time: undefined time unit 14", comes from wgrib2.
So you have to be processing the grib2 files.
I don't work with files that are sub-hourly. So the support for
minutes is not robust. There is no support for minutes by
the program grib2ctl.pl. G2ctl has some support for minutes
but it appears to be broken for your files.
Plan: fix g2ctl and alt_g2ctl. I stopped working
on grib1 utilities about 10 years ago.
Wesley
On Tue, Nov 21, 2017 at 1:20 PM, Ivan Toman <ivtoman at inet.hr> wrote:
> Wesley,
>
> Thank you very much for your efforts to help. However I'm still not sure
> we understand each other well? It is not important to me to use grib2
> format. I just want to be
> able to get data from model using GrADS, regardless of format or tools
> used. But I can't postprocess grib1 also, if I use grib2ctl.pl on grib1
> format, I again get same error like when I use g2ctl on grib2, as I pasted
> below.
>
> So basically for me neither grib2 nor grib1 work.
>
> Ivan
>
>
>
> On 11/21/2017 04:48 PM, Wesley Ebisuzaki - NOAA Federal wrote:
>
> Ivan,
>
> I just modified grb1to2.pl to add support for minutes time units. Both
> g2ctl and alt_g2ctl need some changes to add support for templates
> that use minutes and forecast minutes. I'll get to it soon.
>
> The problem is that you are using cnvgrib to convert from grib1
> to grib2. Cnvgrib does not handle grib1 files that use a 30-minute
> time units (value=14) and produces invalid grib2 files.
>
> I talked with the person who use to support cnvgrib and he stated that
> another person complained about a similar problem and that cnvgrib
> is no longer supported. So you are not going to get the fix from
> the official source of the program. The fix is straight forward;
> however, you should reconsider how you do your processing.
>
> Wesley
>
> On Mon, Nov 20, 2017 at 6:47 PM, Ivan Toman <ivtoman at inet.hr> wrote:
>
>> Thank you Wesley,
>>
>> but it seems that same problem happens with grib1 format, directly from
>> UPP postprocessor:
>>
>> grib2ctl.pl -verf WRFPRS_d01.00_30 >30.ctl
>> add_time: undefined time unit 14
>> PDS_date: problem
>> add_time: undefined time unit 14
>> PDS_date: problem
>>
>> wgrib WRFPRS_d01.00_20
>> 1:0:d=17111700:RWMR:kpds5=170:kpds6=109:kpds7=1:TR=10:P1=0:P2=20:TimeU=0:hybrid lev 1:20min fcst:NAve=0
>> 2:44496:d=17111700:SNMR:kpds5=171:kpds6=109:kpds7=1:TR=10:P1=0:P2=20:TimeU=0:hybrid lev 1:20min fcst:NAve=0
>> 3:88992:d=17111700:REFC:kpds5=212:kpds6=200:kpds7=0:TR=10:P1=0:P2=20:TimeU=0:atmos col:20min fcst:NAve=0
>>
>> But:
>>
>> wgrib WRFPRS_d01.00_30
>> 1:0:d=17111700:RWMR:kpds5=170:kpds6=109:kpds7=1:TR=10:P1=0:P2=1:TimeU=14:hybrid lev 1:30min fcst:NAve=0
>> 2:44496:d=17111700:SNMR:kpds5=171:kpds6=109:kpds7=1:TR=10:P1=0:P2=1:TimeU=14:hybrid lev 1:30min fcst:NAve=0
>> 3:88992:d=17111700:REFC:kpds5=212:kpds6=200:kpds7=0:TR=10:P1=0:P2=1:TimeU=14:atmos col:30min fcst:NAve=0
>>
>>
>> I believe, pretty much the same response on time units on grib1 format.
>>
>> Is this expected behaviour or grib files look wrong? I uploaded them here
>> if you want to take a look:
>> http://gamma.meteoadriatic.net/tmp/jennifer/
>>
>> Thx
>> Ivan
>>
>>
>> On 11/20/2017 01:52 AM, Wesley Ebisuzaki - NOAA Federal wrote:
>>
>> Ivan,
>>
>> Back from a conference. 14 = 1/2 hour for grib1. However
>> there isn't a 1/2 hour time unit for grib2.
>>
>> http://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_table4-4.shtml
>>
>> The code needs to be changed to use minutes as the grib2 time units.
>> Note: grib1 only used 1 byte or 2 bytes to store the forecast length.
>> Therefore they invented a large number of time units. Grib2 uses
>> 4 bytes to store the number of time units.
>>
>> You can try to convince NCO to fix cnvgrib. I know the person
>> responsible for cnvgrib and good luck.
>>
>> My suggestion #1 doesn't work
>>
>> $ wgrib2 wrfprs_d01.00_30.g2 -if ":30 min fcst:" -set_ftime "30 min fcst"
>> -fi -grib wrfprs_d01.00_30.g2.con
>> add_time: undefined time unit 14
>> add_time: undefined time unit 14
>> add_time: undefined time unit 14
>>
>> -if ":30 min fcst:" will always fail because wgrib2 doesn't understand
>> the time code 14.
>> The error message "add_time: undefined time unit 14" comes from the
>> calculation of
>> the verification time.
>>
>> Wesley
>>
>>
>> On Fri, Nov 17, 2017 at 3:34 AM, Ivan Toman <ivtoman at inet.hr> wrote:
>>
>>> Wesley,
>>>
>>> I think this comes from here:
>>>
>>> On grib1 output from UPP - this is for 10min record, all normal:
>>>
>>> $ wgrib WRFPRS_d01.00_10
>>> 1:0:d=17111700:RWMR:kpds5=170:kpds6=109:kpds7=1:TR=10:P1=0:P2=10:TimeU=0:hybrid
>>> lev 1:10min fcst:NAve=0
>>> 2:44496:d=17111700:SNMR:kpds5=171:kpds6=109:kpds7=1:TR=10:P1=0:P2=10:TimeU=0:hybrid
>>> lev 1:10min fcst:NAve=0
>>> 3:88992:d=17111700:REFC:kpds5=212:kpds6=200:kpds7=0:TR=10:P1=0:P2=10:TimeU=0:atmos
>>> col:10min fcst:NAve=0
>>> ...
>>>
>>> however, for 30min record:
>>>
>>> $ wgrib WRFPRS_d01.00_30
>>> 1:0:d=17111700:RWMR:kpds5=170:kpds6=109:kpds7=1:TR=10:P1=0:P2=1:TimeU=14:hybrid
>>> lev 1:30min fcst:NAve=0
>>> 2:44496:d=17111700:SNMR:kpds5=171:kpds6=109:kpds7=1:TR=10:P1=0:P2=1:TimeU=14:hybrid
>>> lev 1:30min fcst:NAve=0
>>> 3:88992:d=17111700:REFC:kpds5=212:kpds6=200:kpds7=0:TR=10:P1=0:P2=1:TimeU=14:atmos
>>> col:30min fcst:NAve=0
>>> ...
>>>
>>> TimeU=14
>>>
>>>
>>> Where this comes from?
>>>
>>> As far as I understand it, UPP follows this time designation for time
>>> units:
>>> http://www.nco.ncep.noaa.gov/pmb/docs/on388/table4.html
>>> 14 = "Half an hour"
>>>
>>> This is from UPP code that sets it:
>>>
>>> ! J. HALLEY GOTWAY, MODIFY HOW THE TIME INFORMATION IS STORED IN
>>> ID(17-20),
>>> ! (FCST TIME UNIT, P1, P2, TIME RANGE INDICATOR).
>>> ! CHECK IF THE NUMBER OF FORECAST MINUTES IS ZERO FOR HOURS OR
>>> NON-ZERO FOR
>>> ! OFF-HOUR FORECASTS. FOR OFF-HOUR FORECASTS, CHECK IF THE TOTAL
>>> NUMBER
>>> ! OF MINUTES IS DIVISIBLE BY 30, 15, OR NEITHER, AND USE THE
>>> APPROPRIATE
>>> ! FCST TIME UNIT VALUE. FOR ANY FIELD OTHER THAN AN INSTANTANEOUS
>>> FIELD,
>>> ! ASSUME ID(18-20), ARE PASSED IN CORRECTLY.
>>>
>>> IF(IFMIN .GE. 1)THEN
>>> ! ID(17) = 0
>>> ! COMPUTE THE TOTAL FORECAST MINUTES.
>>> TOTMIN=IFHR*60+IFMIN
>>>
>>> ! CHECK FOR 1/2 HOURLY INCREMENTS.
>>> IF (MOD(TOTMIN, 30) == 0) THEN
>>> ID(17) = 14
>>> DIV = 30
>>> ! CHECK FOR 1/4 HOURLY INCREMENTS.
>>> ELSEIF (MOD(TOTMIN, 15) == 0) THEN
>>> ID(17) = 13
>>> DIV = 15
>>> ! OTHERWISE, USE MINUTES.
>>> ELSE
>>> ID(17) = 0
>>> DIV = 1
>>> ENDIF
>>>
>>>
>>> Does this look OK to you?
>>>
>>> Thx
>>> Ivan
>>>
>>>
>>>
>>>
>>>
>>> On 11/15/2017 02:39 PM, Wesley Ebisuzaki - NOAA Federal wrote:
>>>
>>> Ivan,
>>>
>>> I am not aware of a time unit 14. It's not in NCEP's documentation.
>>>
>>> Wesley
>>>
>>> http://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_table4-4.shtml
>>>
>>> On Wed, Nov 15, 2017 at 4:27 AM, Ivan Toman <ivtoman at inet.hr> wrote:
>>>
>>>> Wesley,
>>>>
>>>> I finally found time to dig into this.
>>>>
>>>> I actually get "undefined time unit 14" from wgrib2 when trying to
>>>> handle 30 min records. That's probably why g2ctl, alt_g2ctl also does not
>>>> do the job (throwing this message) and so on.
>>>>
>>>> For example, your suggestion #2:
>>>>
>>>> $ wgrib2 wrfprs_d01.00_30.g2 -if ":30 min fcst:" -set_ftime "30 min
>>>> fcst" -fi -grib wrfprs_d01.00_30.g2.con
>>>> add_time: undefined time unit 14
>>>> add_time: undefined time unit 14
>>>> add_time: undefined time unit 14
>>>> 1:0:d=2017111500 <%28201%29%20711-1500>:RWMR:1 hybrid level:1 ? fcst:
>>>> add_time: undefined time unit 14
>>>> add_time: undefined time unit 14
>>>> add_time: undefined time unit 14
>>>> 2:2310:d=2017111500 <%28201%29%20711-1500>:SNMR:1 hybrid level:1 ?
>>>> fcst:
>>>> add_time: undefined time unit 14
>>>> ...
>>>>
>>>>
>>>> Or...
>>>>
>>>> $ alt_gmp -i wrfprs_d01.grib2.ctl
>>>> wgrib2_flags=-npts -set_ext_name 1 -end_FT -ext_name -lev
>>>> wgrib2_inv=.invd01
>>>> dtype: dtype grib2
>>>> pdef: pdef 149 149 lccr -29.363000 45.203000 1 1 -21.633000 -21.633000
>>>> 54.288000 12000.000000 12000.000000tdef: nt=7 start=00Z15nov2017 by=10mn
>>>> zdef: nlevel=25
>>>> resolve_dsets dset=wrfprs_d01.grib2 inctime=10mn
>>>> resolve_dsets: no template
>>>> scanning wrfprs_d01.grib2 (process=0)
>>>> add_time: undefined time unit 14
>>>> add_time: undefined time unit 14
>>>> add_time: undefined time unit 14
>>>> add_time: undefined time unit 14
>>>> add_time: undefined time unit 14
>>>>
>>>>
>>>>
>>>> grib1to2.pl did not work either, it failed:
>>>>
>>>> $ for i in WRFPRS_d01.0* ; do ./grb1to2.pl -o $i.g2 $i ; done
>>>> (((rec 1:0:date 2017111500 <%28201%29%20711-1500> RWMR kpds5=170
>>>> kpds6=109 kpds7=1 levels=(0,1) grid=255 hybrid lev 1 anl:
>>>> RWMR=Rain water mixing ratio [kg/kg]
>>>> timerange 0 P1 0 P2 0 TimeU 1 nx 149 ny 149 GDS grid 3 num_in_ave 0
>>>> missing 0
>>>> center 7 subcenter 0 process 125 Table 2 scan: WE:SN winds(grid)
>>>> Lambert Conf: Lat1 -29.363000 Lon1 45.203000 Lov 54.288000
>>>> Latin1 -21.633000 Latin2 -21.633000 LatSP -90.000000 LonSP
>>>> 0.000000
>>>> South Pole (149 x 149) Dx 12.000000 Dy 12.000000 scan 64 mode 136
>>>> min/max data 0 0 num bits 0 BDS_Ref 0 DecScale 0 BinScale 0
>>>>
>>>> )))
>>>> >> grib2_metadata --- WRFPRS_d01.00_00 wgrib=./wgrib wgrib2=./wgrib2
>>>> Use of uninitialized value $1 in negation (-) at ./grib1to2_metadata.pl
>>>> line 74, <INV> line 1.
>>>> no scan mode at ./grib1to2_metadata.pl line 83, <INV> line 1.
>>>> missing GRIB record(s)
>>>> ((()))
>>>> Problem, winds not defined! old ./wgrib?
>>>> grib2 message ignored (use wgrib2)
>>>> missing GRIB record(s)
>>>> ((()))
>>>> Problem, winds not defined! old ./wgrib?
>>>> (((rec 1:0:date 2017111500 <%28201%29%20711-1500> RWMR kpds5=170
>>>> kpds6=109 kpds7=1 levels=(0,1) grid=255 hybrid lev 1 10min fcst:
>>>> RWMR=Rain water mixing ratio [kg/kg]
>>>> timerange 10 P1 0 P2 10 TimeU 0 nx 149 ny 149 GDS grid 3 num_in_ave
>>>> 0 missing 0
>>>> center 7 subcenter 0 process 125 Table 2 scan: WE:SN winds(grid)
>>>> Lambert Conf: Lat1 -29.363000 Lon1 45.203000 Lov 54.288000
>>>> Latin1 -21.633000 Latin2 -21.633000 LatSP -90.000000 LonSP
>>>> 0.000000
>>>> South Pole (149 x 149) Dx 12.000000 Dy 12.000000 scan 64 mode 136
>>>> min/max data 0 5.31824e-06 num bits 16 BDS_Ref 0 DecScale 11
>>>> BinScale 4
>>>>
>>>>
>>>>
>>>> I have no idea how to proceed further.
>>>> Tks,
>>>> Ivan
>>>>
>>>>
>>>>
>>>>
>>>> On 11/07/2017 08:12 PM, Wesley Ebisuzaki - NOAA Federal wrote:
>>>>
>>>> Ivan,
>>>>
>>>> "UPP follows NCEP code table for timing grib records - for 15 and 30
>>>> minute records use codes 13 and 14 respectively instead of simple
>>>> minutes. This seems to confuse GrADS (or g2ctl/gribmap, I'm not sure)."
>>>>
>>>> "Yes it is fixed. I'm currently need to output 10 min intervals. So,
>>>> for
>>>> full hour, 10m, 20m, 40m and 50m I get OK data. But for 30m I do not."
>>>>
>>>> These quotes suggests that it is a problem with gribmap expecting
>>>> 30 minute_time_units and not being able to handle 1
>>>> 30_minute_time_units.
>>>>
>>>> Suggestion #1
>>>>
>>>> use alt_g2ctl/alt_gmp
>>>>
>>>> Both are based on wgrib2 which uses english rather than code table
>>>> numbers.
>>>>
>>>> Suggestion #2
>>>>
>>>> convert forecast time from 1 (30 minutes) to 30 (minutes)
>>>>
>>>> wgrib2 IN.grb -if ":30 min fcst:" -set_ftime "30 min fcst" -fi
>>>> -grib OUT.grb
>>>>
>>>> Wesley
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Nov 7, 2017 at 1:21 PM, Jeff Duda <jeffduda319 at gmail.com>
>>>> wrote:
>>>>
>>>>> What result do you get when you do
>>>>>
>>>>> wgrib2 -T (grib file)
>>>>>
>>>>> ?
>>>>>
>>>>> Look after the "D=", as the represents the time in the GRIB2 file.
>>>>>
>>>>> Jeff
>>>>>
>>>>> On Tue, Nov 7, 2017 at 12:10 PM, Ivan Toman <ivtoman at inet.hr> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> Yes it is fixed. I'm currently need to output 10 min intervals. So,
>>>>>> for full hour, 10m, 20m, 40m and 50m I get OK data. But for 30m I do not.
>>>>>>
>>>>>> Thank you!
>>>>>> Ivan
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 11/07/2017 05:50 PM, Jeff Duda wrote:
>>>>>>
>>>>>> What is your time interval? If it's not fixed, that's one problem. If
>>>>>> it is fixed, then I don't see why regular templating wouldn't work unless
>>>>>> the time being written to the grib2 file is not right. But there are ways
>>>>>> to fix that.
>>>>>>
>>>>>> Jeff Duda
>>>>>>
>>>>>> On Tue, Nov 7, 2017 at 10:12 AM, Ivan Toman <ivtoman at inet.hr> wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> I'm having difficulties reading sub-hourly time records from WRF grib
>>>>>>> files in GrADS.
>>>>>>>
>>>>>>> UPP follows NCEP code table for timing grib records - for 15 and 30
>>>>>>> minute records use codes 13 and 14 respectively instead of simple
>>>>>>> minutes. This seems to confuse GrADS (or g2ctl/gribmap, I'm not
>>>>>>> sure).
>>>>>>>
>>>>>>> What I get as result is that I can read any sub-hourly record as
>>>>>>> long as
>>>>>>> it is not 15 or 30 minute record. For those, I get undefined grid
>>>>>>> instead of data.
>>>>>>>
>>>>>>> I use this workflow for postprocessing: wfrout >(UPP)> grib1
>>>>>>> >(cnvgrib)>
>>>>>>> grib2 >(g2ctl,gribmap)> GrADS
>>>>>>>
>>>>>>> Does anybody know what is going on there?
>>>>>>>
>>>>>>> Thank you in advance
>>>>>>>
>>>>>>> Ivan Toman
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> gradsusr mailing list
>>>>>>> gradsusr at gradsusr.org
>>>>>>> http://gradsusr.org/mailman/listinfo/gradsusr
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Jeff Duda
>>>>>> Post-doctoral research fellow
>>>>>> University of Oklahoma School of Meteorology
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> gradsusr mailing listgradsusr at gradsusr.orghttp://gradsusr.org/mailman/listinfo/gradsusr
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> gradsusr mailing list
>>>>>> gradsusr at gradsusr.org
>>>>>> http://gradsusr.org/mailman/listinfo/gradsusr
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Jeff Duda
>>>>> Post-doctoral research fellow
>>>>> University of Oklahoma School of Meteorology
>>>>>
>>>>> _______________________________________________
>>>>> gradsusr mailing list
>>>>> gradsusr at gradsusr.org
>>>>> http://gradsusr.org/mailman/listinfo/gradsusr
>>>>>
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> gradsusr mailing listgradsusr at gradsusr.orghttp://gradsusr.org/mailman/listinfo/gradsusr
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> gradsusr mailing list
>>>> gradsusr at gradsusr.org
>>>> http://gradsusr.org/mailman/listinfo/gradsusr
>>>>
>>>>
>>>
>>>
>>> _______________________________________________
>>> gradsusr mailing listgradsusr at gradsusr.orghttp://gradsusr.org/mailman/listinfo/gradsusr
>>>
>>>
>>>
>>> _______________________________________________
>>> gradsusr mailing list
>>> gradsusr at gradsusr.org
>>> http://gradsusr.org/mailman/listinfo/gradsusr
>>>
>>>
>>
>>
>> _______________________________________________
>> gradsusr mailing listgradsusr at gradsusr.orghttp://gradsusr.org/mailman/listinfo/gradsusr
>>
>>
>>
>> _______________________________________________
>> gradsusr mailing list
>> gradsusr at gradsusr.org
>> http://gradsusr.org/mailman/listinfo/gradsusr
>>
>>
>
>
> _______________________________________________
> gradsusr mailing listgradsusr at gradsusr.orghttp://gradsusr.org/mailman/listinfo/gradsusr
>
>
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20171121/e6bb617c/attachment-0001.html
More information about the gradsusr
mailing list