[gradsusr] Getting SREF Probability Variables To Plot In GrADS
Perry, Aaron @ LSC
Aaron.Perry at lsc.vsc.edu
Fri Jun 6 16:55:12 EDT 2014
Hi Jennifer,
Thank you very, very much for all of your assistance this afternoon. Your knowledge about this stuff is very extensive and useful. I will go back and apply your suggestions and see what I can come up with. I guess that part of the solution to a problem is actually understanding the problem.
Also, someone else on the mailing list is currently helping me through the process.
Again, thank you for all of your help and have a great weekend!
Onward and Upward,
Aaron D. Perry
Atmospheric Sciences
Lyndon State College
Class of 2016
Boston, MA
Twitter: @AaronPerryLSC<https://twitter.com/AaronPerryLSC>
Mobile: 617-780-4312
________________________________
From: gradsusr-bounces at gradsusr.org [gradsusr-bounces at gradsusr.org] on behalf of Jennifer Adams [jma at cola.iges.org]
Sent: Friday, June 06, 2014 4:26 PM
To: GrADS Users Forum
Subject: Re: [gradsusr] Getting SREF Probability Variables To Plot In GrADS
On Jun 6, 2014, at 3:02 PM, Perry, Aaron @ LSC wrote:
Great detective work Jennifer!
I've created my subset file by using the following method:
/mnt/grads/bin/wgrib2 /home/aaron/grads/data/${MODEL}_prob/${YMD}/${RUNHR}Z/${MODEL}.${YMD}.${RUNHR}z.prob.grib2 -match '^(182|255|371|487|623|759|895):' -grib /home/aaron/grads/data/${MODEL}_prob/${YMD}/${RUNHR}Z/${MODEL}_prob.${YMD}.${RUNHR}z.grib2
Maybe Wesley can comment on whether this is correct or not.
Originally, I was just playing around with it to get multiple variables but, clearly that has messed me up.
I just used get_grib/get_inv to download only the APCP files in the GRIB2 and then ran wgrib2 sref.t15z.pgrb132.prob_3hrly.grib2 | grep ">0.25" on it and got every timestep through 87hrs.
Just piping the output from wgrib2 through 'grep' will not subset the files. Have a look at the wgrib2 documentation to figure out the correct syntax for extracting specific records from a grib2 file.
But first you must decide if you want probabilities based on 3hr, 6hr, 12hr, or 24hr accumulations.
For the sake of example, suppose you choose 6-hr accumulations.
You want to extract these records and put them in a separate file:
308:20541156:d=2014060415:APCP:surface:0-6 hour acc fcst:prob >0.25
580:41728223:d=2014060415:APCP:surface:6-12 hour acc fcst:prob >0.25
892:66095269:d=2014060415:APCP:surface:12-18 hour acc fcst:prob >0.25
1204:91189108:d=2014060415:APCP:surface:18-24 hour acc fcst:prob >0.25
...and so on for the remaining 6hr periods in the forecast.
Then you change your DSET to point to the new file you created with the subset of records, and change your TDEF to have a 6hr increment, like this:
TDEF 15 linear 15z04jun2014 6hr
Then run gribmap without the -0 (use the end of the accumulation period as the valid time) and with the -v so you can make sure that everything matches. Then if you are still having problems displaying data, you must send the following:
1. the descriptor file
2. the output from gribmap -v
3. the output from running wgrib2 on your subset data file
Note that the first time step (initialization time) will not have any data because your first record is valid at 6hrs past initialization. To make sure everything has worked, use 'set t 2' before displaying anything.
I then ran g2ctl and then gribmap -0, as well as changing the control file with the proper dimensions.
Still not getting anything to display...
Is this hopeless?
It shouldn't be.
--Jennifer
Onward and Upward,
Aaron D. Perry
Atmospheric Sciences
Lyndon State College
Class of 2016
Boston, MA
Twitter: @AaronPerryLSC<https://twitter.com/AaronPerryLSC>
Mobile: 617-780-4312
________________________________
From: gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org> [gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org>] on behalf of Jennifer Adams [jma at cola.iges.org<mailto:jma at cola.iges.org>]
Sent: Friday, June 06, 2014 2:31 PM
To: GrADS Users Forum
Subject: Re: [gradsusr] Getting SREF Probability Variables To Plot In GrADS
So, at last, all the records in your file have matched. Can you display them?
I took a look at the file you pointed to
at:http://ftp.ncep.noaa.gov/data/nccf/com/sref/prod/sref.20140604/15/ensprod/sref.t15z.pgrb132.prob_3hrly.grib2
This file is not the same as the one you are looking at. That file has 4980 records, and you appear to have only 7. So, how exactly did you create your subset file?
I ask this because the file being served by NCEP is peculiar in that is has at most 4 records that would match a given ctl entry. For clarity, I will focus on only one varible, with a descriptor file entry like this:
p3 0,1,0 a1,0.25 0,1,8,1 Probability of accumulated precip above 0.25
It is easy to find the records for this variable by grepping for "0.25" in the wgrib2 output.
# wgrib2 sref.t15z.pgrb132.prob_3hrly.grib2 | grep ">0.25"
182:10551812:d=2014060415:APCP:surface:0-3 hour acc fcst:prob >0.25
298:19945733:d=2014060415:APCP:surface:3-6 hour acc fcst:prob >0.25
308:20541156:d=2014060415:APCP:surface:0-6 hour acc fcst:prob >0.25
434:30428673:d=2014060415:APCP:surface:6-9 hour acc fcst:prob >0.25
444:31008021:d=2014060415:APCP:surface:3-9 hour acc fcst:prob >0.25
570:41155064:d=2014060415:APCP:surface:9-12 hour acc fcst:prob >0.25
580:41728223:d=2014060415:APCP:surface:6-12 hour acc fcst:prob >0.25
590:42476633:d=2014060415:APCP:surface:0-12 hour acc fcst:prob >0.25
726:53157513:d=2014060415:APCP:surface:12-15 hour acc fcst:prob >0.25
736:53746450:d=2014060415:APCP:surface:9-15 hour acc fcst:prob >0.25
746:54503902:d=2014060415:APCP:surface:3-15 hour acc fcst:prob >0.25
882:65476225:d=2014060415:APCP:surface:15-18 hour acc fcst:prob >0.25
892:66095269:d=2014060415:APCP:surface:12-18 hour acc fcst:prob >0.25
902:66876471:d=2014060415:APCP:surface:6-18 hour acc fcst:prob >0.25
1038:78030238:d=2014060415:APCP:surface:18-21 hour acc fcst:prob >0.25
1048:78673344:d=2014060415:APCP:surface:15-21 hour acc fcst:prob >0.25
1058:79484823:d=2014060415:APCP:surface:9-21 hour acc fcst:prob >0.25
1194:90543385:d=2014060415:APCP:surface:21-24 hour acc fcst:prob >0.25
1204:91189108:d=2014060415:APCP:surface:18-24 hour acc fcst:prob >0.25
1214:92014359:d=2014060415:APCP:surface:12-24 hour acc fcst:prob >0.25
1224:93013217:d=2014060415:APCP:surface:0-1 day acc fcst:prob >0.25
etc.
The last four records in this list all have the same valid time, 24hrs from initialization, and they will all MATCH the set of grib codes in the descriptor entry above, yet they are not the same thing. Record 1194 is a 3hr accumulation, record 1204 is a 6hr accumulation, record 1214 is a 12hr accumulation, and record 1224 is a 24hr accumulation. If more than one record MATCHES a descriptor file entry, only the final one will be entered in the index file -- in this case, you would be displaying the data in record 1224, the 24hr accumulation. If you ran gribmap with the -0 option (to match with the inital time in an averaging/accumulation period) you would still have four records that match the 00hr time step: records 182, 308, 590, and 1224.
So, some careful examination of the metadata and then subsetting of the NCEP files is required before you can be 100% certain of what you are looking at when it comes to these SREF probability forecasts. I am not surprised that g2ctl is confused.
--Jennifer
p.s. This time axis covers all the timesteps in the forecast: TDEF 30 linear 15Z04jun2014 3hr
On Jun 6, 2014, at 12:34 PM, Perry, Aaron @ LSC wrote:
Hi Jennifer,
Here is the output from gribmap -v:
[aaron at arrakis 09Z]gribmap -v sref_prob.20140606.09z.ctl
Invalid command line argument: sref_prob.20140606.09z.ctl Ignored.
Enter name of Data Descriptor file: sref_prob.20140606.09z.ctl
gribmap: scanning GRIB2 file: /home/aaron/grads/data/sref_prob/20140606/09Z/sref_prob.20140606.09z.grib2
1.1: lev1=1,0 a1,1.27 var=0,1,8,1 MATCH
2.1: lev1=1,0 a1,12.7 var=0,1,8,1 MATCH
3.1: lev1=1,0 a1,0.25 var=0,1,8,1 MATCH
4.1: lev1=1,0 a1,38.1 var=0,1,8,1 MATCH
5.1: lev1=1,0 a1,2.54 var=0,1,8,1 MATCH
6.1: lev1=1,0 a1,101.6 var=0,1,8,1 MATCH
7.1: lev1=1,0 a1,12.7 var=0,1,8,1 MATCH
gribmap: Writing out the index file
For the record, when I automatically run the grib2 through g2ctl/gribmap, it automatically sets the date to 1 day ahead (09Z07jun2014 instead of 09Z05jun2014), sets TDEF to 10, not 27 and only has one variable, APCP, so I have to copy/paste the 7 defined variables of QPF probability.
Onward and Upward,
Aaron D. Perry
Atmospheric Sciences
Lyndon State College
Class of 2016
Boston, MA
Twitter: @AaronPerryLSC<https://twitter.com/AaronPerryLSC>
Mobile: 617-780-4312
________________________________
From: gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org> [gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org>] on behalf of Jennifer Adams [jma at cola.iges.org<mailto:jma at cola.iges.org>]
Sent: Friday, June 06, 2014 12:15 PM
To: GrADS Users Forum
Subject: Re: [gradsusr] Getting SREF Probability Variables To Plot In GrADS
OK, so what about the output from gribmap with the new TDEF? --Jennifer
On Jun 6, 2014, at 12:12 PM, Perry, Aaron @ LSC wrote:
Sorry about that.
Specifically, I am trying to get the QPF probabilities GRIB2, which to my understanding is located at:http://ftp.ncep.noaa.gov/data/nccf/com/sref/prod/sref.20140604/15/ensprod/sref.t15z.pgrb132.prob_3hrly.grib2
I've then been using g2ctl/gribmap to create the necessary .ctl/.idx files, though more recently with all this troubleshooting, I've just been editing one control file.
Originally, I tried using the method that was posted here (http://gradsusr.org/pipermail/gradsusr/2011-November/031529.html)
Onward and Upward,
Aaron D. Perry
Atmospheric Sciences
Lyndon State College
Class of 2016
Boston, MA
Twitter: @AaronPerryLSC<https://twitter.com/AaronPerryLSC>
Mobile: 617-780-4312
________________________________
From: gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org> [gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org>] on behalf of Jennifer Adams [jma at cola.iges.org<mailto:jma at cola.iges.org>]
Sent: Friday, June 06, 2014 11:03 AM
To: GrADS Users Forum
Subject: Re: [gradsusr] Getting SREF Probability Variables To Plot In GrADS
Aaron, there are quite a file subdirectories under http://ftp.ncep.noaa.gov/data/nccf/com/sref/prod/sref.20140604/15. Could you please be completely specific about the URL that points to the file you are trying to read? Also, your TDEF starts 00z01jun with only 10 3-hour time steps -- this will not cover the apparent time range of your data, it won't even come close to the initial time of your forecast: 15z4jun2014.
What is the output from gribmap -v on you descriptor with a tdef that looks like like this:
TDEF 27 linear 15z4jun2014 3hr
This time axis starts at the initial date of your data file and extend to 21z07jun2014, which is the apparent valid time of the final record in your file.
--Jennifer
On Jun 6, 2014, at 6:40 AM, Perry, Aaron @ LSC wrote:
Hi Jennifer,
Here's my data source:
http://ftp.ncep.noaa.gov/data/nccf/com/sref/prod/
Onward and Upward,
Aaron D. Perry
Atmospheric Sciences
Lyndon State College
Class of 2016
Boston, MA
Twitter: @AaronPerryLSC<https://twitter.com/AaronPerryLSC>
Mobile: 617-780-4312
________________________________
From: gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org> [gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org>] on behalf of Jennifer Adams [jma at cola.iges.org<mailto:jma at cola.iges.org>]
Sent: Friday, June 06, 2014 6:31 AM
To: GrADS Users Forum
Subject: Re: [gradsusr] Getting SREF Probability Variables To Plot In GrADS
Can you point me to the source of your data file? --Jennifer
On Jun 5, 2014, at 1:29 PM, Perry, Aaron @ LSC wrote:
Jennifer,
After carrying out your suggestions, I am still getting the "Data Request Warning/Cannot Contour" errors.
I've attached the control file and the GrADS script that I'm using to try and plot, at this point, anything.
Onward and Upward,
Aaron D. Perry
Atmospheric Sciences
Lyndon State College
Class of 2016
Boston, MA
Twitter: @AaronPerryLSC<https://twitter.com/AaronPerryLSC>
Mobile: 617-780-4312
________________________________
From: gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org> [gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org>] on behalf of Jennifer Adams [jma at cola.iges.org<mailto:jma at cola.iges.org>]
Sent: Thursday, June 05, 2014 6:49 AM
To: GrADS Users Forum
Subject: Re: [gradsusr] Getting SREF Probability Variables To Plot In GrADS
Looks like you need to change your time axis to be 3 hourly and extend it to 00z21jun. --Jennifer
On Jun 4, 2014, at 8:46 PM, Perry, Aaron @ LSC wrote:
Hi Jennifer,
I just went back and fixed those variables in the control file.
Here's the output from gribmap -v:
1.1: lev1=1,0 a1,1.27 var=0,1,8,1 MATCH
2.1: lev1=1,0 a1,12.7 var=0,1,8,1 MATCH
3.1: lev1=1,0 a1,0.25 var=0,1,8,1 valid time 2014060606:00 (t=7.5) has non-integer grid index
4.1: lev1=1,0 a1,38.1 var=0,1,8,1 valid time 2014060612:00 (t=8.5) has non-integer grid index
5.1: lev1=1,0 a1,2.54 var=0,1,8,1 valid time 2014060700:00 (t=10.5) is outside grid limits
6.1: lev1=1,0 a1,101.6 var=0,1,8,1 valid time 2014060709:00 (t=12) is outside grid limits
7.1: lev1=1,0 a1,12.7 var=0,1,8,1 valid time 2014060721:00 (t=14) is outside grid limits
Looks like variables 5-7 are giving me issues, possibly THE issue?
Onward and Upward,
Aaron D. Perry
Atmospheric Sciences
Lyndon State College
Class of 2016
Boston, MA
Twitter: @AaronPerryLSC<https://twitter.com/AaronPerryLSC>
Mobile: 617-780-4312
________________________________
From: gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org> [gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org>] on behalf of Jennifer Adams [jma at cola.iges.org<mailto:jma at cola.iges.org>]
Sent: Wednesday, June 04, 2014 6:36 PM
To: GrADS Users Forum
Subject: Re: [gradsusr] Getting SREF Probability Variables To Plot In GrADS
Hi, Aaron --
You don't need the alias syntax with grib data. The var1.1 syntax is just a dummy variable name that is designed to be unique to the record. You can use use "p1" instead of "var1.1=>p1".
What is the output from gribmap -v?
--Jennifer
On Jun 4, 2014, at 6:25 PM, Perry, Aaron @ LSC wrote:
I took a look at the suggestion documentation and ran grib2scan. Below is a copy/paste link of the output. After reviewing the output, I was able to come up with this modified control file:
grib2scan Output: http://paste2.org/zE92be26
Modified Control File (Not working):
dset /home/aaron/grads/data/sref_prob/20140604/15Z/sref_prob.20140604.15z.grib2
index /home/aaron/grads/data/sref_prob/20140604/15Z/sref_prob.20140604.15z.grib2.idx
undef 9.999E+20
title /home/aaron/grads/data/sref_prob/20140604/15Z/sref_prob.20140604.15z.grib2
* produced by g2ctl v0.0.9
* command line options: /home/aaron/grads/data/sref_prob/20140604/15Z/sref_prob.20140604.15z.grib2
* griddef=1:0:(697 x 553):grid_template=30:winds(grid): Lambert Conformal: (697 x 553) input WE:SN output WE:SN res 8 Lat1 1.000000 Lon1 214.500000 LoV 253.000000 LatD 50.000000 Latin1 50.000000 Latin2 50.000000 LatSP 0.000000 LonSP 0.000000 North Po
dtype grib2
pdef 697 553 lccr 1.000000 214.500000 1 1 50.000000 50.000000 253.000000 16232.000000 16232.000000
xdef 1415 linear 148.637041 0.147586114443721
ydef 573 linear 0.896436 0.147563636363636
tdef 10 linear 15Z04jun2014 6hr
zdef 1 linear 1 1
vars 7
var1.1=>p1 0,1,0 a1,1.27 0,1,8,1 Probability of event above 1.27
var2.1=>p2 0,1,0 a1,12.7 0,1,8,1 Probability of event above 12.7
var3.1=>p3 0,1,0 a1,0.25 0,1,8,1 Probability of event above 0.25
var4.1=>p4 0,1,0 a1,38.1 0,1,8,1 Probability of event above 38.1
var5.1=>p5 0,1,0 a1,2.54 0,1,8,1 Probability of event above 2.54
var6.1=>p6 0,1,0 a1,101.6 0,1,8,1 Probability of event above 101.6
var7.1=>p7 0,1,0 a1,12.7 0,1,8,1 Probability of event above 12.7
ENDVARS
Unfortunately, I am still receiving the "Cannot Contour Grid/Data Request Warning" errors. None of the variables will plot.
Onward and Upward,
Aaron D. Perry
Atmospheric Sciences
Lyndon State College
Class of 2016
Boston, MA
Twitter: @AaronPerryLSC<https://twitter.com/AaronPerryLSC>
Mobile: 617-780-4312
________________________________
From: gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org> [gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org>] on behalf of Jennifer Adams [jma at cola.iges.org<mailto:jma at cola.iges.org>]
Sent: Wednesday, June 04, 2014 11:31 AM
To: GrADS Users Forum
Subject: Re: [gradsusr] Getting SREF Probability Variables To Plot In GrADS
Aaron,
Have a look at the doc page on descriptor file elements, under 'vars' and then 'additional codes'. grib2scan in verbose mode should give you a suitable entry for the descriptor file.
--Jennifer
On Jun 4, 2014, at 8:50 AM, Perry, Aaron @ LSC wrote:
Good Morning All,
I've followed the steps that were outlined in this post (http://gradsusr.org/pipermail/gradsusr/2011-November/031529.html) a while back regarding getting SREF probabilities to plot in GrADS. I keep getting the "Cannot contour grid/data request" error and I was wondering if someone might know of solution.
Onward and Upward,
Aaron D. Perry
Atmospheric Sciences
Lyndon State College
Class of 2016
Boston, MA
Twitter: @AaronPerryLSC<https://twitter.com/AaronPerryLSC>
Mobile: 617-780-4312
_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr
--
Jennifer M. Adams
Center for Ocean-Land-Atmosphere Studies (COLA)
111 Research Hall, Mail Stop 2B3
George Mason University
4400 University Drive
Fairfax, VA 22030
_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr
--
Jennifer M. Adams
Center for Ocean-Land-Atmosphere Studies (COLA)
111 Research Hall, Mail Stop 2B3
George Mason University
4400 University Drive
Fairfax, VA 22030
_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr
--
Jennifer M. Adams
Center for Ocean-Land-Atmosphere Studies (COLA)
111 Research Hall, Mail Stop 2B3
George Mason University
4400 University Drive
Fairfax, VA 22030
<sref_prob.20140604.15z.ctl><sref_qpf_prob.gs>_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr
--
Jennifer M. Adams
Center for Ocean-Land-Atmosphere Studies (COLA)
111 Research Hall, Mail Stop 2B3
George Mason University
4400 University Drive
Fairfax, VA 22030
_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr
--
Jennifer M. Adams
Center for Ocean-Land-Atmosphere Studies (COLA)
111 Research Hall, Mail Stop 2B3
George Mason University
4400 University Drive
Fairfax, VA 22030
_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr
--
Jennifer M. Adams
Center for Ocean-Land-Atmosphere Studies (COLA)
111 Research Hall, Mail Stop 2B3
George Mason University
4400 University Drive
Fairfax, VA 22030
_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr
--
Jennifer M. Adams
Center for Ocean-Land-Atmosphere Studies (COLA)
111 Research Hall, Mail Stop 2B3
George Mason University
4400 University Drive
Fairfax, VA 22030
_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr
--
Jennifer M. Adams
Center for Ocean-Land-Atmosphere Studies (COLA)
111 Research Hall, Mail Stop 2B3
George Mason University
4400 University Drive
Fairfax, VA 22030
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20140606/05bb2cb9/attachment-0001.html
More information about the gradsusr
mailing list