[gradsusr] strange behavior with GrADS 2.0.a8
Huddleston, John
Huddleston at cira.colostate.edu
Wed Sep 1 17:12:12 EDT 2010
Don,
I did some extra reading and solved this problem. FTP sites do not send the headers; so, I edited your nam1hr shell script and changed the http address to a ftp address. The NCEP ftp address is ' ftp://ftp.ncep.noaa.gov/pub/data/nccf/com/nam/prod/...' Don't forget the 'pub' addition.
I also edited Wesley's get_grib perl script and changed http to ftp on line 29.
It will now grab the NAM files correctly.
If you want to keep Wesley's scripts intact, make a copy and edit your shell script to call the new copy.
John Huddleston, PhD
Cooperative Institute for Research in the Atmosphere
-----Original Message-----
From: gradsusr-bounces at gradsusr.org [mailto:gradsusr-bounces at gradsusr.org] On Behalf Of Huddleston, John
Sent: Tuesday, August 31, 2010 1:06 PM
To: GrADS Users Forum
Subject: Re: [gradsusr] strange behavior with GrADS 2.0.a8
Hi Don,
I did some research and it appears that curl is the placing the information in the header.
There is an option using the perl wwwcurl library that suppresses the information, it is CURLOPT_HEADER=0
Since Wesley calls curl from a perl system call and not using the wwwcurl library; I was not successful in figuring out how to call curl and suppress it.
Maybe some curl expert on this list can help out.
I make the Cygwin binaries for Jennifer; although, I believe that Arlindo has the OpenGrads version.
See http://vista.cira.colostate.edu/nco/grads-2.0.a8-i686-pc-cygwin.tar.gz for simply the binaries. You must have Cygwin installed.
I changed to /usr/local and un-tarred them into the bin directory.
John
John Huddleston, PhD
Cooperative Institute for Research in the Atmosphere
-----Original Message-----
From: gradsusr-bounces at gradsusr.org [mailto:gradsusr-bounces at gradsusr.org] On Behalf Of Don.VanDyke at noaa.gov
Sent: Tuesday, August 31, 2010 12:43 PM
To: GrADS Users Forum
Subject: Re: [gradsusr] strange behavior with GrADS 2.0.a8
Hi Jennifer/John,
Interesting, thanks for the replies. I would have never figured that one out in a million years! I didn't even know a unix command called 'od' existed, so thanks for the tip there as well. Interestingly, all of those junk bytes are coming from the original NCEP grib2 file, not my nam1hr.sh script. The place to download them for testing is at http://www.ftp.ncep.noaa.gov/data/nccf/com/nam/prod/ . They're located in the nam.yyyymmdd folders and have the exact same naming convention as my files. I downloaded a full file to try to just do a simple plot of tmp2m, and it gave me the same error with grads2a8. This would also seem to explain why I was only having trouble with the NAM files and not GFS files, because downloading a GFS file and using that 'od' command to look at it reveals no "junk" bytes before the "GRIB" string. By the way, I'm using the precompiled CentOS5.4-x86_64 binaries on a Red Hat Enterprise 5 computer at work. However, I have cygwin installed at hom
e and use the precompiled grads2a71 binaries for cygwin, but I would really love if someone donated their GrADS 2.0.a8 precompiled cygwin binaries if they have them. :-)
To be honest, I have no idea how to remove those junk bytes out of the grib2 files, but if this can be fixed on the next release, then I'll probably just wait for that. Thanks again for all the help!
Don Van Dyke
NWS Tallahassee, FL
----- Original Message -----
From: Jennifer Adams <jma at cola.iges.org>
Date: Tuesday, August 31, 2010 10:43 am
Subject: Re: [gradsusr] strange behavior with GrADS 2.0.a8
To: GrADS Users Forum <gradsusr at gradsusr.org>
> Hi, Don --
> This one was tricky! Thank you for providing all the info I needed to
>
> diagnose the problem.
>
> First of all, your grib files have 112 bytes of junk at the beginning
>
> of the file. One of my favorite debugging tools, the unix command 'od'
>
> gives me this:
>
> > od -c -N 116 nam.t00z.awphys01.grb2.tm00
> 0000000 \r \n - - 4 c 7 c 8 0 6 5 3 d e
> 3
> 0000020 \r \n C o n t e n t - t y p e :
> 0000040 t e x t / p l a i n ; c h a
> r
> 0000060 s e t = i s o - 8 8 5 9 - 1 \r
> \n
> 0000100 C o n t e n t - r a n g e :
> b
> 0000120 y t e s 6 6 9 5 3 5 - 1 0 4
> 7
> 0000140 9 1 0 / 2 6 7 7 4 7 2 2 \r \n \r
> \n
> 0000160 G R I B
>
> In other words, your GRIB file has the following text prefixed:
>
> --4c7c80653de3
> Content-type: text/plain; charset=iso-8859-1
> Content-range: bytes 669535-1047910/26774722
>
> Check your nam1hr.sh script and see if you can figure out how to get
>
> rid of that.
>
> Secondly, as far as GrADS is concerned, the GRIB interface changed
> between a7 and a8 in order to handle files > 2Gb. This meant we had to
>
> stop using a routine in the GRIB2 library (seekgb) because it does not
>
> use 8-byte integers for file position offsets. The work normally done
>
> by seekgb is coded directly in the GrADS I/O layer for version a8.
> Unfortunately, one thing that seekgb does that the new code in a8 does
>
> not is check for junk bytes before the 4-character string "GRIB" at
> the beginning of the record. I will fix this for the new release.
>
> In the meanwhile, try to avoid contaminating your GRIB file with junk.
>
> And thanks for the bug report!
> --Jennifer
>
>
>
>
> On Aug 31, 2010, at 1:04 AM, Don.VanDyke at noaa.gov wrote:
>
> > Hello,
> >
> > I am noticing some strange behavior with GrADS 2.0.a8 when I try to
>
> > write a netCDF file from the following grib2 files at this link.
> > I created those grib2 files with the nam1hr.sh script at that link
>
> > following the example shown at
> > . Basically, they were created from standard NCEP 12 km NAM grib2
>
> > files, but they only have certain variables contained within them in
>
> > order to save bandwidth. My goal is to write those variables to
> > netCDF files, but my writenc.gs example script at
> > does not work with grads2a8. It produces an error message shown at
>
> > .
> >
> > The interesting thing is that everything works perfectly using the
>
> > previous version of GrADS (2.0.a7.1). However, I would like to use
>
> > the -flt option with the sdfwrite command, and that's only available
>
> > for 2.0.a8 and higher. Can anybody explain what I'm doing wrong or
>
> > if this is a bug introduced with grads2a8? Thanks!
> >
> > Don Van Dyke
> > NWS Tallahassee, FL
> > _______________________________________________
> > gradsusr mailing list
> > gradsusr at gradsusr.org
> >
>
> --
> Jennifer M. Adams
> IGES/COLA
> 4041 Powder Mill Road, Suite 302
> Calverton, MD 20705
> jma at cola.iges.org
>
>
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
>
_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org
http://gradsusr.org/mailman/listinfo/gradsusr
_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org
http://gradsusr.org/mailman/listinfo/gradsusr
More information about the gradsusr
mailing list