[gradsusr] Open GRIB Files over THREDDS Opendap Server

Thu Feb 1 12:33:35 EST 2018

If my sample descriptor works just fine, then the problem is not with GrADS.  Using template strings with URLs is no problem. Just be aware that you’re doing I/O over the internet, and it may be slow. Use define/fwrite/sdfwrite to copy data you’re going to look at more than once to local memory/disk. 

Since ncdump dies midway through parsing the file, then I would check with tech support at NCEI. The netcdf errror message points to line 473 from vardata.c, which contains the following library call:

	NC_CHECK(nc_get_vara(ncid, varid, cor, edg, (void *)valp));

Something is funky with that variable; the data owners should be able to track it down. 
—Jennifer

> On Feb 1, 2018, at 10:40 AM, William Daniel Scheftic <scheftic at email.arizona.edu> wrote:
> 
> Hi, Jennifer
> 
> My responses are below...
> 
>> Hi, William — You need to be using an SSL-enabled build of the curl library, which is used in the opendap protocol in the netcdf library. Rebuild libcurl using the “—with-ssl” option, then relink netcdf, then relink GrADS. (Actually, the middle step may not be required, but I do it anyway just to be sure.) You could also try using my build of the netcdf library. The SSL library is dynamically linked, so you’ll need to have it installed on your system if it is not already there. 
> 
> I have built --with-ssl but perhaps that are some issues with curl, although all of the netcdf tests passed.
> ~/grads/supplibs/src/curl-7.35.0> ./configure --prefix=$SUPPLIBS --with-ssl --without-libidn --enable-static --disable-ldap
> ...
> checking for OpenSSL headers version... unknown - 0x100020afL
> checking for OpenSSL library version... 1.0.1
> checking for OpenSSL headers and library versions matching... no
> configure: WARNING: OpenSSL headers and library versions do not match.
> ...
>  curl version:     7.35.0
>  Host setup:       x86_64-unknown-linux-gnu
>  Install prefix:   /home/bill/grads/supplibs
>  Compiler:         gcc
>  SSL support:      enabled (OpenSSL)
>  SSH support:      no      (--with-libssh2)
>  zlib support:     enabled
>  GSSAPI support:   no      (--with-gssapi)
>  SPNEGO support:   no      (--with-spnego)
>  TLS-SRP support:  enabled
>  resolver:         default (--enable-ares / --enable-threaded-resolver)
>  ipv6 support:     enabled
>  IDN support:      no      (--with-{libidn,winidn})
>  Build libcurl:    Shared=yes, Static=yes
>  Built-in manual:  enabled
>  --libcurl option: enabled (--disable-libcurl-option)
>  Verbose errors:   enabled (--disable-verbose)
>  SSPI support:     no      (--enable-sspi)
>  ca cert bundle:   no
>  ca cert path:     /etc/ssl/certs/
>  LDAP support:     no      (--enable-ldap / --with-ldap-lib / --with-lber-lib)
>  LDAPS support:    no      (--enable-ldaps)
>  RTSP support:     enabled
>  RTMP support:     no      (--with-librtmp)
>  metalink support: no      (--with-libmetalink)
>  HTTP2 support:    disabled (--with-nghttp2)
>  Protocols:        DICT FILE FTP FTPS GOPHER HTTP HTTPS IMAP IMAPS POP3 POP3S RTSP SMTP SMTPS TELNET TFTP
> 
>> The metadata from a THREDDS server is not particularly GrADS-friendly, so you will need to create a descriptor file.  Use your SSL-enabled ncdump executable to get the metadata from the file that you will need to populate the descriptor. 
> 
> I was able to get all of the metadata upon dumping with ncdump, however it did fail on dumping one of variables in the data section.
>        float Cloud_mixing_ratio_isobaric_136_Hour_AverageNanalysis(time, isobaric, lat, lon) ;
> CURL Error: Couldn't connect to server
> curl error details:
> NetCDF: I/O failure
> Location: file vardata.c; line 473
> 
> 
>> I created a sample descriptor with only two variables, one of them is Z-varying:
>> 
>> dset https://secure-web.cisco.com/1IryGopmXFP_EJNwK3RC_5fzKkhyPoWQ3VCv-nQB1KehpG97Yo7xamoL6TzP19rDDvHyU_v-3NwBdA5ZYkEB0u2ttx-JMuyS5gjksDtXtZeOjftsAWGeQlGp51pW-3-pbzkVe7UX0K5W6-bOzH63YnX81gxUUhgXnosqJJL7zGPTqKKU9pIUV-G8fFina_QFt1WwOxr1OhDb1IiAk1c85QLbGt-ogOBUBjKVkj2lp1fJ0MnpeltvIzqKTUZDFTHv8xkUCl9QEn1sZTUHMCaSsm_1spvtHZyS9IeZ7oQbw9_10RYsCi7Eafht-qMNlkXz-IWbv8Ob77LeeoF5jCRF9nyzEEr48-mDbU4QrKivQy7eNC792766ofBBYcXwOO9FSCiNoj_FJTh9CosGNxCWN6l7TAoIKK1h24kkj5qQJXeZ2-bzsqNt2KTVIbD9q16Bf/https%3A%2F%2Fwww.ncei.noaa.gov%2Fthredds%2FdodsC%2Fcfs_v2_anl_mm_pgb%2F2017%2F201712%2Fpgblnl.gdas.201712.grib2
>> dtype netcdf
>> options yrev zrev
>> undef -9.99e8 missing_value
>> xdef 144 linear 0 2.5
>> ydef 73 linear -90 2.5
>> zdef 37 levels 1000 975 950 925 900 875 850 82500
>> 800 775 750 700 650 600 550 500 450 40000
>> 350 300 250 225 200 175 150 125 100 7000
>> 50 30 20 10 7 5 3 2 1
>> tdef 1 linear 06z01dec2017 6hr
>> vars 2
>> Relative_humidity_isobaric_136_Hour_AverageNanalysis=>rh  37  t,z,y,x  rh at pressure levels
>> Relative_humidity_entire_atmosphere_single_layer_136_Hour_AverageNanalysis=>rh1  0  t,y,x  rh entire column
>> endvars
>> 
> 
> Your descriptor file works.  I could plot rh at multiple levels and rh1.
> 
> Is it permitted to use the date templates for dods urls?
> 
>> —Jennifer
> 
> Thanks,
> Bill
> 
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://secure-web.cisco.com/1BXJCJT3xGVUsGcWurSuiv2wh0Wj54Brdmf8rcBBbSGcx7VbY6GizyXNYAVhjzSazKezxpFH6aTE2XvPUhfym-Ou2yHd7bU3AToBaxVscpQZ0DWYe3D1sgJnr9lsjvh5qdecQObm8Rm9MNwzf5mOda88m-vC2TzHFRTJGYNzo3IKMtudJe31PlHDsId64PRU6ZgAphbhYy4OleQE6OLTbTDmUBCxlf2UJzHKTx3Afk4sumaAi_5CM95BG9TzcCk33z2w0mzgGo3UOpqFiHZMkfmxfCnWrfdKNu5-ZgU4E3vg3a4excaYUN45Km9fGeQaMNOkGbP-TDAOswKPoZLvArZNlur1DSBlwi0Yp8hIZKLYbgVUTGw3g4_rUDZcIpbkL_vToOKFFCv16JwjgHxUDWKmhnLK3wYXXDnLp5kLb7DIGEPrDDXliYseCf5DyvsSw/http%3A%2F%2Fgradsusr.org%2Fmailman%2Flistinfo%2Fgradsusr

--
Jennifer Miletta Adams
Center for Ocean-Land-Atmosphere Studies (COLA)
George Mason University