[gradsusr] Impending shutdown of GDS server nomads.ncdc.noaa.gov/dods

Tue Jan 17 14:03:18 EST 2017

Hi, Dan —
This is very unfortunate news!

We have not received any new funding for GDS modifications. Updating the tomcat version is probably not a difficult task for someone who is handy with apache, tomcat, java, and GDS functionality. Unfortunately, we simply do not have the resources to dedicate to that project at COLA right now.

I know that NASA GSFC has managed to find a way to implement their operational GDS in an https environment, perhaps you could consult with Dana Ostrenga on how it was acomplished. I think they have integrated GDS with apache using port 80, maybe that strategy could bypass the tomcat version issues? I noticed that some of the links in their GDS’s browser interface still have ‘http’ in them, but that is more of a minor user-interface issue… not mission critical.

GrADS will open a url with https, as long as the netcdf library has been built with an ssl-enabled version of the curl library. All of COLA’s 2.1.0 builds have recently been re-packaged with ssl so they will work with https-based data servers. The ssl library is linked dynamically, so it needs to be present on the user’s local server. I just tried the URL you offered below, but got a different error I haven’t seen before:

ga-> sdfopen https://nomads.ncdc.noaa.gov/dods/NCEP_GFS/201612/20161204/gfs_3_20161204_0000_fff
Scanning self-describing file:  https://nomads.ncdc.noaa.gov/dods/NCEP_GFS/201612/20161204/gfs_3_20161204_0000_fff
Duplicate dataset field names: NCEP_GFS/201612/20161204/gfs_3_20161204_0000_fff
context: Dataset { Grid { ARRAY: Float32 absv[time = 61][lev = 26][lat = 181][lon = 360]; <rest of output snipped>.

I don’t know what this means. The "Duplicate dataset field names” message comes from the netcdf library (dapparse.c). I will have to investigate further to figure out what’s going wrong with this particular dataset.

However, this NCDC GDS URL worked just fine with my new build:
https://nomads.ncdc.noaa.gov/dods/NCDC_CLIMATE_SST/185401/18540101/ersst_999_18540101_0000_000

All new GrADS builds going forward will be ssl-enabled.
—Jennifer

On Jan 12, 2017, at 5:01 PM, Dan Swank - NOAA Affiliate <dan.swank at noaa.gov<mailto:dan.swank at noaa.gov>> wrote:

Source: https://www.whitehouse.gov/blog/2015/06/08/https-everywhere-government

All .gov's http sites are supported to have switched already, or be going to https.
It can cause quite some headaches with client connections to services,
   especially with no oppertunity to test client programs.

Not all sites are compliant yet.. but you can expect it may happen anytime.

On Thu, Jan 12, 2017 at 11:25 AM, Anthony Stump <f00dl3a at gmail.com<mailto:f00dl3a at gmail.com>> wrote:
Will this impact data retrieved from http://tgftp.nws.noaa.gov/SL.us008001/ST.opnl/ ?

On Jan 12, 2017 9:53 AM, "Dan Swank - NOAA Affiliate" <dan.swank at noaa.gov<mailto:dan.swank at noaa.gov>> wrote:
This is to inform GrADS Users of https://nomads.ncdc.noaa.gov/dods/ ~

Regretfully,
This service is scheduled to be taken offline by the end of February 2017.

The two primary reasons being the US Government HTTPS mandate
   (Jan 2017) rendering the server largely useless.
   This was essentially"Death from above".

And 2, the changing server enviornments are demand the application
   be run under a recent version of Tomcat 7+.  Preferably delivered by a .war file
   that can be dropped into the container..

If these two issues could be addressed, we may be
   able to resurrect this service.

This message has no direct affiliation with:
http://nomads<http://nomads/>.ncep.noaa.gov/dods-alt<http://noaa.gov/dods-alt>
The NCEP component will remain up,
but is also in some degree of danger, I cannot address further.

NCDC-NOMADS THREDDS services can be used in some cases
to compensate opendap functionality for NCEP model datasets
but generally will not work via. a local GrADS client via. sdfopen.

https://nomads.ncdc.noaa.gov/thredds/catalog.html

Some more detail:

GDS generates links (hrefs) in the form of HTTP://hostname:443/URI<http://hostname:443/URI> instead of HTTPS://hostname/URI<https://hostname/URI>

Even so... Trying sdfopen fails :

Scanning self-describing file:  http://nomads.ncdc.noaa.gov:443/dods/NCEP_GFS/201612/20161204/gfs_3_20161204_0000_fff
syntax error, unexpected SCAN_WORD, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
context: <!DOCTYPE^ HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"><html><head><title>400 Bad Request</title></head><body><h1>Bad Request</h1><p>Your browser sent a request that this server could not understand.<br />Reason: You're speaking plain HTTP to an SSL-enabled server port.<br />Instead use the HTTPS scheme to access this URL, please.<br /><blockquote>Hint: <a href="https://nomads.ncdc.noaa.gov/"><b>https://nomads.ncdc.noaa.gov/</b></a></blockquote></p></body></html>
Error: nc_open failed to open file http://nomads.ncdc.noaa.gov:443/dods/NCEP_GFS/201612/20161204/gfs_3_20161204_0000_fff
NetCDF: Malformed or inaccessible DAP DDS
gadsdf: Couldn't ingest SDF metadata.

An alternate form, also "no dice":

ga-> sdfopen https://nomads.ncdc.noaa.gov/dods/NCEP_GFS/201612/20161204/gfs_3_20161204_0000_fff
Scanning self-describing file:  https://nomads.ncdc.noaa.gov/dods/NCEP_GFS/201612/20161204/gfs_3_20161204_0000_fff
Error: nc_open failed to open file https://nomads.ncdc.noaa.gov/dods/NCEP_GFS/201612/20161204/gfs_3_20161204_0000_fff
NetCDF: I/O failure
gadsdf: Couldn't ingest SDF metadata.

--

Dan Swank
NOMADS System Support, STG Inc. - Federal Government Contractor
NOAA's National Centers for Environmental Information (NCEI)
Data Stewardship Devision (DSD)
151 Patton Ave
Asheville, NC 28801
dan.swank at noaa.gov<mailto:dan.swank at noaa.gov>
Phone: 828-271-4007
Customer Support:  828-271-4800 or http://www.ncdc.noaa.gov/contact

The newly formed NCEI merges the National Climatic Data Center (NCDC), the National Geophysical Data Center (NGDC), and the National Oceanographic Data Center (NODC).

Facebook: http://www.facebook.com/NOAANationalClimaticDataCenter Twitter: @NOAANCDC @NOAAOceanData

======================= Disclaimer ===========
I am not a Federal Employee.
Any opinions expressed in this message are mine personally and
do not represent official positions from STG Inc, ERT, NCEI, or NOAA
============================================

_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr

_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr

--

Dan Swank
NOMADS System Support, STG Inc. - Federal Government Contractor
NOAA's National Centers for Environmental Information (NCEI)
Data Stewardship Devision (DSD)
151 Patton Ave
Asheville, NC 28801
dan.swank at noaa.gov<mailto:dan.swank at noaa.gov>
Phone: 828-271-4007
Customer Support:  828-271-4800 or http://www.ncdc.noaa.gov/contact

The newly formed NCEI merges the National Climatic Data Center (NCDC), the National Geophysical Data Center (NGDC), and the National Oceanographic Data Center (NODC).

Facebook: http://www.facebook.com/NOAANationalClimaticDataCenter Twitter: @NOAANCDC @NOAAOceanData

======================= Disclaimer ===========
I am not a Federal Employee.
Any opinions expressed in this message are mine personally and
do not represent official positions from STG Inc, ERT, NCEI, or NOAA
============================================
_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr

--
Jennifer Miletta Adams
Center for Ocean-Land-Atmosphere Studies (COLA)
George Mason University

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20170117/92348736/attachment.html 