[gradsusr] Impending shutdown of GDS server nomads.ncdc.noaa.gov/dods

Tue Jan 17 17:09:02 EST 2017

Jennifer:
Our system admins / IT security just don't want to
deal with external web applications that no longer have
direct development support.   Nothing I can do.

Aye ~ I was using a older client lacking ssl support.
This is good to know the newer versions do.
The other issues were mos tikely
a result of bad GFS metafiles (.ctl) on our side or some
kind of cnvgrib converter mangling up the GRIB metadata.

Thanks for your outstanding support over the years
and I hope we can someday bring this service back.

   -Dan

On Tue, Jan 17, 2017 at 2:03 PM, Jennifer M Adams <jadams21 at gmu.edu> wrote:

> Hi, Dan —
> This is very unfortunate news!
>
> We have not received any new funding for GDS modifications. Updating the
> tomcat version is probably not a difficult task for someone who is handy
> with apache, tomcat, java, and GDS functionality. Unfortunately, we simply
> do not have the resources to dedicate to that project at COLA right now.
>
> I know that NASA GSFC has managed to find a way to implement their
> operational GDS in an https environment, perhaps you could consult with
> Dana Ostrenga on how it was acomplished. I think they have integrated GDS
> with apache using port 80, maybe that strategy could bypass the tomcat
> version issues? I noticed that some of the links in their GDS’s browser
> interface still have ‘http’ in them, but that is more of a minor
> user-interface issue… not mission critical.
>
> GrADS will open a url with https, as long as the netcdf library has been
> built with an ssl-enabled version of the curl library. All of COLA’s 2.1.0
> builds have recently been re-packaged with ssl so they will work with
> https-based data servers. The ssl library is linked dynamically, so it
> needs to be present on the user’s local server. I just tried the URL you
> offered below, but got a different error I haven’t seen before:
>
> ga-> sdfopen https://nomads.ncdc.noaa.gov/dods/NCEP_GFS/201612/20161204/
> gfs_3_20161204_0000_fff
> Scanning self-describing file:  https://nomads.ncdc.noaa.gov/
> dods/NCEP_GFS/201612/20161204/gfs_3_20161204_0000_fff
> Duplicate dataset field names: NCEP_GFS/201612/20161204/gfs_
> 3_20161204_0000_fff
> context: Dataset { Grid { ARRAY: Float32 absv[time = 61][lev = 26][lat =
> 181][lon = 360]; <rest of output snipped>.
>
> I don’t know what this means. The "Duplicate dataset field names” message
> comes from the netcdf library (dapparse.c). I will have to investigate
> further to figure out what’s going wrong with this particular dataset.
>
> However, this NCDC GDS URL worked just fine with my new build:
> https://nomads.ncdc.noaa.gov/dods/NCDC_CLIMATE_SST/185401/
> 18540101/ersst_999_18540101_0000_000
>
> All new GrADS builds going forward will be ssl-enabled.
> —Jennifer
>
>
>
>
> On Jan 12, 2017, at 5:01 PM, Dan Swank - NOAA Affiliate <
> dan.swank at noaa.gov> wrote:
>
> Source: https://www.whitehouse.gov/blog/2015/06/08/https-
> everywhere-government
>
>
> All .gov's http sites are supported to have switched already, or be going
> to https.
> It can cause quite some headaches with client connections to services,
>    especially with no oppertunity to test client programs.
>
> Not all sites are compliant yet.. but you can expect it may happen anytime.
>
> On Thu, Jan 12, 2017 at 11:25 AM, Anthony Stump <f00dl3a at gmail.com> wrote:
>
>> Will this impact data retrieved from http://tgftp.nws.noaa.gov
>> /SL.us008001/ST.opnl/ ?
>>
>> On Jan 12, 2017 9:53 AM, "Dan Swank - NOAA Affiliate" <dan.swank at noaa.gov>
>> wrote:
>>
>>> This is to inform GrADS Users of https://nomads.ncdc.noaa.gov/dods/ ~
>>>
>>> Regretfully,
>>> This service is scheduled to be taken offline by the* end of February*
>>> 2017.
>>>
>>> The two primary reasons being the US Government HTTPS mandate
>>>    (Jan 2017) rendering the server largely useless.
>>>    This was essentially"Death from above".
>>>
>>> And 2, the changing server enviornments are demand the application
>>>    be run under a recent version of Tomcat 7+.  Preferably delivered by
>>> a .war file
>>>    that can be dropped into the container..
>>>
>>> If these two issues could be addressed, we may be
>>>    able to resurrect this service.
>>>
>>> This message has *no direct affiliation *with:
>>> http://nomads.*ncep*.noaa.gov/dods-alt
>>> The NCEP component will remain up,
>>> but is also in some degree of danger, I cannot address further.
>>>
>>> NCDC-NOMADS THREDDS services can be used in some cases
>>> to compensate opendap functionality for NCEP model datasets
>>> but generally will not work via. a local GrADS client via. sdfopen.
>>>
>>> https://nomads.ncdc.noaa.gov/thredds/catalog.html
>>>
>>>
>>>
>>>
>>>
>>> Some more detail:
>>>
>>> GDS generates links (hrefs) in the form of HTTP://hostname:443/URI
>>> <http://hostname:443/URI> instead of HTTPS://hostname/URI
>>> <https://hostname/URI>
>>>
>>> Even so... Trying sdfopen fails :
>>>
>>>
>>> Scanning self-describing file:  http://nomads.ncdc.noaa.gov:44
>>> 3/dods/NCEP_GFS/201612/20161204/gfs_3_20161204_0000_fff
>>> syntax error, unexpected SCAN_WORD, expecting SCAN_ATTR or SCAN_DATASET
>>> or SCAN_ERROR
>>> context: <!DOCTYPE^ HTML PUBLIC "-//IETF//DTD HTML
>>> 2.0//EN"><html><head><title>400 Bad Request</title></head><body><h1>Bad
>>> Request</h1><p>Your browser sent a request that this server could not
>>> understand.<br />Reason: You're speaking plain HTTP to an SSL-enabled
>>> server port.<br />Instead use the HTTPS scheme to access this URL,
>>> please.<br /><blockquote>Hint: <a href="https://nomads.ncdc.noaa.gov/
>>> "><b>https://nomads.ncdc.noaa.gov/</b></a></blockquote>
>>> </p></body></html>
>>> Error: nc_open failed to open file http://nomads.ncdc.noaa.gov:44
>>> 3/dods/NCEP_GFS/201612/20161204/gfs_3_20161204_0000_fff
>>> NetCDF: Malformed or inaccessible DAP DDS
>>> gadsdf: Couldn't ingest SDF metadata.
>>>
>>> An alternate form, also "no dice":
>>>
>>> ga-> sdfopen https://nomads.ncdc.noaa.gov/d
>>> ods/NCEP_GFS/201612/20161204/gfs_3_20161204_0000_fff
>>> Scanning self-describing file:  https://nomads.ncdc.noaa.gov/d
>>> ods/NCEP_GFS/201612/20161204/gfs_3_20161204_0000_fff
>>> Error: nc_open failed to open file https://nomads.ncdc.noaa.gov/d
>>> ods/NCEP_GFS/201612/20161204/gfs_3_20161204_0000_fff
>>> NetCDF: I/O failure
>>> gadsdf: Couldn't ingest SDF metadata.
>>>
>>>
>>> --
>>>
>>> Dan Swank
>>> NOMADS System Support, STG Inc. - Federal Government Contractor
>>> NOAA's National Centers for Environmental Information (NCEI)
>>> Data Stewardship Devision (DSD)
>>> 151 Patton Ave
>>> Asheville, NC 28801
>>> dan.swank at noaa.gov
>>> Phone: 828-271-4007
>>> Customer Support:  828-271-4800 or http://www.ncdc.noaa.gov/contact
>>>
>>> The newly formed NCEI merges the National Climatic Data Center (NCDC),
>>> the National Geophysical Data Center (NGDC), and the National Oceanographic
>>> Data Center (NODC).
>>>
>>> Facebook: http://www.facebook.com/NOAANationalClimaticDataCenter
>>> Twitter: @NOAANCDC @NOAAOceanData
>>>
>>> ======================= Disclaimer ===========
>>> I am not a Federal Employee.
>>> Any opinions expressed in this message are mine personally and
>>> do not represent official positions from STG Inc, ERT, NCEI, or NOAA
>>> ============================================
>>>
>>> _______________________________________________
>>> gradsusr mailing list
>>> gradsusr at gradsusr.org
>>> http://gradsusr.org/mailman/listinfo/gradsusr
>>>
>>>
>> _______________________________________________
>> gradsusr mailing list
>> gradsusr at gradsusr.org
>> http://gradsusr.org/mailman/listinfo/gradsusr
>>
>>
>
>
> --
>
> Dan Swank
> NOMADS System Support, STG Inc. - Federal Government Contractor
> NOAA's National Centers for Environmental Information (NCEI)
> Data Stewardship Devision (DSD)
> 151 Patton Ave
> Asheville, NC 28801
> dan.swank at noaa.gov
> Phone: 828-271-4007
> Customer Support:  828-271-4800 or http://www.ncdc.noaa.gov/contact
>
> The newly formed NCEI merges the National Climatic Data Center (NCDC), the
> National Geophysical Data Center (NGDC), and the National Oceanographic
> Data Center (NODC).
>
> Facebook: http://www.facebook.com/NOAANationalClimaticDataCenter Twitter:
> @NOAANCDC @NOAAOceanData
>
> ======================= Disclaimer ===========
> I am not a Federal Employee.
> Any opinions expressed in this message are mine personally and
> do not represent official positions from STG Inc, ERT, NCEI, or NOAA
> ============================================
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
>
>
> --
> Jennifer Miletta Adams
> Center for Ocean-Land-Atmosphere Studies (COLA)
> George Mason University
>
>
>
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
>
>

-- 

Dan Swank
NOMADS System Support, STG Inc. - Federal Government Contractor
NOAA's National Centers for Environmental Information (NCEI)
Data Stewardship Devision (DSD)
151 Patton Ave
Asheville, NC 28801
dan.swank at noaa.gov
Phone: 828-271-4007
Customer Support:  828-271-4800 or http://www.ncdc.noaa.gov/contact

The newly formed NCEI merges the National Climatic Data Center (NCDC), the
National Geophysical Data Center (NGDC), and the National Oceanographic
Data Center (NODC).

Facebook: http://www.facebook.com/NOAANationalClimaticDataCenter Twitter:
@NOAANCDC @NOAAOceanData

======================= Disclaimer ===========
I am not a Federal Employee.
Any opinions expressed in this message are mine personally and
do not represent official positions from STG Inc, ERT, NCEI, or NOAA
============================================
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20170117/188b4c6a/attachment-0001.html 