[gradsusr] Will grads be in spack?
Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]
matthew.thompson at nasa.gov
Tue Jan 8 13:21:28 EST 2019
Wesley,
Spack seems to have 1.25.7 as a Safe Version. Is that sufficient?
Matt
--
Matt Thompson, SSAI, Sr Scientific Programmer/Analyst
NASA GSFC, Global Modeling and Assimilation Office
Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771
Phone: 301-614-6712 Fax: 301-614-6246
http://science.gsfc.nasa.gov/sed/bio/matthew.thompson
From: Wesley Ebisuzaki - NOAA Federal <wesley.ebisuzaki at noaa.gov>
Date: Tuesday, January 8, 2019 at 12:17 PM
To: GrADS Users Forum <gradsusr at gradsusr.org>
Cc: "Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]" <matthew.thompson at nasa.gov>
Subject: Re: [gradsusr] Will grads be in spack?
Matt,
The original jasper 1.900.1 libraries from the univ of victoria had problems and
both the redhat and debian updates fixed the problems. After many years,
the original author started updating jasper and fixing the bugs. The last
time that I looked (1+ years ago), there was a small API change which would
require a change in g2clib. So you will have to check which version of jasper
is already in spark.
G2clib requires libpng 1.2.xx. Not sure 1.5.xx will work. PNG grib files are
rare but not extinct. ESRL uses them for processed radar fields.
Wesley Ebisuzaki
On Tue, Jan 8, 2019 at 11:56 AM Adams, Jennifer M. (GSFC-610.2)[ADNET SYSTEMS INC] <jennifer.m.adams at nasa.gov<mailto:jennifer.m.adams at nasa.gov>> wrote:
Hi, Matt --
1. The shapelib library has been around for a while, and it's very easy to build, if a Spack package doesn't exist, it won't be hard to create.
2. You definitely want to enable opendap in the NetCDF library. The dap/gadap libs in GrADS are for accessing station data (as opposed to grids) via GDS, but I don't think there is anyone out there putting station data behind GDS anymore.
3. GrADS will require some code changes to use udunits2. For now, you'll have to wrangle udunits1 into submission.
4. As for GRIB2, forget ECMWF; GrADS relies on NCEP's g2clib to handle the I/O. In sickness and in health, for better or worse, in good times and bad, etc. To properly support JPEG2000 compression (which is used in some NCEP products), a special version of the jasper library is needed (ftp://cola.gmu.edu/grads/Supplibs/2.2/src/jasper-1.900.1-14ubuntu3.2.debian.tar.gz).
--Jennifer
--
Jennifer Miletta Adams
ADNET Systems, Inc.
NASA/GSFC, Code 610.2
Building 32, Room S159
(301) 614-6070
On 1/8/19, 8:45 AM, "Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]" <matthew.thompson at nasa.gov<mailto:matthew.thompson at nasa.gov>> wrote:
Jennifer, et al,
I just did a quick scan of this page:
http://cola.gmu.edu/grads/gadoc/supplibs2.html
and tried to match with spack packages. Most had pretty good matches
(though they might be called "libX" instead of "X"), and the only
libraries I couldn't find are:
dap
gadap
shapelib
and there are two that I don't think are quite right:
udunits ==> spack has udunits2
g2clib ==> spack has grib-api, eccodes
You said to ignore dap/gadap, so I'm happy with that. Note that in my
testing I'm always building netcdf with the dap variant and it works for
that at least:
[(513) 08:43 AM] $ ncdump -hsc
https://opendap.nccs.nasa.gov/dods/GEOS-5/fp/0.25_deg/fcast/inst1_2d_hwl_Nx/inst1_2d_hwl_Nx.20181222_12
| head -20
netcdf inst1_2d_hwl_Nx {
dimensions:
lat = 721 ;
lon = 1152 ;
time = 123 ;
variables:
double time(time) ;
time:grads_dim = "t" ;
time:grads_mapping = "linear" ;
time:grads_size = "123" ;
shapelib seems pretty standard in its build, so a package for it seems
possible.
udunits was the precursor to udunits2, so a package for it probably
isn't impossible either though we might need to talk with spack about
naming it as "udunits" nowadays usually refers to the udunits2 package
(udunits1?).
You have instructions for g2clib as well so it's probably doable as
well, but ECMWF's (deprecated) grib-api and eccodes are in spack. I'm
guessing, though, that the APIs are not the same?
Matt
On 1/7/19 12:17 PM, Adams, Jennifer M. (GSFC-610.2)[ADNET SYSTEMS INC]
wrote:
> Matt, I see you are taking on the role of GMAO’s Fairy Godmother … Bravo!
>
> The m4 macros used by the GrADS autoconfigure scripts with the --enable-dyn-supplibs option are not perfect. I was rushing to get them into some kind of working state when releasing version 2.2.1 right before my position at COLA ended. Please use the source code at https://github.com/j-m-adams/GrADS, and not the tarball from COLA, which is missing a few scripts. I will try to incorporate the patches from Steve Woodbridge as quickly as I can. Not all libraries use the PKG_CONFIG model, so it is a mix of strategies for finding what is needed. I definitely rely on $LDFLAGS, $CFLAGS, $CPPFLAGS, and $LD_LIBRARY_PATH to find everything. The grib2 library is a good example of one with non-standard features -- that lib is barely supported within NCEP and has zero bells and whistles for simplifying your life; you have to edit the makefile to tweak the build and then copy files manually to install them outside of the local directory. Don't bother with dap/gadap. What else is not available in Spack?
> --Jennifer
>
>
> --
> Jennifer Miletta Adams
> ADNET Systems, Inc.
> NASA/GSFC, Code 610.2
> Building 32, Room S159
> (301) 614-6070
>
> ________________________________________
> From: gradsusr [gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org>] on behalf of Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] [matthew.thompson at nasa.gov<mailto:matthew.thompson at nasa.gov>]
> Sent: Monday, January 07, 2019 10:47 AM
> To: GrADS Users Forum
> Subject: Re: [gradsusr] Will grads be in spack?
>
> All,
>
> I've been trying to slowly move GMAO's GEOS model's Baselibs to spack. This has involved learning how to make new spack packages, which has been interesting (and not quite fully successful yet, to be honest, but I'm learning!). Now, Baselibs started as an offshoot of Supplibs, so I am learning some useful and perhaps relevant lessons.
>
> Looking at how GrADS is built (which I've never done as the binaries work fine for my needs), the main issue I see at present is figuring out how to build GrADS without the SUPPLIBS being in one place. GEOS inherited this as it expects Baselibs to all be under one place as well. With GEOS I have the "luxury" of knowing its very odd non-standard GNU Make build system and could probably hack it to find spack's libraries without too much trouble, but GEOS relies on fewer libraries than GrADS.
>
> GrADS, though, uses configure. Yay for us builders and it looks like some of the libraries seem exposed to configure (--with-netcdf, etc.) but not all. Does anyone out there have experience building GrADS with --enable-dyn-supplibs? If you've documented how to get GrADS to find and use system libraries, spack might be okay with that (with, say, loooong LDFLAGS, LIBS, CPPFLAGS lines). The only other thought would be finding an autotools hacker who could expose all the supplibs with --with-PKG options.
>
> As an aside, it looks like most of Supplibs is in spack, but not all (g2clib, a few others), so eventually a limited GrADS might be doable, with a nigh-complete build if other Supplibs were ported.
>
> Matt
>
>
> --
> Matt Thompson, SSAI, Sr Scientific Programmer/Analyst
> NASA GSFC, Global Modeling and Assimilation Office
> Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771
> Phone: 301-614-6712 Fax: 301-614-6246
> http://science.gsfc.nasa.gov/sed/bio/matthew.thompson
>
> On 1/4/19, 2:08 PM, "gradsusr on behalf of Tom Robinson" <gradsusr-bounces at gradsusr.org<mailto:gradsusr-bounces at gradsusr.org> on behalf of ter at hawaii.edu<mailto:ter at hawaii.edu>> wrote:
>
> I have recently been using spack to install scientific software. It really makes things easy. Is there any plan to add grads to spack?
>
> -Tom
>
> Sent from my iPhone
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
> http://gradsusr.org/mailman/listinfo/gradsusr
>
>
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
> http://gradsusr.org/mailman/listinfo/gradsusr
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
> http://gradsusr.org/mailman/listinfo/gradsusr
>
--
Matt Thompson, SSAI, Sr Scientific Programmer/Analyst
NASA GSFC, Global Modeling and Assimilation Office
Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771
Phone: 301-614-6712 Fax: 301-614-6246
http://science.gsfc.nasa.gov/sed/bio/matthew.thompson
_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gradsusr.org/pipermail/gradsusr/attachments/20190108/50083e41/attachment-0001.html>
More information about the gradsusr
mailing list