lats4d

[Brian Doty]

Mike, the new Version 2 of GrADS is being released in alpha version
and incomplete so that people would have access to the new GRIB2
support.   With no funding for many years and key personnel working
limited part time hours we do not have the resources to support old
code and also move forward to provide essential new functionality for
present and future research needs.  GrADS development first started
in the late 1980s and it has taken us several years of work to clean
up the 15 years of accumulated mods and patches on top of the
original code base in order to come out with a new code base so we
could move forward.   I have made every effort over the years to
ensure upward compatibility and maintain stability.  There is no way
that a 20 year old code base can be patched and hacked for another 20
years or even 5 years and maintain stability and still support the
100TB data sets you talk about and the other projects probably in the
pipeline.   As new versions of data formats appear -- grib2, hdf5,
new netcdf on the way, etc ---  we have no choice.  We cannot keep
using old libraries and maintain them ourselves.  We cannot patch and
hack old code that we didn't write to begin with and which in some
cases has restrictive copyrights.

As I say, version 2 as it stands now is incomplete.   Key areas of
functionality which need to be re-implemented have not yet been
done.  Other major parts of grads have not yet been upgraded, such as
the graphics package.   As Jennifer said in her original email, the
output of various data formats is high on the list of things yet to
be done...   Brian

On Jan 29, 2008, at 4:34 PM, Mike Bosilovich wrote:

>
> I have to admit an ignorance to some of the inner workings of Grads
> development. But I am quite surprised to hear that lats4d is not
> presently supported in Grads 2.
>
> I have been using Grads for 10 years now (I can still recall the
> relief after years of ncargf77 programming :-) Of course, I still
> write code when appropriate, but grads and lats4d have been the
> main tools in my work. Lats4d fills a critical void. By pointing it
> at any grads formatted file (netcdf, hdf, binary or grib), I can
> reformat the data to what ever a colleague may need. I can also,
> easily, no, effortlessly subset variables space or time. Coupled
> with a call to regrid, this subsetting utility is beyond compare.
> In a c shell, it can rip through huge data files with simplicity. I
> hope this does not sound like exaggeration, lats4d is the strongest
> data tool I have used.
>
> In our office, we have just begun production of a new reanalysis
> data product. While it will take some time to complete, we are
> beginning to develop examples on how users can access and analyze
> the data. Comparison with other reanalyses is obvious and will be
> in high demand. To accomplish that, they will need to regrid our
> data sets to the existing coarser reanalysis data sets. Or, they
> may need to change the format out of our native HDF. The easiest
> way to explain to others how to do this is with lats4d.
>
> There will be 100Tb of reanalysis data available through a GDS.
> With Lats4d and gradsdods (or gradsdap), this would be much more
> accessible. Users, with some examples, will access the data through
> online capabilities, rather than bulk downloading of the native HDF
> files (a throttle may be needed for access if that is the preferred
> by users). In addition, a plan is being prepared to develop a DVD,
> similar to the NCEP reanalyses CD and NARR DVD. Personally, I would
> like to see a flavor of grads and lats4d included therein
> (admittedly, I have not gotten to discuss this with the grads
> developers yet).
>
> It doesn't stop there. At AMS last week, NCEP and NCDC held a town
> hall meeting to discuss their plans for the next reanalyses. They
> expect to have nearly 1000Tb of data from three different
> reanalyses, and their production has also started. I don't see
> storage as a barrier, but bandwidth is. Too many users making too
> many big requests will limit accessibility. Lats4d access to their
> GDS will become an important function.
>
> Again, I have to admit I do not know the extent of the issues here.
> By necessity, I will have to use versions of grads that include
> lats4d not only in my work, but as I show other how to use our
> data. I felt the need to speak up and I hope that these issues will
> be considered.
>
> A sincere grads and lats4d user,
>
> Mike Bosilovich
>