[gradsusr] Running GrADS (Specifically g2grb) in Parallel

[Wesley Ebisuzaki - NOAA Federal]

Josh,

   Assuming the problem is not with your scripts, then
what could go wrong with GrADS?

1)  Many years ago, we had a busy web server running grads
to produce user-defined plots.  Therefore at that time, a single
user was running many grads sessions at once with no problems.
Therefore, it may be a new problem or a problem with your system.

a)  Are your scripts running GrADS in "batch" mode?
There may be some system limitation with too many
display windows.  (grads -b)

b) Are you running out of memory?

The script "g2grb.gs" is an easy-to-use script to convert
a gridded field into grib2.  It is slow because it does
repeated generation of the template.  Sounds
like you trying to speed up the conversion into grib by
running your job into many pieces at one time.  A
conversion of a binary/ieee file into grib2 can be done
outside of grads.

Version 1

step 1: generate template file  (see g2grb.gs)
step 2: generate ieee/bin stream (no header/trailer) or sequential (f77
   header trailer) data file.
step 3: generate metadata file (one line per field)  (see your script)
step 4: run wgrib2 with template file, ieee file and metadata file
            and you have a grib file.

Version 2
  use wgrib2api
    http://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/wgrib2api.html
  The fortran version is good to go.  The C version is 90% ready to release.

Wesley



On Wed, Jul 11, 2018 at 6:02 PM, Josh Cohen <development at innovationwx.com>
wrote:

> Wesley,
>
> I tried doing that but to no avail.  My guess is something is going on
> with my installation/internals of GrADS. I come from more of a
> programming/computer science background, so when I saw that my code had no
> race conditions but still broke when running in parallel, I immediately
> thought it must have been something I wasn’t familiar with GrADS (as GrADS
> from my point of view is not intuitive at all).  I appreciate your advice.
>
> Thank you,
>
> Josh
>
> Josh,
>
>   Problems in running jobs in parallel usually are caused by a file that is
> being "shared" when it shouldn't be.
> g2grb.gs uses temporary files but there names are based on the output
> file which is different in the parallel runs.
>
> wgrib2 doesn't make temporary files and doesn't lock files.  So I don't
> know what is causing your problem.   My suggestion is turn each of the
> parallel jobs in its own directory.
>
> Wesley
>
>
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gradsusr.org/pipermail/gradsusr/attachments/20180712/8aba9671/attachment.html>