[gradsusr] Problem: extracting a variable from a big file

Fri Nov 19 09:40:04 EST 2010

Dear Jennifer,

Thank you for your response. Sorry I forgot to provide more
information.  I'm using GrADS version 2.0.a7 oga.3 on linux (fedora
10) and this configuration:

ga-> q config
Config: v2.0.a7.oga.3 little-endian readline printim grib2 netcdf
hdf4-sds hdf5 opendap-grids,stn athena geotiff
Grid Analysis and Display System (GrADS) Version 2.0.a7.oga.3
Copyright (c) 1988-2009 by Brian Doty and the
Institute for Global Environment and Society (IGES)
This program is distributed WITHOUT ANY WARRANTY
See file COPYRIGHT for more information.

Built Thu Oct 29 17:28:29 EDT 2009 for i686-pc-linux-gnu

This version of GrADS has been configured with the following options:
  o Built on a LITTLE ENDIAN machine
  o Command line editing ENABLED
      http://tiswww.case.edu/php/chet/readline/rltop.html
  o printim command for image output ENABLED
      http://www.zlib.net
      http://www.libpng.org/pub/png/libpng.html
      http://www.libgd.org/Main_Page
  o GRIB2 interface ENABLED
      http://www.ijg.org
      http://www.ece.uvic.ca/~mdadams/jasper
      http://www.nco.ncep.noaa.gov/pmb/codes/GRIB2
      g2clib-1.0.5
  o NetCDF interface ENABLED
      http://www.opendap.org
      libnc-dap 4.0.1-beta3-snapshot2009021712 of Feb 21 2009 14:55:15 $
  o HDF interface ENABLED
      http://hdfgroup.org
      HDF 4.2r3
      HDF5 1.8.2
  o Athena Widget GUI ENABLED
  o OPeNDAP gridded data interface ENABLED
      http://www.opendap.org
      libdap 3.7.10
  o OPeNDAP station data interface ENABLED
      http://iges.org/grads/gadoc/supplibs.html
      libgadap 2.0.oga.1
  o GeoTIFF and KML output ENABLED
      http://www.libtiff.org
      http://geotiff.osgeo.org

I'll try your suggestion (breaking up output).  Is there any other
option (fortran, C, etc) to read and write a variable?

Thanks in advance.

Regards.
Matias

On Thu, Nov 18, 2010 at 8:18 AM, Jennifer Adams <jma at cola.iges.org> wrote:
> Whenever you submit a question, it is impossible to answer without knowing
> two things: the version of GrADS you are running, and the platform you are
> running it on. THIS REALLY MATTERS!
> Your 'subset' file should be 10242422568 bytes (399*279*31*742*4)
> If it is not, then you can blame mm5_to_grads.sh -- maybe it doesn't have
> the capability to write out a 9.5Gb file... Try breaking up your output into
> 742 separate files and template them together with GrADS.
> The error messages you're getting indicate that GrADS is having trouble
> moving to the correct position in the file to fulfill your I/O request.
> Older versions of GrADS did not have the capability to read files > 2GB, but
> who can tell whether this is your problem because you didn't provide enough
> information about your GrADS build.
> --Jennifer
>
> On Nov 18, 2010, at 8:42 AM, Matias Mendez wrote:
>
> Dear GrADS users,
>
> I have a big MM5 output (~100GB, with 31 sigma-levels, 742 hourly time
> steps, 23 variables). I have used mm5_to_grads.sh to extract a
> variable (u-wind). After that I tried to display this variable,
> everything  was working fine but when I did:
>
> d u(t=623)
>
> I got the following error mesage:
>
> Notice:  Automatic Grid Interpolation Taking Place
> Low Level I/O Error:  Seek error on data file
>  Data file name = /home/XXXXX/U_ENE-2006.dat
> 523 rc=-1 fpos=-2145136144 pfi->fhdr=-1
>  Error occurred when seeking to byte 9390016
> Data Request Error:  Error for variable 'u'
>  Error ocurred at column 1
> DISPLAY error:  Invalid expression
>  Expression = u(t=623)
>
> What does this mean this message and how can I solve this problem?.
> Is there another way to extract a variable from a big file?. Any
> sugesstion is welcome. Thanks in advance.
>
> Best regards,
> Matías
>
> P.S.
> This is the ctl file created by mm5_to_grads.sh
>
> dset /home/XXXXX/U_ENE-2006.dat
> byteswapped
> title MM5 data
> undef -9999.
> pdef  399  279 lcc   24.00 -102.00  200.00  140.00   30.00   60.00 -102.00
>   900
> 0.   9000.
> xdef  972 linear -121.69  0.0405
> ydef  571 linear   12.15  0.0405
> zdef  31 levels
> 0.99950
> 0.99800
> 0.99600
> 0.99400
> 0.99200
> 0.98950
> 0.98700
> 0.98500
> 0.98300
> 0.98100
> 0.97500
> 0.96000
> 0.94000
> 0.92000
> 0.90500
> 0.89000
> 0.87000
> 0.85000
> 0.83000
> 0.81000
> 0.79000
> 0.76000
> 0.72000
> 0.68000
> 0.64000
> 0.61000
> 0.55000
> 0.45000
> 0.35000
> 0.20000
> 0.05000
> tdef 742 linear 00:00Z01JAN2006   60MN
> vars  1
> u         31 99 u wind (m/s)
> endvars
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
>
> --
> Jennifer M. Adams
> IGES/COLA
> 4041 Powder Mill Road, Suite 302
> Calverton, MD 20705
> jma at cola.iges.org
>
>
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
>
>