[gradsusr] Problem: extracting a variable from a big file

Thu Nov 25 09:36:06 EST 2010

Hi Mitias

You can use cdo to separate your variables into separate files and use the
GrADS for analysis.

Here is cdo command to do it:

cdo selname,variableName ifilename ofilename

e.g. cdo selname,precip combined.nc precip.nc
(take it just as an example,,,because the variable names might be diffrent
in your file)

Hope it works,,,

Cheers
Rashed Mahmood
Institute of Atmospheric Physics
Chinese Academy of Sciences
Beijing, China.

On Fri, Nov 19, 2010 at 10:40 PM, Matias Mendez <matias.mendezp at gmail.com>wrote:

> Dear Jennifer,
>
> Thank you for your response. Sorry I forgot to provide more
> information.  I'm using GrADS version 2.0.a7 oga.3 on linux (fedora
> 10) and this configuration:
>
> ga-> q config
> Config: v2.0.a7.oga.3 little-endian readline printim grib2 netcdf
> hdf4-sds hdf5 opendap-grids,stn athena geotiff
> Grid Analysis and Display System (GrADS) Version 2.0.a7.oga.3
> Copyright (c) 1988-2009 by Brian Doty and the
> Institute for Global Environment and Society (IGES)
> This program is distributed WITHOUT ANY WARRANTY
> See file COPYRIGHT for more information.
>
> Built Thu Oct 29 17:28:29 EDT 2009 for i686-pc-linux-gnu
>
> This version of GrADS has been configured with the following options:
>  o Built on a LITTLE ENDIAN machine
>  o Command line editing ENABLED
>      http://tiswww.case.edu/php/chet/readline/rltop.html
>  o printim command for image output ENABLED
>      http://www.zlib.net
>      http://www.libpng.org/pub/png/libpng.html
>      http://www.libgd.org/Main_Page
>  o GRIB2 interface ENABLED
>      http://www.ijg.org
>      http://www.ece.uvic.ca/~mdadams/jasper
>      http://www.nco.ncep.noaa.gov/pmb/codes/GRIB2
>      g2clib-1.0.5
>  o NetCDF interface ENABLED
>      http://www.opendap.org
>      libnc-dap 4.0.1-beta3-snapshot2009021712 of Feb 21 2009 14:55:15 $
>  o HDF interface ENABLED
>      http://hdfgroup.org
>      HDF 4.2r3
>      HDF5 1.8.2
>  o Athena Widget GUI ENABLED
>  o OPeNDAP gridded data interface ENABLED
>      http://www.opendap.org
>      libdap 3.7.10
>  o OPeNDAP station data interface ENABLED
>      http://iges.org/grads/gadoc/supplibs.html
>      libgadap 2.0.oga.1
>  o GeoTIFF and KML output ENABLED
>      http://www.libtiff.org
>      http://geotiff.osgeo.org
>
>
> I'll try your suggestion (breaking up output).  Is there any other
> option (fortran, C, etc) to read and write a variable?
>
> Thanks in advance.
>
> Regards.
> Matias
>
>
> On Thu, Nov 18, 2010 at 8:18 AM, Jennifer Adams <jma at cola.iges.org> wrote:
> > Whenever you submit a question, it is impossible to answer without
> knowing
> > two things: the version of GrADS you are running, and the platform you
> are
> > running it on. THIS REALLY MATTERS!
> > Your 'subset' file should be 10242422568 bytes (399*279*31*742*4)
> > If it is not, then you can blame mm5_to_grads.sh -- maybe it doesn't have
> > the capability to write out a 9.5Gb file... Try breaking up your output
> into
> > 742 separate files and template them together with GrADS.
> > The error messages you're getting indicate that GrADS is having trouble
> > moving to the correct position in the file to fulfill your I/O request.
> > Older versions of GrADS did not have the capability to read files > 2GB,
> but
> > who can tell whether this is your problem because you didn't provide
> enough
> > information about your GrADS build.
> > --Jennifer
> >
> > On Nov 18, 2010, at 8:42 AM, Matias Mendez wrote:
> >
> > Dear GrADS users,
> >
> > I have a big MM5 output (~100GB, with 31 sigma-levels, 742 hourly time
> > steps, 23 variables). I have used mm5_to_grads.sh to extract a
> > variable (u-wind). After that I tried to display this variable,
> > everything  was working fine but when I did:
> >
> > d u(t=623)
> >
> > I got the following error mesage:
> >
> > Notice:  Automatic Grid Interpolation Taking Place
> > Low Level I/O Error:  Seek error on data file
> >  Data file name = /home/XXXXX/U_ENE-2006.dat
> > 523 rc=-1 fpos=-2145136144 pfi->fhdr=-1
> >  Error occurred when seeking to byte 9390016
> > Data Request Error:  Error for variable 'u'
> >  Error ocurred at column 1
> > DISPLAY error:  Invalid expression
> >  Expression = u(t=623)
> >
> > What does this mean this message and how can I solve this problem?.
> > Is there another way to extract a variable from a big file?. Any
> > sugesstion is welcome. Thanks in advance.
> >
> > Best regards,
> > Matías
> >
> > P.S.
> > This is the ctl file created by mm5_to_grads.sh
> >
> > dset /home/XXXXX/U_ENE-2006.dat
> > byteswapped
> > title MM5 data
> > undef -9999.
> > pdef  399  279 lcc   24.00 -102.00  200.00  140.00   30.00   60.00
> -102.00
> >   900
> > 0.   9000.
> > xdef  972 linear -121.69  0.0405
> > ydef  571 linear   12.15  0.0405
> > zdef  31 levels
> > 0.99950
> > 0.99800
> > 0.99600
> > 0.99400
> > 0.99200
> > 0.98950
> > 0.98700
> > 0.98500
> > 0.98300
> > 0.98100
> > 0.97500
> > 0.96000
> > 0.94000
> > 0.92000
> > 0.90500
> > 0.89000
> > 0.87000
> > 0.85000
> > 0.83000
> > 0.81000
> > 0.79000
> > 0.76000
> > 0.72000
> > 0.68000
> > 0.64000
> > 0.61000
> > 0.55000
> > 0.45000
> > 0.35000
> > 0.20000
> > 0.05000
> > tdef 742 linear 00:00Z01JAN2006   60MN
> > vars  1
> > u         31 99 u wind (m/s)
> > endvars
> >
> > _______________________________________________
> > gradsusr mailing list
> > gradsusr at gradsusr.org
> > http://gradsusr.org/mailman/listinfo/gradsusr
> >
> > --
> > Jennifer M. Adams
> > IGES/COLA
> > 4041 Powder Mill Road, Suite 302
> > Calverton, MD 20705
> > jma at cola.iges.org
> >
> >
> >
> > _______________________________________________
> > gradsusr mailing list
> > gradsusr at gradsusr.org
> > http://gradsusr.org/mailman/listinfo/gradsusr
> >
> >
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20101125/daad474f/attachment-0001.html 