[gradsusr] Low Level I/O Error: Seek error on data file [GrADS] -

Mingxuan Chen chen at atmos.colostate.edu
Tue Sep 24 14:38:49 EDT 2013


ok
On Sep 24, 2013, at 12:02 PM, Leopoldo Alvarez <lap.bolsa at gmail.com> wrote:

> Jennifer, 
> 
> I'm going to make climatic modeling (>30 years) and I can not be cutting files for years, buff ... it should work for powerpc with appropriate arguments ... Can be compiled with gfortran compiler?
> This morning I opened a file of 7.7 Gb...
> 
> Leopoldo
> 
> 
> -------- Mensaje original --------
> Asunto:	[gradsusr] Low Level I/O Error: Seek error on data file [GrADS] -
> Fecha:	Tue, 24 Sep 2013 15:02:30 +0100
> De:	Leopoldo Alvarez <lap.bolsa at gmail.com>
> Para:	gradsusr at gradsusr.org
> 
> Dear Jennifer, 
> 
> Now I'm more confused because I was able to open a file of 7.7 Gb
> 8173088 - 7.8G 2013-09-20 09:57 test.dat
> Why I can not open the other file ???
> 
> My full GrADS descriptor file is (similar) ->
> 
> dset ^test.dat
> options big_endian
> title MM5 data
> undef -9999.
> pdef   60   60 lcc   32.63 -112.54    1.00    1.00   30.00   60.00 -110.34   6000.   6000.
> xdef  149 linear -112.65  0.0270
> ydef  123 linear   32.63  0.0270
> zdef  25 levels
>  0.99689
>  0.99191
>  0.98944
>  0.98818
>  0.97377
>  0.94500
>  0.91000
>  0.87000
>  0.82500
>  0.77500
>  0.72500
>  0.67500
>  0.62500
>  0.57500
>  0.52500
>  0.47500
>  0.42500
>  0.37500
>  0.32500
>  0.27500
>  0.22500
>  0.17500
>  0.12500
>  0.07500
>  0.02500
> tdef 1036 linear 00:00Z02JAN2012   60MN
> vars 33
> rc       0 99 accum conv pcn (cm)
> rn       0 99 accum non-c pcn (cm)
> ter      0 99 ter elevation (m)
> xlat     0 99 cross lat (degree)
> xlon     0 99 cross lon (degree)
> lu       0 99 land use
> t2m      0 99 2 m temperature (K)
> q2m      0 99 2m mix ratio (kg/kg)
> u10      0 99 10 m u wind (m/sec)
> v10      0 99 10 m v wind (m/sec)
> pslv     0 99 sea level prs (mb)
> u         25 99 u wind (m/s)
> v         25 99 v wind (m/s)
> w         25 99 vertical vel (m/s)
> pp        25 99 prs pert (Pa)
> t         25 99 temperature (C)
> q         25 99 mixing ratio (kg/kg)
> clw       25 99 cloud water (kg/kg)
> rnw       25 99 rain water (kg/kg)
> rtnd      25 99 rad tend (K/day)
> z         25 99 height AGL (m)
> h         25 99 geopot height (m)
> td        25 99 dewpoint temp (C)
> rh        25 99 rel humidity (%)
> th        25 99 potential temp (K)
> the       25 99 theta-e (K)
> prs       25 99 pressure (Pa)
> vor       25 99 vorticity (s-1)
> pv        25 99 potential vort (pvu)
> dbz       25 99 reflectivity (dbz)
> div       25 99 divergence (s-1)
> dir       25 99 dir (degrees)
> tadv      25 99 temp advection (K/s)
> endvars
> 
> Leopoldo
> 
> 
> 
> 
> 
> 
> 
> Dear Jennifer,
> 
> I don´t know how to fix it... :-( mmmh
> 
> ga-> q config
> Config: v2.0.2 big-endian readline
> Grid Analysis and Display System (GrADS) Version 2.0.2
> Copyright (c) 1988-2011 by Brian Doty and the
> Institute for Global Environment and Society (IGES)
> This program is distributed WITHOUT ANY WARRANTY
> See file COPYRIGHT for more information.
> 
> Built Thu Sep  5 12:08:35 BST 2013 for powerpc64-unknown-linux-gnu
> 
> This version of GrADS has been configured with the following options:
>    o Built on a BIG ENDIAN machine
>    o Athena Widget GUI DISABLED
>    o Command line editing ENABLED
> http://tiswww.case.edu/php/chet/readline/rltop.html
>    o printim command DISABLED
>    o GRIB2 interface DISABLED
>    o NetCDF interface DISABLED
>    o OPeNDAP gridded data interface DISABLED
>    o OPeNDAP station data interface DISABLED
>    o HDF4 and HDF5 interfaces DISABLED
>    o GeoTIFF and KML/TIFF output DISABLED
>    o KML contour output ENABLED
>    o Shapefile interface DISABLED
> 
> For additional information please consult http://iges.org/grads
> 
> Leopoldo ÁLVAREZ
> 
> 
> 
> 
> 
> Here the configuration to compile for power pc that I used ->
>              grep Linux .tmpfile ; \
>           if [ $$? = 0 ]; then echo "Compiling for PowerPC" 
> ;               \
>                   ( $(CD) src ; $(MAKE) all                               \
>                   "RM             = $(RM)"        "RM_LIST        =
> $(RM_LIST)"   \
>                   "LN             = $(LN)"        "MACH           =
> IBM"          \
>                   "MAKE           = $(MAKE)"      "CPP            =
> /lib/cpp"     \
>                   "CPPFLAGS       = -I. -C -P -DIBM -DBIT32
> -traditional"        \
>                   "FC             = xlf90 "  "FCFLAGS        = -qfree=f90
> -qlanglvl=90pure -I../util"   \
>                   "LDOPTIONS      = "             "CFLAGS         = -I.
> -DUNDERSCORE"             \
>                   "LOCAL_LIBRARIES= " ) ; \
> 
> Leopoldo
> 
> 
> El 23/09/13 21:35, Leopoldo Alvarez escribió:
>  > Dear Jennifer and James,
>  >
>  > The size file that I generated with my mm5_to_grads.csh  is ->
>  > [login1 17] /gpfs/projects/XXXXX/XXXXXX/mesoescala/MM5toGrADS > du -k
>  > test.dat
>  > *68926752* 2013-09-22 11:11 test.dat
>  >
>  > I think it can be that grad.exe is bad compiled?  Is this file
>  > (test.dat) bad generated? Is it really a problem of file size? how I
>  > fix it?
>  > >>>> I'm on a machine that works -> uname -a
>  > >>>> Linux login1 2.6.16.53-0.16-ppc64 #1 SMP Tue Oct 2 16:57:49 UTC
>  > 2007 ppc64 ppc64 ppc64 GNU/Linux
>  >
>  > > When you are using PDEF, the xsize and ysize come from the first two
>  > args in the PDEF entry. Here is my estimate of what the file size
>  > should be: sfc_vars * x * y * t *  4 bytes = 11* 60 * 60 * 1036 * 4 =
>  > 164102400
>  > > plus
>  > > z-varying-vars * z * x * y * t * 4 bytes = 22 * 25 * 60 *60 * 1036 *
>  > 4 = 8205120000
>  > > equals 8369222400 bytes.
>  >
>  > I paste another GrADS descriptor file, sorry.
>  > I can see the data well, until the time of step 1064 (set t 1064, then
>  > 'd t' it`s ok), but when i exceeded from 1064 time of step gives me
>  > that error ...
>  >
>  > /ga-> set t *1065*
>  > Time values set: 2012:2:15:8 2012:2:15:8
>  > ga-> d t
>  > Notice:  Automatic Grid Interpolation Taking Place
>  > Low Level I/O Error:  Seek error on data file
>  >   Data file name = test.dat
>  > 650 rc=-1 fpos=-2145713296 pfi->fhdr=0
>  >   Error occurred when seeking to byte 7081408
>  > Data Request Error:  Error for variable 't'
>  >   Error ocurred at column 1
>  > DISPLAY error:  Invalid expression
>  >   Expression = t/
>  >
>  > My full GrADS descriptor file is OK to *8737 *time values (one year) ->
>  >
>  > dset ^test.dat
>  > options big_endian
>  > title MM5 data
>  > undef -9999.
>  > pdef   60   60 lcc   32.63 -112.54    1.00    1.00   30.00 60.00
>  > -110.34   6000.   6000.
>  > xdef  149 linear -112.65  0.0270
>  > ydef  123 linear   32.63  0.0270
>  > zdef  25 levels
>  >  0.99689
>  >  0.99191
>  >  0.98944
>  >  0.98818
>  >  0.97377
>  >  0.94500
>  >  0.91000
>  >  0.87000
>  >  0.82500
>  >  0.77500
>  >  0.72500
>  >  0.67500
>  >  0.62500
>  >  0.57500
>  >  0.52500
>  >  0.47500
>  >  0.42500
>  >  0.37500
>  >  0.32500
>  >  0.27500
>  >  0.22500
>  >  0.17500
>  >  0.12500
>  >  0.07500
>  >  0.02500
>  > tdef*8737 *linear 00:00Z02JAN2012   60MN
>  > vars 33
>  > rc       0 99 accum conv pcn (cm)
>  > rn       0 99 accum non-c pcn (cm)
>  > ter      0 99 ter elevation (m)
>  > xlat     0 99 cross lat (degree)
>  > xlon     0 99 cross lon (degree)
>  > lu       0 99 land use
>  > t2m      0 99 2 m temperature (K)
>  > q2m      0 99 2m mix ratio (kg/kg)
>  > u10      0 99 10 m u wind (m/sec)
>  > v10      0 99 10 m v wind (m/sec)
>  > pslv     0 99 sea level prs (mb)
>  > u         25 99 u wind (m/s)
>  > v         25 99 v wind (m/s)
>  > w         25 99 vertical vel (m/s)
>  > pp        25 99 prs pert (Pa)
>  > t         25 99 temperature (C)
>  > q         25 99 mixing ratio (kg/kg)
>  > clw       25 99 cloud water (kg/kg)
>  > rnw       25 99 rain water (kg/kg)
>  > rtnd      25 99 rad tend (K/day)
>  > z         25 99 height AGL (m)
>  > h         25 99 geopot height (m)
>  > td        25 99 dewpoint temp (C)
>  > rh        25 99 rel humidity (%)
>  > th        25 99 potential temp (K)
>  > the       25 99 theta-e (K)
>  > prs       25 99 pressure (Pa)
>  > vor       25 99 vorticity (s-1)
>  > pv        25 99 potential vort (pvu)
>  > dbz       25 99 reflectivity (dbz)
>  > div       25 99 divergence (s-1)
>  > dir       25 99 dir (degrees)
>  > tadv      25 99 temp advection (K/s)
>  > endvars
>  >
>  >
>  > Leopoldo
>  >
>  >
>  >
>  > On Sep 23, 2013, at 3:03 PM, James T. Potemra wrote:
>  >
>  > >
>  > > Leopoldo:
>  > >
>  > > You have a couple problems here.  First, your descriptor file
>  > specifies 1036 time values, but you are trying to access t=1064 (?).
>  > You should be sure about the number of time steps in your data set.
>  >
>  > > Second, as Jennifer suggested, x*y*z*t*4 should be your file size.
>  > In this case you have 22 variables with 25 levels, and 11 with 1; 149
>  > x-points; 123 y-points and 1036 time points. I think your file should
>  > therefore be (22*25+11)*149*123*1036*4 = 426,063,116,368 bytes.
>  >
>  > When you are using PDEF, the xsize and ysize come from the first two
>  > args in the PDEF entry. Here is my estimate of what the file size
>  > should be:
>  > sfc_vars * x * y * t *  4 bytes = 11* 60 * 60 * 1036 * 4 = 164102400
>  > plus
>  > z-varying-vars * z * x * y * t * 4 bytes = 22 * 25 * 60 *60 * 1036 * 4
>  > = 8205120000
>  > equals 8369222400 bytes.
>  >
>  > If you have 1064 time steps instead of 1036 then you would have a file
>  > size of 8595417600 bytes.
>  >
>  > --Jennifer
>  >
>  >
>  > >
>  > > Jim
>  > >
>  > > On 9/23/13 8:24 AM, Leopoldo Alvarez wrote:
>  > >> Hi Jennifer,
>  > >>
>  > >> >> and my data file .dat weighs 32 Gb
>  > >> >A precise file size is required, down to the last byte.
>  > >>
>  > >> Now my data file .dat weighs 66 Gb
>  > >> 68926752 - 66G 2013-09-22 11:11 test.dat
>  > >>
>  > >> I can see the data well, until the time of step 1064 (set t 1064,
>  > then 'd t' it`s ok), but when i exceeded from 1064 time of step gives
>  > me that error ...
>  > >>
>  > >> My full GrADS descriptor file ->
>  > >>
>  > >> dset ^test.dat
>  > >> options big_endian
>  > >> title MM5 data
>  > >> undef -9999.
>  > >> pdef   60   60 lcc   32.63 -112.54    1.00    1.00 30.00   60.00
>  > -110.34   6000.   6000.
>  > >> xdef  149 linear -112.65  0.0270
>  > >> ydef  123 linear   32.63  0.0270
>  > >> zdef  25 levels
>  > >>  0.99689
>  > >>  0.99191
>  > >>  0.98944
>  > >>  0.98818
>  > >>  0.97377
>  > >>  0.94500
>  > >>  0.91000
>  > >>  0.87000
>  > >>  0.82500
>  > >>  0.77500
>  > >>  0.72500
>  > >>  0.67500
>  > >>  0.62500
>  > >>  0.57500
>  > >>  0.52500
>  > >>  0.47500
>  > >>  0.42500
>  > >>  0.37500
>  > >>  0.32500
>  > >>  0.27500
>  > >>  0.22500
>  > >>  0.17500
>  > >>  0.12500
>  > >>  0.07500
>  > >>  0.02500
>  > >> tdef 1036 linear 00:00Z02JAN2012   60MN
>  > >> vars 33
>  > >> rc       0 99 accum conv pcn (cm)
>  > >> rn       0 99 accum non-c pcn (cm)
>  > >> ter      0 99 ter elevation (m)
>  > >> xlat     0 99 cross lat (degree)
>  > >> xlon     0 99 cross lon (degree)
>  > >> lu       0 99 land use
>  > >> t2m      0 99 2 m temperature (K)
>  > >> q2m      0 99 2m mix ratio (kg/kg)
>  > >> u10      0 99 10 m u wind (m/sec)
>  > >> v10      0 99 10 m v wind (m/sec)
>  > >> pslv     0 99 sea level prs (mb)
>  > >> u         25 99 u wind (m/s)
>  > >> v         25 99 v wind (m/s)
>  > >> w         25 99 vertical vel (m/s)
>  > >> pp        25 99 prs pert (Pa)
>  > >> t         25 99 temperature (C)
>  > >> q         25 99 mixing ratio (kg/kg)
>  > >> clw       25 99 cloud water (kg/kg)
>  > >> rnw       25 99 rain water (kg/kg)
>  > >> rtnd      25 99 rad tend (K/day)
>  > >> z         25 99 height AGL (m)
>  > >> h         25 99 geopot height (m)
>  > >> td        25 99 dewpoint temp (C)
>  > >> rh        25 99 rel humidity (%)
>  > >> th        25 99 potential temp (K)
>  > >> the       25 99 theta-e (K)
>  > >> prs       25 99 pressure (Pa)
>  > >> vor       25 99 vorticity (s-1)
>  > >> pv        25 99 potential vort (pvu)
>  > >> dbz       25 99 reflectivity (dbz)
>  > >> div       25 99 divergence (s-1)
>  > >> dir       25 99 dir (degrees)
>  > >> tadv      25 99 temp advection (K/s)
>  > >> endvars
>  > >>
>  > >>
>  > >> El 21/09/13 09:35, Leopoldo Alvarez escribió:
>  > >>> Hi Jennifer,
>  > >>>
>  > >>> My binary data grad.exe file is 210 Kb. and data file .dat weighs
>  > 32 Gb.
>  > >>> xsize*ysize*zsize*tsize*esize*4 , Where I see these parameters? i
>  > don´t know how to fix it, where ?, I need to recompile ?.
>  > >>>
>  > >>> best regards,
>  > >>>
>  > >>> Leopoldo ÁLVAREZ
>  > >>> Tenerife - Canary Islands
>  > >>>
>  > >>>
>  > >>>
>  > >>>
>  > >>> El 20/09/13 23:13, Leopoldo Alvarez escribió:
>  > >>>> Hi jennifer,
>  > >>>>
>  > >>>> I am running a script (nuevo.gs) to extract info domain mm5 model
>  > data. I use GrADS 2.0.2 version. I use 'options big_endian' but my
>  > data file weighs 32 GB and
>  > >>>> shows multiple mutiple on screen- ->
>  > >>>>
>  > >>>> Low Level I/O Error:  Seek error on data file
>  > >>>>   Data file name = test.dat
>  > >>>> 650 rc=-1 fpos=-172300 pfi->fhdr=0
>  > >>>>
>  > >>>> I think the problem is when running the mm5_to_grads.csh and
>  > creates the .ctl and .dat and not script. I can see the data well,
>  > until the time of step 1062 (set t 1062, then 'd t' it`s ok), but when
>  > i exceeded from 1062 time of step gives me that error ... How i can
>  > run my script saving this problem???. One clue
>  > http://gradsusr.org/pipermail/gradsusr/2008-April/006883.html
>  > >>>>
>  > >>>> I'm on a machine that works -> uname -a
>  > >>>> Linux login1 2.6.16.53-0.16-ppc64 #1 SMP Tue Oct 2 16:57:49 UTC
>  > 2007 ppc64 ppc64 ppc64 GNU/Linux
>  > >>>>
>  > >>>>
>  > >>>> nuevo.gs script
>  > >>>> ***************
>  > >>>> ***************
>  > >>>> 'set lon -110.1829'
>  > >>>> 'set lat 34.6518'
>  > >>>> 'set lev 0.99199'
>  > >>>>
>  > >>>> #8737
>  > >>>> timeend = 4198
>  > >>>> i = 1
>  > >>>> while(i <= timeend)
>  > >>>> 'set t 'i''
>  > >>>> 'q time'
>  > >>>> timehr = substr(result, 8, 3)
>  > >>>> timedy = substr(result, 11, 2)
>  > >>>> timemo = substr(result, 13, 3)
>  > >>>> timeyr = substr(result, 16, 4)
>  > >>>>
>  > >>>> #velocidad viento
>  > >>>> 'tg1=mag(u,v)'
>  > >>>> 'd tg1'
>  > >>>> a.nr1 = subwrd(result,4)
>  > >>>> #direccion viento
>  > >>>> 'd dir'
>  > >>>> a.nr11 = subwrd(result,10)
>  > >>>> # tv de Jeffrey iowa
>  > >>>> 'tv=t+t*0.61*q'
>  > >>>> 'rho=prs/(286.9*tv)'
>  > >>>> 'd rho'
>  > >>>> a.nr = subwrd(result,4)
>  > >>>> 'd t-273.15'
>  > >>>> a.nr2 = subwrd(result,10)
>  > >>>> 'd rh'
>  > >>>> a.nr3 = subwrd(result,10)
>  > >>>> 'd prs/100'
>  > >>>> a.nr4 = subwrd(result,10)
>  > >>>>
>  > >>>> val = timeyr%"-"%timemo%"-"%timedy"-"%timehr" "%a.nr1" "%a.nr11"
>  > "%a.nr" "%a.nr2" "%a.nr3" "%a.nr4
>  > >>>> rec = write('drylake_80v.txt',val, append)
>  > >>>> i = i + 1
>  > >>>> endwhile
>  > >>>> *******************
>  > >>>> *******************
>  > >>>>
>  > >>>> best regards,
>  > >>>>
>  > >>>> Leopoldo ÁLVAREZ
>  > >>>> Tenerife - Canary Islands
>  > >>>
>  > >>
>  > >>
>  > >>
>  > >> _______________________________________________
>  > >> gradsusr mailing list
>  > >> gradsusr at gradsusr.org
>  > >> http://gradsusr.org/mailman/listinfo/gradsusr
>  > >
>  > > _______________________________________________
>  > > gradsusr mailing list
>  > > gradsusr at gradsusr.org
>  > > http://gradsusr.org/mailman/listinfo/gradsusr
>  >
>  > --
>  > Jennifer M. Adams
>  > Center for Ocean-Land-Atmosphere Studies (COLA)
>  > 111 Research Hall, Mail Stop 2B3
>  > George Mason University
>  > 4400 University Drive
>  > Fairfax, VA 22030
>  >
> 
> 
> 
> 
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20130924/b738d82c/attachment-0001.html 


More information about the gradsusr mailing list