[gradsusr] Fwd: Low Level I/O Error: Seek error on data file [GrADS] -

Leopoldo Alvarez lap.bolsa at gmail.com
Tue Sep 24 14:02:23 EDT 2013


Jennifer,

I'm going to make climatic modeling (>30 years) and I can not be cutting 
files for years, buff ... it should work for powerpc with appropriate 
arguments ... Can be compiled with gfortran compiler?
This morning I opened a file of 7.7 Gb...

Leopoldo


-------- Mensaje original --------
Asunto: 	[gradsusr] Low Level I/O Error: Seek error on data file [GrADS] -
Fecha: 	Tue, 24 Sep 2013 15:02:30 +0100
De: 	Leopoldo Alvarez <lap.bolsa at gmail.com>
Para: 	gradsusr at gradsusr.org



Dear Jennifer,

Now I'm more confused because I was able to open a file of 7.7 Gb
8173088 - 7.8G 2013-09-20 09:57 test.dat
Why I can not open the other file ???

My full GrADS descriptor file is (similar) ->

dset ^test.dat
options big_endian
title MM5 data
undef -9999.
pdef   60   60 lcc   32.63 -112.54    1.00    1.00   30.00 60.00 
-110.34   6000.   6000.
xdef  149 linear -112.65  0.0270
ydef  123 linear   32.63  0.0270
zdef  25 levels
  0.99689
  0.99191
  0.98944
  0.98818
  0.97377
  0.94500
  0.91000
  0.87000
  0.82500
  0.77500
  0.72500
  0.67500
  0.62500
  0.57500
  0.52500
  0.47500
  0.42500
  0.37500
  0.32500
  0.27500
  0.22500
  0.17500
  0.12500
  0.07500
  0.02500
tdef 1036 linear 00:00Z02JAN2012   60MN
vars 33
rc       0 99 accum conv pcn (cm)
rn       0 99 accum non-c pcn (cm)
ter      0 99 ter elevation (m)
xlat     0 99 cross lat (degree)
xlon     0 99 cross lon (degree)
lu       0 99 land use
t2m      0 99 2 m temperature (K)
q2m      0 99 2m mix ratio (kg/kg)
u10      0 99 10 m u wind (m/sec)
v10      0 99 10 m v wind (m/sec)
pslv     0 99 sea level prs (mb)
u         25 99 u wind (m/s)
v         25 99 v wind (m/s)
w         25 99 vertical vel (m/s)
pp        25 99 prs pert (Pa)
t         25 99 temperature (C)
q         25 99 mixing ratio (kg/kg)
clw       25 99 cloud water (kg/kg)
rnw       25 99 rain water (kg/kg)
rtnd      25 99 rad tend (K/day)
z         25 99 height AGL (m)
h         25 99 geopot height (m)
td        25 99 dewpoint temp (C)
rh        25 99 rel humidity (%)
th        25 99 potential temp (K)
the       25 99 theta-e (K)
prs       25 99 pressure (Pa)
vor       25 99 vorticity (s-1)
pv        25 99 potential vort (pvu)
dbz       25 99 reflectivity (dbz)
div       25 99 divergence (s-1)
dir       25 99 dir (degrees)
tadv      25 99 temp advection (K/s)
endvars

Leopoldo







Dear Jennifer,

I don´t know how to fix it... :-( mmmh

ga-> q config
Config: v2.0.2 big-endian readline
Grid Analysis and Display System (GrADS) Version 2.0.2
Copyright (c) 1988-2011 by Brian Doty and the
Institute for Global Environment and Society (IGES)
This program is distributed WITHOUT ANY WARRANTY
See file COPYRIGHT for more information.

Built Thu Sep  5 12:08:35 BST 2013 for powerpc64-unknown-linux-gnu

This version of GrADS has been configured with the following options:
    o Built on a BIG ENDIAN machine
    o Athena Widget GUI DISABLED
    o Command line editing ENABLED
http://tiswww.case.edu/php/chet/readline/rltop.html
    o printim command DISABLED
    o GRIB2 interface DISABLED
    o NetCDF interface DISABLED
    o OPeNDAP gridded data interface DISABLED
    o OPeNDAP station data interface DISABLED
    o HDF4 and HDF5 interfaces DISABLED
    o GeoTIFF and KML/TIFF output DISABLED
    o KML contour output ENABLED
    o Shapefile interface DISABLED

For additional information please consult http://iges.org/grads

Leopoldo ÁLVAREZ





Here the configuration to compile for power pc that I used ->
              grep Linux .tmpfile ; \
           if [ $$? = 0 ]; then echo "Compiling for PowerPC"
;               \
                   ( $(CD) src ; $(MAKE) all                               \
                   "RM             = $(RM)" "RM_LIST        =
$(RM_LIST)"   \
                   "LN             = $(LN)" "MACH           =
IBM"          \
                   "MAKE           = $(MAKE)" "CPP            =
/lib/cpp"     \
                   "CPPFLAGS       = -I. -C -P -DIBM -DBIT32
-traditional"        \
                   "FC             = xlf90 "  "FCFLAGS        = -qfree=f90
-qlanglvl=90pure -I../util"   \
                   "LDOPTIONS      = " "CFLAGS         = -I.
-DUNDERSCORE"             \
                   "LOCAL_LIBRARIES= " ) ; \

Leopoldo


El 23/09/13 21:35, Leopoldo Alvarez escribió:
  > Dear Jennifer and James,
  >
  > The size file that I generated with my mm5_to_grads.csh is ->
  > [login1 17] /gpfs/projects/XXXXX/XXXXXX/mesoescala/MM5toGrADS > du -k
  > test.dat
  > *68926752* 2013-09-22 11:11 test.dat
  >
  > I think it can be that grad.exe is bad compiled?  Is this file
  > (test.dat) bad generated? Is it really a problem of file size? how I
  > fix it?
  > >>>> I'm on a machine that works -> uname -a
  > >>>> Linux login1 2.6.16.53-0.16-ppc64 #1 SMP Tue Oct 2 16:57:49 UTC
  > 2007 ppc64 ppc64 ppc64 GNU/Linux
  >
  > > When you are using PDEF, the xsize and ysize come from the first two
  > args in the PDEF entry. Here is my estimate of what the file size
  > should be: sfc_vars * x * y * t *  4 bytes = 11* 60 * 60 * 1036 * 4 =
  > 164102400
  > > plus
  > > z-varying-vars * z * x * y * t * 4 bytes = 22 * 25 * 60 *60 * 1036 *
  > 4 = 8205120000
  > > equals 8369222400 bytes.
  >
  > I paste another GrADS descriptor file, sorry.
  > I can see the data well, until the time of step 1064 (set t 1064, then
  > 'd t' it`s ok), but when i exceeded from 1064 time of step gives me
  > that error ...
  >
  > /ga-> set t *1065*
  > Time values set: 2012:2:15:8 2012:2:15:8
  > ga-> d t
  > Notice:  Automatic Grid Interpolation Taking Place
  > Low Level I/O Error:  Seek error on data file
  >   Data file name = test.dat
  > 650 rc=-1 fpos=-2145713296 pfi->fhdr=0
  >   Error occurred when seeking to byte 7081408
  > Data Request Error:  Error for variable 't'
  >   Error ocurred at column 1
  > DISPLAY error:  Invalid expression
  >   Expression = t/
  >
  > My full GrADS descriptor file is OK to *8737 *time values (one year) ->
  >
  > dset ^test.dat
  > options big_endian
  > title MM5 data
  > undef -9999.
  > pdef   60   60 lcc   32.63 -112.54    1.00    1.00   30.00 60.00
  > -110.34   6000.   6000.
  > xdef  149 linear -112.65  0.0270
  > ydef  123 linear   32.63  0.0270
  > zdef  25 levels
  >  0.99689
  >  0.99191
  >  0.98944
  >  0.98818
  >  0.97377
  >  0.94500
  >  0.91000
  >  0.87000
  >  0.82500
  >  0.77500
  >  0.72500
  >  0.67500
  >  0.62500
  >  0.57500
  >  0.52500
  >  0.47500
  >  0.42500
  >  0.37500
  >  0.32500
  >  0.27500
  >  0.22500
  >  0.17500
  >  0.12500
  >  0.07500
  >  0.02500
  > tdef*8737 *linear 00:00Z02JAN2012   60MN
  > vars 33
  > rc       0 99 accum conv pcn (cm)
  > rn       0 99 accum non-c pcn (cm)
  > ter      0 99 ter elevation (m)
  > xlat     0 99 cross lat (degree)
  > xlon     0 99 cross lon (degree)
  > lu       0 99 land use
  > t2m      0 99 2 m temperature (K)
  > q2m      0 99 2m mix ratio (kg/kg)
  > u10      0 99 10 m u wind (m/sec)
  > v10      0 99 10 m v wind (m/sec)
  > pslv     0 99 sea level prs (mb)
  > u         25 99 u wind (m/s)
  > v         25 99 v wind (m/s)
  > w         25 99 vertical vel (m/s)
  > pp        25 99 prs pert (Pa)
  > t         25 99 temperature (C)
  > q         25 99 mixing ratio (kg/kg)
  > clw       25 99 cloud water (kg/kg)
  > rnw       25 99 rain water (kg/kg)
  > rtnd      25 99 rad tend (K/day)
  > z         25 99 height AGL (m)
  > h         25 99 geopot height (m)
  > td        25 99 dewpoint temp (C)
  > rh        25 99 rel humidity (%)
  > th        25 99 potential temp (K)
  > the       25 99 theta-e (K)
  > prs       25 99 pressure (Pa)
  > vor       25 99 vorticity (s-1)
  > pv        25 99 potential vort (pvu)
  > dbz       25 99 reflectivity (dbz)
  > div       25 99 divergence (s-1)
  > dir       25 99 dir (degrees)
  > tadv      25 99 temp advection (K/s)
  > endvars
  >
  >
  > Leopoldo
  >
  >
  >
  > On Sep 23, 2013, at 3:03 PM, James T. Potemra wrote:
  >
  > >
  > > Leopoldo:
  > >
  > > You have a couple problems here.  First, your descriptor file
  > specifies 1036 time values, but you are trying to access t=1064 (?).
  > You should be sure about the number of time steps in your data set.
  >
  > > Second, as Jennifer suggested, x*y*z*t*4 should be your file size.
  > In this case you have 22 variables with 25 levels, and 11 with 1; 149
  > x-points; 123 y-points and 1036 time points. I think your file should
  > therefore be (22*25+11)*149*123*1036*4 = 426,063,116,368 bytes.
  >
  > When you are using PDEF, the xsize and ysize come from the first two
  > args in the PDEF entry. Here is my estimate of what the file size
  > should be:
  > sfc_vars * x * y * t *  4 bytes = 11* 60 * 60 * 1036 * 4 = 164102400
  > plus
  > z-varying-vars * z * x * y * t * 4 bytes = 22 * 25 * 60 *60 * 1036 * 4
  > = 8205120000
  > equals 8369222400 bytes.
  >
  > If you have 1064 time steps instead of 1036 then you would have a file
  > size of 8595417600 bytes.
  >
  > --Jennifer
  >
  >
  > >
  > > Jim
  > >
  > > On 9/23/13 8:24 AM, Leopoldo Alvarez wrote:
  > >> Hi Jennifer,
  > >>
  > >> >> and my data file .dat weighs 32 Gb
  > >> >A precise file size is required, down to the last byte.
  > >>
  > >> Now my data file .dat weighs 66 Gb
  > >> 68926752 - 66G 2013-09-22 11:11 test.dat
  > >>
  > >> I can see the data well, until the time of step 1064 (set t 1064,
  > then 'd t' it`s ok), but when i exceeded from 1064 time of step gives
  > me that error ...
  > >>
  > >> My full GrADS descriptor file ->
  > >>
  > >> dset ^test.dat
  > >> options big_endian
  > >> title MM5 data
  > >> undef -9999.
  > >> pdef   60   60 lcc   32.63 -112.54    1.00 1.00 30.00   60.00
  > -110.34   6000.   6000.
  > >> xdef  149 linear -112.65  0.0270
  > >> ydef  123 linear   32.63  0.0270
  > >> zdef  25 levels
  > >>  0.99689
  > >>  0.99191
  > >>  0.98944
  > >>  0.98818
  > >>  0.97377
  > >>  0.94500
  > >>  0.91000
  > >>  0.87000
  > >>  0.82500
  > >>  0.77500
  > >>  0.72500
  > >>  0.67500
  > >>  0.62500
  > >>  0.57500
  > >>  0.52500
  > >>  0.47500
  > >>  0.42500
  > >>  0.37500
  > >>  0.32500
  > >>  0.27500
  > >>  0.22500
  > >>  0.17500
  > >>  0.12500
  > >>  0.07500
  > >>  0.02500
  > >> tdef 1036 linear 00:00Z02JAN2012   60MN
  > >> vars 33
  > >> rc       0 99 accum conv pcn (cm)
  > >> rn       0 99 accum non-c pcn (cm)
  > >> ter      0 99 ter elevation (m)
  > >> xlat     0 99 cross lat (degree)
  > >> xlon     0 99 cross lon (degree)
  > >> lu       0 99 land use
  > >> t2m      0 99 2 m temperature (K)
  > >> q2m      0 99 2m mix ratio (kg/kg)
  > >> u10      0 99 10 m u wind (m/sec)
  > >> v10      0 99 10 m v wind (m/sec)
  > >> pslv     0 99 sea level prs (mb)
  > >> u         25 99 u wind (m/s)
  > >> v         25 99 v wind (m/s)
  > >> w         25 99 vertical vel (m/s)
  > >> pp        25 99 prs pert (Pa)
  > >> t         25 99 temperature (C)
  > >> q         25 99 mixing ratio (kg/kg)
  > >> clw       25 99 cloud water (kg/kg)
  > >> rnw       25 99 rain water (kg/kg)
  > >> rtnd      25 99 rad tend (K/day)
  > >> z         25 99 height AGL (m)
  > >> h         25 99 geopot height (m)
  > >> td        25 99 dewpoint temp (C)
  > >> rh        25 99 rel humidity (%)
  > >> th        25 99 potential temp (K)
  > >> the       25 99 theta-e (K)
  > >> prs       25 99 pressure (Pa)
  > >> vor       25 99 vorticity (s-1)
  > >> pv        25 99 potential vort (pvu)
  > >> dbz       25 99 reflectivity (dbz)
  > >> div       25 99 divergence (s-1)
  > >> dir       25 99 dir (degrees)
  > >> tadv      25 99 temp advection (K/s)
  > >> endvars
  > >>
  > >>
  > >> El 21/09/13 09:35, Leopoldo Alvarez escribió:
  > >>> Hi Jennifer,
  > >>>
  > >>> My binary data grad.exe file is 210 Kb. and data file .dat weighs
  > 32 Gb.
  > >>> xsize*ysize*zsize*tsize*esize*4 , Where I see these parameters? i
  > don´t know how to fix it, where ?, I need to recompile ?.
  > >>>
  > >>> best regards,
  > >>>
  > >>> Leopoldo ÁLVAREZ
  > >>> Tenerife - Canary Islands
  > >>>
  > >>>
  > >>>
  > >>>
  > >>> El 20/09/13 23:13, Leopoldo Alvarez escribió:
  > >>>> Hi jennifer,
  > >>>>
  > >>>> I am running a script (nuevo.gs) to extract info domain mm5 model
  > data. I use GrADS 2.0.2 version. I use 'options big_endian' but my
  > data file weighs 32 GB and
  > >>>> shows multiple mutiple on screen- ->
  > >>>>
  > >>>> Low Level I/O Error:  Seek error on data file
  > >>>>   Data file name = test.dat
  > >>>> 650 rc=-1 fpos=-172300 pfi->fhdr=0
  > >>>>
  > >>>> I think the problem is when running the mm5_to_grads.csh and
  > creates the .ctl and .dat and not script. I can see the data well,
  > until the time of step 1062 (set t 1062, then 'd t' it`s ok), but when
  > i exceeded from 1062 time of step gives me that error ... How i can
  > run my script saving this problem???. One clue
  > http://gradsusr.org/pipermail/gradsusr/2008-April/006883.html
  > >>>>
  > >>>> I'm on a machine that works -> uname -a
  > >>>> Linux login1 2.6.16.53-0.16-ppc64 #1 SMP Tue Oct 2 16:57:49 UTC
  > 2007 ppc64 ppc64 ppc64 GNU/Linux
  > >>>>
  > >>>>
  > >>>> nuevo.gs script
  > >>>> ***************
  > >>>> ***************
  > >>>> 'set lon -110.1829'
  > >>>> 'set lat 34.6518'
  > >>>> 'set lev 0.99199'
  > >>>>
  > >>>> #8737
  > >>>> timeend = 4198
  > >>>> i = 1
  > >>>> while(i <= timeend)
  > >>>> 'set t 'i''
  > >>>> 'q time'
  > >>>> timehr = substr(result, 8, 3)
  > >>>> timedy = substr(result, 11, 2)
  > >>>> timemo = substr(result, 13, 3)
  > >>>> timeyr = substr(result, 16, 4)
  > >>>>
  > >>>> #velocidad viento
  > >>>> 'tg1=mag(u,v)'
  > >>>> 'd tg1'
  > >>>> a.nr1 = subwrd(result,4)
  > >>>> #direccion viento
  > >>>> 'd dir'
  > >>>> a.nr11 = subwrd(result,10)
  > >>>> # tv de Jeffrey iowa
  > >>>> 'tv=t+t*0.61*q'
  > >>>> 'rho=prs/(286.9*tv)'
  > >>>> 'd rho'
  > >>>> a.nr = subwrd(result,4)
  > >>>> 'd t-273.15'
  > >>>> a.nr2 = subwrd(result,10)
  > >>>> 'd rh'
  > >>>> a.nr3 = subwrd(result,10)
  > >>>> 'd prs/100'
  > >>>> a.nr4 = subwrd(result,10)
  > >>>>
  > >>>> val = timeyr%"-"%timemo%"-"%timedy"-"%timehr" "%a.nr1" "%a.nr11"
  > "%a.nr" "%a.nr2" "%a.nr3" "%a.nr4
  > >>>> rec = write('drylake_80v.txt',val, append)
  > >>>> i = i + 1
  > >>>> endwhile
  > >>>> *******************
  > >>>> *******************
  > >>>>
  > >>>> best regards,
  > >>>>
  > >>>> Leopoldo ÁLVAREZ
  > >>>> Tenerife - Canary Islands
  > >>>
  > >>
  > >>
  > >>
  > >> _______________________________________________
  > >> gradsusr mailing list
  > >> gradsusr at gradsusr.org
  > >> http://gradsusr.org/mailman/listinfo/gradsusr
  > >
  > > _______________________________________________
  > > gradsusr mailing list
  > > gradsusr at gradsusr.org
  > > http://gradsusr.org/mailman/listinfo/gradsusr
  >
  > --
  > Jennifer M. Adams
  > Center for Ocean-Land-Atmosphere Studies (COLA)
  > 111 Research Hall, Mail Stop 2B3
  > George Mason University
  > 4400 University Drive
  > Fairfax, VA 22030
  >




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20130924/d87e05a1/attachment-0001.html 


More information about the gradsusr mailing list