[gradsusr] Low Level I/O Error: Seek error on data file [GrADS] -
Jennifer Adams
jma at cola.iges.org
Tue Sep 24 13:36:33 EDT 2013
Dear Leopoldo --
I have no experience working with powerpc64-unknown-linux-gnu. If you feel that this OS is not capable of reading files greater than 2Gb, then you should follow the advice of Matias Mendez and use the MM5 tools at your disposal to split your files into smaller pieces and use templating to aggregate them together into the full timeseries.
http://iges.org/grads/gadoc/templates.html
--Jennifer
On Sep 24, 2013, at 7:11 AM, Leopoldo Alvarez wrote:
> Dear Jennifer,
>
> I don´t know how to fix it... :-( mmmh
>
> ga-> q config
> Config: v2.0.2 big-endian readline
> Grid Analysis and Display System (GrADS) Version 2.0.2
> Copyright (c) 1988-2011 by Brian Doty and the
> Institute for Global Environment and Society (IGES)
> This program is distributed WITHOUT ANY WARRANTY
> See file COPYRIGHT for more information.
>
> Built Thu Sep 5 12:08:35 BST 2013 for powerpc64-unknown-linux-gnu
>
> This version of GrADS has been configured with the following options:
> o Built on a BIG ENDIAN machine
> o Athena Widget GUI DISABLED
> o Command line editing ENABLED
> http://tiswww.case.edu/php/chet/readline/rltop.html
> o printim command DISABLED
> o GRIB2 interface DISABLED
> o NetCDF interface DISABLED
> o OPeNDAP gridded data interface DISABLED
> o OPeNDAP station data interface DISABLED
> o HDF4 and HDF5 interfaces DISABLED
> o GeoTIFF and KML/TIFF output DISABLED
> o KML contour output ENABLED
> o Shapefile interface DISABLED
>
> For additional information please consult http://iges.org/grads
>
> Leopoldo ÁLVAREZ
>
>
>
>
>
> Here the configuration to compile for power pc that I used ->
> grep Linux .tmpfile ; \
> if [ $$? = 0 ]; then echo "Compiling for PowerPC" ; \
> ( $(CD) src ; $(MAKE) all \
> "RM = $(RM)" "RM_LIST =
> $(RM_LIST)" \
> "LN = $(LN)" "MACH =
> IBM" \
> "MAKE = $(MAKE)" "CPP =
> /lib/cpp" \
> "CPPFLAGS = -I. -C -P -DIBM -DBIT32
> -traditional" \
> "FC = xlf90 " "FCFLAGS = -qfree=f90
> -qlanglvl=90pure -I../util" \
> "LDOPTIONS = " "CFLAGS = -I.
> -DUNDERSCORE" \
> "LOCAL_LIBRARIES= " ) ; \
>
> Leopoldo
>
>
> El 23/09/13 21:35, Leopoldo Alvarez escribió:
> > Dear Jennifer and James,
> >
> > The size file that I generated with my mm5_to_grads.csh is ->
> > [login1 17] /gpfs/projects/XXXXX/XXXXXX/mesoescala/MM5toGrADS > du -k
> > test.dat
> > *68926752* 2013-09-22 11:11 test.dat
> >
> > I think it can be that grad.exe is bad compiled? Is this file
> > (test.dat) bad generated? Is it really a problem of file size? how I
> > fix it?
> > >>>> I'm on a machine that works -> uname -a
> > >>>> Linux login1 2.6.16.53-0.16-ppc64 #1 SMP Tue Oct 2 16:57:49 UTC
> > 2007 ppc64 ppc64 ppc64 GNU/Linux
> >
> > > When you are using PDEF, the xsize and ysize come from the first two
> > args in the PDEF entry. Here is my estimate of what the file size
> > should be: sfc_vars * x * y * t * 4 bytes = 11* 60 * 60 * 1036 * 4 =
> > 164102400
> > > plus
> > > z-varying-vars * z * x * y * t * 4 bytes = 22 * 25 * 60 *60 * 1036 *
> > 4 = 8205120000
> > > equals 8369222400 bytes.
> >
> > I paste another GrADS descriptor file, sorry.
> > I can see the data well, until the time of step 1064 (set t 1064, then
> > 'd t' it`s ok), but when i exceeded from 1064 time of step gives me
> > that error ...
> >
> > /ga-> set t *1065*
> > Time values set: 2012:2:15:8 2012:2:15:8
> > ga-> d t
> > Notice: Automatic Grid Interpolation Taking Place
> > Low Level I/O Error: Seek error on data file
> > Data file name = test.dat
> > 650 rc=-1 fpos=-2145713296 pfi->fhdr=0
> > Error occurred when seeking to byte 7081408
> > Data Request Error: Error for variable 't'
> > Error ocurred at column 1
> > DISPLAY error: Invalid expression
> > Expression = t/
> >
> > My full GrADS descriptor file is OK to *8737 *time values (one year) ->
> >
> > dset ^test.dat
> > options big_endian
> > title MM5 data
> > undef -9999.
> > pdef 60 60 lcc 32.63 -112.54 1.00 1.00 30.00 60.00
> > -110.34 6000. 6000.
> > xdef 149 linear -112.65 0.0270
> > ydef 123 linear 32.63 0.0270
> > zdef 25 levels
> > 0.99689
> > 0.99191
> > 0.98944
> > 0.98818
> > 0.97377
> > 0.94500
> > 0.91000
> > 0.87000
> > 0.82500
> > 0.77500
> > 0.72500
> > 0.67500
> > 0.62500
> > 0.57500
> > 0.52500
> > 0.47500
> > 0.42500
> > 0.37500
> > 0.32500
> > 0.27500
> > 0.22500
> > 0.17500
> > 0.12500
> > 0.07500
> > 0.02500
> > tdef*8737 *linear 00:00Z02JAN2012 60MN
> > vars 33
> > rc 0 99 accum conv pcn (cm)
> > rn 0 99 accum non-c pcn (cm)
> > ter 0 99 ter elevation (m)
> > xlat 0 99 cross lat (degree)
> > xlon 0 99 cross lon (degree)
> > lu 0 99 land use
> > t2m 0 99 2 m temperature (K)
> > q2m 0 99 2m mix ratio (kg/kg)
> > u10 0 99 10 m u wind (m/sec)
> > v10 0 99 10 m v wind (m/sec)
> > pslv 0 99 sea level prs (mb)
> > u 25 99 u wind (m/s)
> > v 25 99 v wind (m/s)
> > w 25 99 vertical vel (m/s)
> > pp 25 99 prs pert (Pa)
> > t 25 99 temperature (C)
> > q 25 99 mixing ratio (kg/kg)
> > clw 25 99 cloud water (kg/kg)
> > rnw 25 99 rain water (kg/kg)
> > rtnd 25 99 rad tend (K/day)
> > z 25 99 height AGL (m)
> > h 25 99 geopot height (m)
> > td 25 99 dewpoint temp (C)
> > rh 25 99 rel humidity (%)
> > th 25 99 potential temp (K)
> > the 25 99 theta-e (K)
> > prs 25 99 pressure (Pa)
> > vor 25 99 vorticity (s-1)
> > pv 25 99 potential vort (pvu)
> > dbz 25 99 reflectivity (dbz)
> > div 25 99 divergence (s-1)
> > dir 25 99 dir (degrees)
> > tadv 25 99 temp advection (K/s)
> > endvars
> >
> >
> > Leopoldo
> >
> >
> >
> > On Sep 23, 2013, at 3:03 PM, James T. Potemra wrote:
> >
> > >
> > > Leopoldo:
> > >
> > > You have a couple problems here. First, your descriptor file
> > specifies 1036 time values, but you are trying to access t=1064 (?).
> > You should be sure about the number of time steps in your data set.
> >
> > > Second, as Jennifer suggested, x*y*z*t*4 should be your file size.
> > In this case you have 22 variables with 25 levels, and 11 with 1; 149
> > x-points; 123 y-points and 1036 time points. I think your file should
> > therefore be (22*25+11)*149*123*1036*4 = 426,063,116,368 bytes.
> >
> > When you are using PDEF, the xsize and ysize come from the first two
> > args in the PDEF entry. Here is my estimate of what the file size
> > should be:
> > sfc_vars * x * y * t * 4 bytes = 11* 60 * 60 * 1036 * 4 = 164102400
> > plus
> > z-varying-vars * z * x * y * t * 4 bytes = 22 * 25 * 60 *60 * 1036 * 4
> > = 8205120000
> > equals 8369222400 bytes.
> >
> > If you have 1064 time steps instead of 1036 then you would have a file
> > size of 8595417600 bytes.
> >
> > --Jennifer
> >
> >
> > >
> > > Jim
> > >
> > > On 9/23/13 8:24 AM, Leopoldo Alvarez wrote:
> > >> Hi Jennifer,
> > >>
> > >> >> and my data file .dat weighs 32 Gb
> > >> >A precise file size is required, down to the last byte.
> > >>
> > >> Now my data file .dat weighs 66 Gb
> > >> 68926752 - 66G 2013-09-22 11:11 test.dat
> > >>
> > >> I can see the data well, until the time of step 1064 (set t 1064,
> > then 'd t' it`s ok), but when i exceeded from 1064 time of step gives
> > me that error ...
> > >>
> > >> My full GrADS descriptor file ->
> > >>
> > >> dset ^test.dat
> > >> options big_endian
> > >> title MM5 data
> > >> undef -9999.
> > >> pdef 60 60 lcc 32.63 -112.54 1.00 1.00 30.00 60.00
> > -110.34 6000. 6000.
> > >> xdef 149 linear -112.65 0.0270
> > >> ydef 123 linear 32.63 0.0270
> > >> zdef 25 levels
> > >> 0.99689
> > >> 0.99191
> > >> 0.98944
> > >> 0.98818
> > >> 0.97377
> > >> 0.94500
> > >> 0.91000
> > >> 0.87000
> > >> 0.82500
> > >> 0.77500
> > >> 0.72500
> > >> 0.67500
> > >> 0.62500
> > >> 0.57500
> > >> 0.52500
> > >> 0.47500
> > >> 0.42500
> > >> 0.37500
> > >> 0.32500
> > >> 0.27500
> > >> 0.22500
> > >> 0.17500
> > >> 0.12500
> > >> 0.07500
> > >> 0.02500
> > >> tdef 1036 linear 00:00Z02JAN2012 60MN
> > >> vars 33
> > >> rc 0 99 accum conv pcn (cm)
> > >> rn 0 99 accum non-c pcn (cm)
> > >> ter 0 99 ter elevation (m)
> > >> xlat 0 99 cross lat (degree)
> > >> xlon 0 99 cross lon (degree)
> > >> lu 0 99 land use
> > >> t2m 0 99 2 m temperature (K)
> > >> q2m 0 99 2m mix ratio (kg/kg)
> > >> u10 0 99 10 m u wind (m/sec)
> > >> v10 0 99 10 m v wind (m/sec)
> > >> pslv 0 99 sea level prs (mb)
> > >> u 25 99 u wind (m/s)
> > >> v 25 99 v wind (m/s)
> > >> w 25 99 vertical vel (m/s)
> > >> pp 25 99 prs pert (Pa)
> > >> t 25 99 temperature (C)
> > >> q 25 99 mixing ratio (kg/kg)
> > >> clw 25 99 cloud water (kg/kg)
> > >> rnw 25 99 rain water (kg/kg)
> > >> rtnd 25 99 rad tend (K/day)
> > >> z 25 99 height AGL (m)
> > >> h 25 99 geopot height (m)
> > >> td 25 99 dewpoint temp (C)
> > >> rh 25 99 rel humidity (%)
> > >> th 25 99 potential temp (K)
> > >> the 25 99 theta-e (K)
> > >> prs 25 99 pressure (Pa)
> > >> vor 25 99 vorticity (s-1)
> > >> pv 25 99 potential vort (pvu)
> > >> dbz 25 99 reflectivity (dbz)
> > >> div 25 99 divergence (s-1)
> > >> dir 25 99 dir (degrees)
> > >> tadv 25 99 temp advection (K/s)
> > >> endvars
> > >>
> > >>
> > >> El 21/09/13 09:35, Leopoldo Alvarez escribió:
> > >>> Hi Jennifer,
> > >>>
> > >>> My binary data grad.exe file is 210 Kb. and data file .dat weighs
> > 32 Gb.
> > >>> xsize*ysize*zsize*tsize*esize*4 , Where I see these parameters? i
> > don´t know how to fix it, where ?, I need to recompile ?.
> > >>>
> > >>> best regards,
> > >>>
> > >>> Leopoldo ÁLVAREZ
> > >>> Tenerife - Canary Islands
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> El 20/09/13 23:13, Leopoldo Alvarez escribió:
> > >>>> Hi jennifer,
> > >>>>
> > >>>> I am running a script (nuevo.gs) to extract info domain mm5 model
> > data. I use GrADS 2.0.2 version. I use 'options big_endian' but my
> > data file weighs 32 GB and
> > >>>> shows multiple mutiple on screen- ->
> > >>>>
> > >>>> Low Level I/O Error: Seek error on data file
> > >>>> Data file name = test.dat
> > >>>> 650 rc=-1 fpos=-172300 pfi->fhdr=0
> > >>>>
> > >>>> I think the problem is when running the mm5_to_grads.csh and
> > creates the .ctl and .dat and not script. I can see the data well,
> > until the time of step 1062 (set t 1062, then 'd t' it`s ok), but when
> > i exceeded from 1062 time of step gives me that error ... How i can
> > run my script saving this problem???. One clue
> > http://gradsusr.org/pipermail/gradsusr/2008-April/006883.html
> > >>>>
> > >>>> I'm on a machine that works -> uname -a
> > >>>> Linux login1 2.6.16.53-0.16-ppc64 #1 SMP Tue Oct 2 16:57:49 UTC
> > 2007 ppc64 ppc64 ppc64 GNU/Linux
> > >>>>
> > >>>>
> > >>>> nuevo.gs script
> > >>>> ***************
> > >>>> ***************
> > >>>> 'set lon -110.1829'
> > >>>> 'set lat 34.6518'
> > >>>> 'set lev 0.99199'
> > >>>>
> > >>>> #8737
> > >>>> timeend = 4198
> > >>>> i = 1
> > >>>> while(i <= timeend)
> > >>>> 'set t 'i''
> > >>>> 'q time'
> > >>>> timehr = substr(result, 8, 3)
> > >>>> timedy = substr(result, 11, 2)
> > >>>> timemo = substr(result, 13, 3)
> > >>>> timeyr = substr(result, 16, 4)
> > >>>>
> > >>>> #velocidad viento
> > >>>> 'tg1=mag(u,v)'
> > >>>> 'd tg1'
> > >>>> a.nr1 = subwrd(result,4)
> > >>>> #direccion viento
> > >>>> 'd dir'
> > >>>> a.nr11 = subwrd(result,10)
> > >>>> # tv de Jeffrey iowa
> > >>>> 'tv=t+t*0.61*q'
> > >>>> 'rho=prs/(286.9*tv)'
> > >>>> 'd rho'
> > >>>> a.nr = subwrd(result,4)
> > >>>> 'd t-273.15'
> > >>>> a.nr2 = subwrd(result,10)
> > >>>> 'd rh'
> > >>>> a.nr3 = subwrd(result,10)
> > >>>> 'd prs/100'
> > >>>> a.nr4 = subwrd(result,10)
> > >>>>
> > >>>> val = timeyr%"-"%timemo%"-"%timedy"-"%timehr" "%a.nr1" "%a.nr11"
> > "%a.nr" "%a.nr2" "%a.nr3" "%a.nr4
> > >>>> rec = write('drylake_80v.txt',val, append)
> > >>>> i = i + 1
> > >>>> endwhile
> > >>>> *******************
> > >>>> *******************
> > >>>>
> > >>>> best regards,
> > >>>>
> > >>>> Leopoldo ÁLVAREZ
> > >>>> Tenerife - Canary Islands
> > >>>
> > >>
> > >>
> > >>
> > >> _______________________________________________
> > >> gradsusr mailing list
> > >> gradsusr at gradsusr.org
> > >> http://gradsusr.org/mailman/listinfo/gradsusr
> > >
> > > _______________________________________________
> > > gradsusr mailing list
> > > gradsusr at gradsusr.org
> > > http://gradsusr.org/mailman/listinfo/gradsusr
> >
> > --
> > Jennifer M. Adams
> > Center for Ocean-Land-Atmosphere Studies (COLA)
> > 111 Research Hall, Mail Stop 2B3
> > George Mason University
> > 4400 University Drive
> > Fairfax, VA 22030
> >
>
>
>
>
>
>
>
>
>
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
--
Jennifer M. Adams
Center for Ocean-Land-Atmosphere Studies (COLA)
111 Research Hall, Mail Stop 2B3
George Mason University
4400 University Drive
Fairfax, VA 22030
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20130924/d23d3e1a/attachment-0001.html
More information about the gradsusr
mailing list