[gradsusr] Fwd: Low Level I/O Error: Seek error on data file [GrADS] -
Leopoldo Alvarez
lap.bolsa at gmail.com
Tue Sep 24 14:02:23 EDT 2013
Jennifer,
I'm going to make climatic modeling (>30 years) and I can not be cutting
files for years, buff ... it should work for powerpc with appropriate
arguments ... Can be compiled with gfortran compiler?
This morning I opened a file of 7.7 Gb...
Leopoldo
-------- Mensaje original --------
Asunto: [gradsusr] Low Level I/O Error: Seek error on data file [GrADS] -
Fecha: Tue, 24 Sep 2013 15:02:30 +0100
De: Leopoldo Alvarez <lap.bolsa at gmail.com>
Para: gradsusr at gradsusr.org
Dear Jennifer,
Now I'm more confused because I was able to open a file of 7.7 Gb
8173088 - 7.8G 2013-09-20 09:57 test.dat
Why I can not open the other file ???
My full GrADS descriptor file is (similar) ->
dset ^test.dat
options big_endian
title MM5 data
undef -9999.
pdef 60 60 lcc 32.63 -112.54 1.00 1.00 30.00 60.00
-110.34 6000. 6000.
xdef 149 linear -112.65 0.0270
ydef 123 linear 32.63 0.0270
zdef 25 levels
0.99689
0.99191
0.98944
0.98818
0.97377
0.94500
0.91000
0.87000
0.82500
0.77500
0.72500
0.67500
0.62500
0.57500
0.52500
0.47500
0.42500
0.37500
0.32500
0.27500
0.22500
0.17500
0.12500
0.07500
0.02500
tdef 1036 linear 00:00Z02JAN2012 60MN
vars 33
rc 0 99 accum conv pcn (cm)
rn 0 99 accum non-c pcn (cm)
ter 0 99 ter elevation (m)
xlat 0 99 cross lat (degree)
xlon 0 99 cross lon (degree)
lu 0 99 land use
t2m 0 99 2 m temperature (K)
q2m 0 99 2m mix ratio (kg/kg)
u10 0 99 10 m u wind (m/sec)
v10 0 99 10 m v wind (m/sec)
pslv 0 99 sea level prs (mb)
u 25 99 u wind (m/s)
v 25 99 v wind (m/s)
w 25 99 vertical vel (m/s)
pp 25 99 prs pert (Pa)
t 25 99 temperature (C)
q 25 99 mixing ratio (kg/kg)
clw 25 99 cloud water (kg/kg)
rnw 25 99 rain water (kg/kg)
rtnd 25 99 rad tend (K/day)
z 25 99 height AGL (m)
h 25 99 geopot height (m)
td 25 99 dewpoint temp (C)
rh 25 99 rel humidity (%)
th 25 99 potential temp (K)
the 25 99 theta-e (K)
prs 25 99 pressure (Pa)
vor 25 99 vorticity (s-1)
pv 25 99 potential vort (pvu)
dbz 25 99 reflectivity (dbz)
div 25 99 divergence (s-1)
dir 25 99 dir (degrees)
tadv 25 99 temp advection (K/s)
endvars
Leopoldo
Dear Jennifer,
I don´t know how to fix it... :-( mmmh
ga-> q config
Config: v2.0.2 big-endian readline
Grid Analysis and Display System (GrADS) Version 2.0.2
Copyright (c) 1988-2011 by Brian Doty and the
Institute for Global Environment and Society (IGES)
This program is distributed WITHOUT ANY WARRANTY
See file COPYRIGHT for more information.
Built Thu Sep 5 12:08:35 BST 2013 for powerpc64-unknown-linux-gnu
This version of GrADS has been configured with the following options:
o Built on a BIG ENDIAN machine
o Athena Widget GUI DISABLED
o Command line editing ENABLED
http://tiswww.case.edu/php/chet/readline/rltop.html
o printim command DISABLED
o GRIB2 interface DISABLED
o NetCDF interface DISABLED
o OPeNDAP gridded data interface DISABLED
o OPeNDAP station data interface DISABLED
o HDF4 and HDF5 interfaces DISABLED
o GeoTIFF and KML/TIFF output DISABLED
o KML contour output ENABLED
o Shapefile interface DISABLED
For additional information please consult http://iges.org/grads
Leopoldo ÁLVAREZ
Here the configuration to compile for power pc that I used ->
grep Linux .tmpfile ; \
if [ $$? = 0 ]; then echo "Compiling for PowerPC"
; \
( $(CD) src ; $(MAKE) all \
"RM = $(RM)" "RM_LIST =
$(RM_LIST)" \
"LN = $(LN)" "MACH =
IBM" \
"MAKE = $(MAKE)" "CPP =
/lib/cpp" \
"CPPFLAGS = -I. -C -P -DIBM -DBIT32
-traditional" \
"FC = xlf90 " "FCFLAGS = -qfree=f90
-qlanglvl=90pure -I../util" \
"LDOPTIONS = " "CFLAGS = -I.
-DUNDERSCORE" \
"LOCAL_LIBRARIES= " ) ; \
Leopoldo
El 23/09/13 21:35, Leopoldo Alvarez escribió:
> Dear Jennifer and James,
>
> The size file that I generated with my mm5_to_grads.csh is ->
> [login1 17] /gpfs/projects/XXXXX/XXXXXX/mesoescala/MM5toGrADS > du -k
> test.dat
> *68926752* 2013-09-22 11:11 test.dat
>
> I think it can be that grad.exe is bad compiled? Is this file
> (test.dat) bad generated? Is it really a problem of file size? how I
> fix it?
> >>>> I'm on a machine that works -> uname -a
> >>>> Linux login1 2.6.16.53-0.16-ppc64 #1 SMP Tue Oct 2 16:57:49 UTC
> 2007 ppc64 ppc64 ppc64 GNU/Linux
>
> > When you are using PDEF, the xsize and ysize come from the first two
> args in the PDEF entry. Here is my estimate of what the file size
> should be: sfc_vars * x * y * t * 4 bytes = 11* 60 * 60 * 1036 * 4 =
> 164102400
> > plus
> > z-varying-vars * z * x * y * t * 4 bytes = 22 * 25 * 60 *60 * 1036 *
> 4 = 8205120000
> > equals 8369222400 bytes.
>
> I paste another GrADS descriptor file, sorry.
> I can see the data well, until the time of step 1064 (set t 1064, then
> 'd t' it`s ok), but when i exceeded from 1064 time of step gives me
> that error ...
>
> /ga-> set t *1065*
> Time values set: 2012:2:15:8 2012:2:15:8
> ga-> d t
> Notice: Automatic Grid Interpolation Taking Place
> Low Level I/O Error: Seek error on data file
> Data file name = test.dat
> 650 rc=-1 fpos=-2145713296 pfi->fhdr=0
> Error occurred when seeking to byte 7081408
> Data Request Error: Error for variable 't'
> Error ocurred at column 1
> DISPLAY error: Invalid expression
> Expression = t/
>
> My full GrADS descriptor file is OK to *8737 *time values (one year) ->
>
> dset ^test.dat
> options big_endian
> title MM5 data
> undef -9999.
> pdef 60 60 lcc 32.63 -112.54 1.00 1.00 30.00 60.00
> -110.34 6000. 6000.
> xdef 149 linear -112.65 0.0270
> ydef 123 linear 32.63 0.0270
> zdef 25 levels
> 0.99689
> 0.99191
> 0.98944
> 0.98818
> 0.97377
> 0.94500
> 0.91000
> 0.87000
> 0.82500
> 0.77500
> 0.72500
> 0.67500
> 0.62500
> 0.57500
> 0.52500
> 0.47500
> 0.42500
> 0.37500
> 0.32500
> 0.27500
> 0.22500
> 0.17500
> 0.12500
> 0.07500
> 0.02500
> tdef*8737 *linear 00:00Z02JAN2012 60MN
> vars 33
> rc 0 99 accum conv pcn (cm)
> rn 0 99 accum non-c pcn (cm)
> ter 0 99 ter elevation (m)
> xlat 0 99 cross lat (degree)
> xlon 0 99 cross lon (degree)
> lu 0 99 land use
> t2m 0 99 2 m temperature (K)
> q2m 0 99 2m mix ratio (kg/kg)
> u10 0 99 10 m u wind (m/sec)
> v10 0 99 10 m v wind (m/sec)
> pslv 0 99 sea level prs (mb)
> u 25 99 u wind (m/s)
> v 25 99 v wind (m/s)
> w 25 99 vertical vel (m/s)
> pp 25 99 prs pert (Pa)
> t 25 99 temperature (C)
> q 25 99 mixing ratio (kg/kg)
> clw 25 99 cloud water (kg/kg)
> rnw 25 99 rain water (kg/kg)
> rtnd 25 99 rad tend (K/day)
> z 25 99 height AGL (m)
> h 25 99 geopot height (m)
> td 25 99 dewpoint temp (C)
> rh 25 99 rel humidity (%)
> th 25 99 potential temp (K)
> the 25 99 theta-e (K)
> prs 25 99 pressure (Pa)
> vor 25 99 vorticity (s-1)
> pv 25 99 potential vort (pvu)
> dbz 25 99 reflectivity (dbz)
> div 25 99 divergence (s-1)
> dir 25 99 dir (degrees)
> tadv 25 99 temp advection (K/s)
> endvars
>
>
> Leopoldo
>
>
>
> On Sep 23, 2013, at 3:03 PM, James T. Potemra wrote:
>
> >
> > Leopoldo:
> >
> > You have a couple problems here. First, your descriptor file
> specifies 1036 time values, but you are trying to access t=1064 (?).
> You should be sure about the number of time steps in your data set.
>
> > Second, as Jennifer suggested, x*y*z*t*4 should be your file size.
> In this case you have 22 variables with 25 levels, and 11 with 1; 149
> x-points; 123 y-points and 1036 time points. I think your file should
> therefore be (22*25+11)*149*123*1036*4 = 426,063,116,368 bytes.
>
> When you are using PDEF, the xsize and ysize come from the first two
> args in the PDEF entry. Here is my estimate of what the file size
> should be:
> sfc_vars * x * y * t * 4 bytes = 11* 60 * 60 * 1036 * 4 = 164102400
> plus
> z-varying-vars * z * x * y * t * 4 bytes = 22 * 25 * 60 *60 * 1036 * 4
> = 8205120000
> equals 8369222400 bytes.
>
> If you have 1064 time steps instead of 1036 then you would have a file
> size of 8595417600 bytes.
>
> --Jennifer
>
>
> >
> > Jim
> >
> > On 9/23/13 8:24 AM, Leopoldo Alvarez wrote:
> >> Hi Jennifer,
> >>
> >> >> and my data file .dat weighs 32 Gb
> >> >A precise file size is required, down to the last byte.
> >>
> >> Now my data file .dat weighs 66 Gb
> >> 68926752 - 66G 2013-09-22 11:11 test.dat
> >>
> >> I can see the data well, until the time of step 1064 (set t 1064,
> then 'd t' it`s ok), but when i exceeded from 1064 time of step gives
> me that error ...
> >>
> >> My full GrADS descriptor file ->
> >>
> >> dset ^test.dat
> >> options big_endian
> >> title MM5 data
> >> undef -9999.
> >> pdef 60 60 lcc 32.63 -112.54 1.00 1.00 30.00 60.00
> -110.34 6000. 6000.
> >> xdef 149 linear -112.65 0.0270
> >> ydef 123 linear 32.63 0.0270
> >> zdef 25 levels
> >> 0.99689
> >> 0.99191
> >> 0.98944
> >> 0.98818
> >> 0.97377
> >> 0.94500
> >> 0.91000
> >> 0.87000
> >> 0.82500
> >> 0.77500
> >> 0.72500
> >> 0.67500
> >> 0.62500
> >> 0.57500
> >> 0.52500
> >> 0.47500
> >> 0.42500
> >> 0.37500
> >> 0.32500
> >> 0.27500
> >> 0.22500
> >> 0.17500
> >> 0.12500
> >> 0.07500
> >> 0.02500
> >> tdef 1036 linear 00:00Z02JAN2012 60MN
> >> vars 33
> >> rc 0 99 accum conv pcn (cm)
> >> rn 0 99 accum non-c pcn (cm)
> >> ter 0 99 ter elevation (m)
> >> xlat 0 99 cross lat (degree)
> >> xlon 0 99 cross lon (degree)
> >> lu 0 99 land use
> >> t2m 0 99 2 m temperature (K)
> >> q2m 0 99 2m mix ratio (kg/kg)
> >> u10 0 99 10 m u wind (m/sec)
> >> v10 0 99 10 m v wind (m/sec)
> >> pslv 0 99 sea level prs (mb)
> >> u 25 99 u wind (m/s)
> >> v 25 99 v wind (m/s)
> >> w 25 99 vertical vel (m/s)
> >> pp 25 99 prs pert (Pa)
> >> t 25 99 temperature (C)
> >> q 25 99 mixing ratio (kg/kg)
> >> clw 25 99 cloud water (kg/kg)
> >> rnw 25 99 rain water (kg/kg)
> >> rtnd 25 99 rad tend (K/day)
> >> z 25 99 height AGL (m)
> >> h 25 99 geopot height (m)
> >> td 25 99 dewpoint temp (C)
> >> rh 25 99 rel humidity (%)
> >> th 25 99 potential temp (K)
> >> the 25 99 theta-e (K)
> >> prs 25 99 pressure (Pa)
> >> vor 25 99 vorticity (s-1)
> >> pv 25 99 potential vort (pvu)
> >> dbz 25 99 reflectivity (dbz)
> >> div 25 99 divergence (s-1)
> >> dir 25 99 dir (degrees)
> >> tadv 25 99 temp advection (K/s)
> >> endvars
> >>
> >>
> >> El 21/09/13 09:35, Leopoldo Alvarez escribió:
> >>> Hi Jennifer,
> >>>
> >>> My binary data grad.exe file is 210 Kb. and data file .dat weighs
> 32 Gb.
> >>> xsize*ysize*zsize*tsize*esize*4 , Where I see these parameters? i
> don´t know how to fix it, where ?, I need to recompile ?.
> >>>
> >>> best regards,
> >>>
> >>> Leopoldo ÁLVAREZ
> >>> Tenerife - Canary Islands
> >>>
> >>>
> >>>
> >>>
> >>> El 20/09/13 23:13, Leopoldo Alvarez escribió:
> >>>> Hi jennifer,
> >>>>
> >>>> I am running a script (nuevo.gs) to extract info domain mm5 model
> data. I use GrADS 2.0.2 version. I use 'options big_endian' but my
> data file weighs 32 GB and
> >>>> shows multiple mutiple on screen- ->
> >>>>
> >>>> Low Level I/O Error: Seek error on data file
> >>>> Data file name = test.dat
> >>>> 650 rc=-1 fpos=-172300 pfi->fhdr=0
> >>>>
> >>>> I think the problem is when running the mm5_to_grads.csh and
> creates the .ctl and .dat and not script. I can see the data well,
> until the time of step 1062 (set t 1062, then 'd t' it`s ok), but when
> i exceeded from 1062 time of step gives me that error ... How i can
> run my script saving this problem???. One clue
> http://gradsusr.org/pipermail/gradsusr/2008-April/006883.html
> >>>>
> >>>> I'm on a machine that works -> uname -a
> >>>> Linux login1 2.6.16.53-0.16-ppc64 #1 SMP Tue Oct 2 16:57:49 UTC
> 2007 ppc64 ppc64 ppc64 GNU/Linux
> >>>>
> >>>>
> >>>> nuevo.gs script
> >>>> ***************
> >>>> ***************
> >>>> 'set lon -110.1829'
> >>>> 'set lat 34.6518'
> >>>> 'set lev 0.99199'
> >>>>
> >>>> #8737
> >>>> timeend = 4198
> >>>> i = 1
> >>>> while(i <= timeend)
> >>>> 'set t 'i''
> >>>> 'q time'
> >>>> timehr = substr(result, 8, 3)
> >>>> timedy = substr(result, 11, 2)
> >>>> timemo = substr(result, 13, 3)
> >>>> timeyr = substr(result, 16, 4)
> >>>>
> >>>> #velocidad viento
> >>>> 'tg1=mag(u,v)'
> >>>> 'd tg1'
> >>>> a.nr1 = subwrd(result,4)
> >>>> #direccion viento
> >>>> 'd dir'
> >>>> a.nr11 = subwrd(result,10)
> >>>> # tv de Jeffrey iowa
> >>>> 'tv=t+t*0.61*q'
> >>>> 'rho=prs/(286.9*tv)'
> >>>> 'd rho'
> >>>> a.nr = subwrd(result,4)
> >>>> 'd t-273.15'
> >>>> a.nr2 = subwrd(result,10)
> >>>> 'd rh'
> >>>> a.nr3 = subwrd(result,10)
> >>>> 'd prs/100'
> >>>> a.nr4 = subwrd(result,10)
> >>>>
> >>>> val = timeyr%"-"%timemo%"-"%timedy"-"%timehr" "%a.nr1" "%a.nr11"
> "%a.nr" "%a.nr2" "%a.nr3" "%a.nr4
> >>>> rec = write('drylake_80v.txt',val, append)
> >>>> i = i + 1
> >>>> endwhile
> >>>> *******************
> >>>> *******************
> >>>>
> >>>> best regards,
> >>>>
> >>>> Leopoldo ÁLVAREZ
> >>>> Tenerife - Canary Islands
> >>>
> >>
> >>
> >>
> >> _______________________________________________
> >> gradsusr mailing list
> >> gradsusr at gradsusr.org
> >> http://gradsusr.org/mailman/listinfo/gradsusr
> >
> > _______________________________________________
> > gradsusr mailing list
> > gradsusr at gradsusr.org
> > http://gradsusr.org/mailman/listinfo/gradsusr
>
> --
> Jennifer M. Adams
> Center for Ocean-Land-Atmosphere Studies (COLA)
> 111 Research Hall, Mail Stop 2B3
> George Mason University
> 4400 University Drive
> Fairfax, VA 22030
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20130924/d87e05a1/attachment-0001.html
More information about the gradsusr
mailing list