[gradsusr] Low Level I/O Error: Seek error on data file [GrADS] -
Leopoldo Alvarez
lap.bolsa at gmail.com
Tue Sep 24 15:24:26 EDT 2013
Hi,
I tried to follow your advice and use SplitMMFiles.f for MM5, I've
compiled doing ->
xlf -qfree=f90 -convert big_endian SplitMMFiles.f
I have a file MM5 with 1 year of data, ie 8737 time steps of 1 hour.
I run in the following way ->. /a.out -s 728 MMOUT_DOMAIN3
728time steps (1 month) = 8737 hours / 12 months
It generates bad... it start to build files grouped with 7 time steps type->
Write To File Following MMOUT_DOMAIN3_36 Times
2012-01-12_04: 58:48.0000 244.98003 Hours
2012-01-12_06: 00:54.0000 246.01503 Hours
2012-01-12_07: 00:18.0000 247.00504 Hours
2012-01-12_07: 59:42.0000 247.99503 Hours
2012-01-12_08: 59:06.0000 248.98503 Hours
2012-01-12_10: 01:12.0000 250.02002 Hours
2012-01-12_11: 00:36.0000 251.01003 Hours
.....
Write To File Following MMOUT_DOMAIN3_37 Times
.....
It would have to build 12 MMOUT_DOMAIN3_12 with 728 steps each time???
not? headache!!!
Leopoldo
ok
On Sep 24, 2013, at 12:02 PM, Leopoldo Alvarez <lap.bolsa at gmail.com>
wrote:
> Jennifer,
>
> I'm going to make climatic modeling (>30 years) and I can not be
cutting files for years, buff ... it should work for powerpc with
appropriate arguments ... Can be compiled with gfortran compiler?
> This morning I opened a file of 7.7 Gb...
>
> Leopoldo
>
>
> -------- Mensaje original --------
> Asunto: [gradsusr] Low Level I/O Error: Seek error on data file
[GrADS] -
> Fecha: Tue, 24 Sep 2013 15:02:30 +0100
> De: Leopoldo Alvarez <lap.bolsa at gmail.com>
> Para: gradsusr at gradsusr.org
>
> Dear Jennifer,
>
> Now I'm more confused because I was able to open a file of 7.7 Gb
> 8173088 - 7.8G 2013-09-20 09:57 test.dat
> Why I can not open the other file ???
>
> My full GrADS descriptor file is (similar) ->
>
> dset ^test.dat
> options big_endian
> title MM5 data
> undef -9999.
> pdef 60 60 lcc 32.63 -112.54 1.00 1.00 30.00 60.00
-110.34 6000. 6000.
> xdef 149 linear -112.65 0.0270
> ydef 123 linear 32.63 0.0270
> zdef 25 levels
> 0.99689
> 0.99191
> 0.98944
> 0.98818
> 0.97377
> 0.94500
> 0.91000
> 0.87000
> 0.82500
> 0.77500
> 0.72500
> 0.67500
> 0.62500
> 0.57500
> 0.52500
> 0.47500
> 0.42500
> 0.37500
> 0.32500
> 0.27500
> 0.22500
> 0.17500
> 0.12500
> 0.07500
> 0.02500
> tdef 1036 linear 00:00Z02JAN2012 60MN
> vars 33
> rc 0 99 accum conv pcn (cm)
> rn 0 99 accum non-c pcn (cm)
> ter 0 99 ter elevation (m)
> xlat 0 99 cross lat (degree)
> xlon 0 99 cross lon (degree)
> lu 0 99 land use
> t2m 0 99 2 m temperature (K)
> q2m 0 99 2m mix ratio (kg/kg)
> u10 0 99 10 m u wind (m/sec)
> v10 0 99 10 m v wind (m/sec)
> pslv 0 99 sea level prs (mb)
> u 25 99 u wind (m/s)
> v 25 99 v wind (m/s)
> w 25 99 vertical vel (m/s)
> pp 25 99 prs pert (Pa)
> t 25 99 temperature (C)
> q 25 99 mixing ratio (kg/kg)
> clw 25 99 cloud water (kg/kg)
> rnw 25 99 rain water (kg/kg)
> rtnd 25 99 rad tend (K/day)
> z 25 99 height AGL (m)
> h 25 99 geopot height (m)
> td 25 99 dewpoint temp (C)
> rh 25 99 rel humidity (%)
> th 25 99 potential temp (K)
> the 25 99 theta-e (K)
> prs 25 99 pressure (Pa)
> vor 25 99 vorticity (s-1)
> pv 25 99 potential vort (pvu)
> dbz 25 99 reflectivity (dbz)
> div 25 99 divergence (s-1)
> dir 25 99 dir (degrees)
> tadv 25 99 temp advection (K/s)
> endvars
>
> Leopoldo
>
>
>
>
>
>
>
> Dear Jennifer,
>
> I don´t know how to fix it... :-( mmmh
>
> ga-> q config
> Config: v2.0.2 big-endian readline
> Grid Analysis and Display System (GrADS) Version 2.0.2
> Copyright (c) 1988-2011 by Brian Doty and the
> Institute for Global Environment and Society (IGES)
> This program is distributed WITHOUT ANY WARRANTY
> See file COPYRIGHT for more information.
>
> Built Thu Sep 5 12:08:35 BST 2013 for powerpc64-unknown-linux-gnu
>
> This version of GrADS has been configured with the following options:
> o Built on a BIG ENDIAN machine
> o Athena Widget GUI DISABLED
> o Command line editing ENABLED
> http://tiswww.case.edu/php/chet/readline/rltop.html
> o printim command DISABLED
> o GRIB2 interface DISABLED
> o NetCDF interface DISABLED
> o OPeNDAP gridded data interface DISABLED
> o OPeNDAP station data interface DISABLED
> o HDF4 and HDF5 interfaces DISABLED
> o GeoTIFF and KML/TIFF output DISABLED
> o KML contour output ENABLED
> o Shapefile interface DISABLED
>
> For additional information please consult http://iges.org/grads
>
> Leopoldo ÁLVAREZ
>
>
>
>
>
> Here the configuration to compile for power pc that I used ->
> grep Linux .tmpfile ; \
> if [ $$? = 0 ]; then echo "Compiling for PowerPC"
> ; \
> ( $(CD) src ; $(MAKE)
all \
> "RM = $(RM)" "RM_LIST =
> $(RM_LIST)" \
> "LN = $(LN)" "MACH =
> IBM" \
> "MAKE = $(MAKE)" "CPP =
> /lib/cpp" \
> "CPPFLAGS = -I. -C -P -DIBM -DBIT32
> -traditional" \
> "FC = xlf90 " "FCFLAGS = -qfree=f90
> -qlanglvl=90pure -I../util" \
> "LDOPTIONS = " "CFLAGS = -I.
> -DUNDERSCORE" \
> "LOCAL_LIBRARIES= " ) ; \
>
> Leopoldo
>
>
> El 23/09/13 21:35, Leopoldo Alvarez escribió:
> > Dear Jennifer and James,
> >
> > The size file that I generated with my mm5_to_grads.csh is ->
> > [login1 17] /gpfs/projects/XXXXX/XXXXXX/mesoescala/MM5toGrADS > du -k
> > test.dat
> > *68926752* 2013-09-22 11:11 test.dat
> >
> > I think it can be that grad.exe is bad compiled? Is this file
> > (test.dat) bad generated? Is it really a problem of file size? how I
> > fix it?
> > >>>> I'm on a machine that works -> uname -a
> > >>>> Linux login1 2.6.16.53-0.16-ppc64 #1 SMP Tue Oct 2 16:57:49 UTC
> > 2007 ppc64 ppc64 ppc64 GNU/Linux
> >
> > > When you are using PDEF, the xsize and ysize come from the first two
> > args in the PDEF entry. Here is my estimate of what the file size
> > should be: sfc_vars * x * y * t * 4 bytes = 11* 60 * 60 * 1036 * 4 =
> > 164102400
> > > plus
> > > z-varying-vars * z * x * y * t * 4 bytes = 22 * 25 * 60 *60 * 1036 *
> > 4 = 8205120000
> > > equals 8369222400 bytes.
> >
> > I paste another GrADS descriptor file, sorry.
> > I can see the data well, until the time of step 1064 (set t 1064, then
> > 'd t' it`s ok), but when i exceeded from 1064 time of step gives me
> > that error ...
> >
> > /ga-> set t *1065*
> > Time values set: 2012:2:15:8 2012:2:15:8
> > ga-> d t
> > Notice: Automatic Grid Interpolation Taking Place
> > Low Level I/O Error: Seek error on data file
> > Data file name = test.dat
> > 650 rc=-1 fpos=-2145713296 pfi->fhdr=0
> > Error occurred when seeking to byte 7081408
> > Data Request Error: Error for variable 't'
> > Error ocurred at column 1
> > DISPLAY error: Invalid expression
> > Expression = t/
> >
> > My full GrADS descriptor file is OK to *8737 *time values (one
year) ->
> >
> > dset ^test.dat
> > options big_endian
> > title MM5 data
> > undef -9999.
> > pdef 60 60 lcc 32.63 -112.54 1.00 1.00 30.00 60.00
> > -110.34 6000. 6000.
> > xdef 149 linear -112.65 0.0270
> > ydef 123 linear 32.63 0.0270
> > zdef 25 levels
> > 0.99689
> > 0.99191
> > 0.98944
> > 0.98818
> > 0.97377
> > 0.94500
> > 0.91000
> > 0.87000
> > 0.82500
> > 0.77500
> > 0.72500
> > 0.67500
> > 0.62500
> > 0.57500
> > 0.52500
> > 0.47500
> > 0.42500
> > 0.37500
> > 0.32500
> > 0.27500
> > 0.22500
> > 0.17500
> > 0.12500
> > 0.07500
> > 0.02500
> > tdef*8737 *linear 00:00Z02JAN2012 60MN
> > vars 33
> > rc 0 99 accum conv pcn (cm)
> > rn 0 99 accum non-c pcn (cm)
> > ter 0 99 ter elevation (m)
> > xlat 0 99 cross lat (degree)
> > xlon 0 99 cross lon (degree)
> > lu 0 99 land use
> > t2m 0 99 2 m temperature (K)
> > q2m 0 99 2m mix ratio (kg/kg)
> > u10 0 99 10 m u wind (m/sec)
> > v10 0 99 10 m v wind (m/sec)
> > pslv 0 99 sea level prs (mb)
> > u 25 99 u wind (m/s)
> > v 25 99 v wind (m/s)
> > w 25 99 vertical vel (m/s)
> > pp 25 99 prs pert (Pa)
> > t 25 99 temperature (C)
> > q 25 99 mixing ratio (kg/kg)
> > clw 25 99 cloud water (kg/kg)
> > rnw 25 99 rain water (kg/kg)
> > rtnd 25 99 rad tend (K/day)
> > z 25 99 height AGL (m)
> > h 25 99 geopot height (m)
> > td 25 99 dewpoint temp (C)
> > rh 25 99 rel humidity (%)
> > th 25 99 potential temp (K)
> > the 25 99 theta-e (K)
> > prs 25 99 pressure (Pa)
> > vor 25 99 vorticity (s-1)
> > pv 25 99 potential vort (pvu)
> > dbz 25 99 reflectivity (dbz)
> > div 25 99 divergence (s-1)
> > dir 25 99 dir (degrees)
> > tadv 25 99 temp advection (K/s)
> > endvars
> >
> >
> > Leopoldo
> >
> >
> >
> > On Sep 23, 2013, at 3:03 PM, James T. Potemra wrote:
> >
> > >
> > > Leopoldo:
> > >
> > > You have a couple problems here. First, your descriptor file
> > specifies 1036 time values, but you are trying to access t=1064 (?).
> > You should be sure about the number of time steps in your data set.
> >
> > > Second, as Jennifer suggested, x*y*z*t*4 should be your file size.
> > In this case you have 22 variables with 25 levels, and 11 with 1; 149
> > x-points; 123 y-points and 1036 time points. I think your file should
> > therefore be (22*25+11)*149*123*1036*4 = 426,063,116,368 bytes.
> >
> > When you are using PDEF, the xsize and ysize come from the first two
> > args in the PDEF entry. Here is my estimate of what the file size
> > should be:
> > sfc_vars * x * y * t * 4 bytes = 11* 60 * 60 * 1036 * 4 = 164102400
> > plus
> > z-varying-vars * z * x * y * t * 4 bytes = 22 * 25 * 60 *60 * 1036 * 4
> > = 8205120000
> > equals 8369222400 bytes.
> >
> > If you have 1064 time steps instead of 1036 then you would have a file
> > size of 8595417600 bytes.
> >
> > --Jennifer
> >
> >
> > >
> > > Jim
> > >
> > > On 9/23/13 8:24 AM, Leopoldo Alvarez wrote:
> > >> Hi Jennifer,
> > >>
> > >> >> and my data file .dat weighs 32 Gb
> > >> >A precise file size is required, down to the last byte.
> > >>
> > >> Now my data file .dat weighs 66 Gb
> > >> 68926752 - 66G 2013-09-22 11:11 test.dat
> > >>
> > >> I can see the data well, until the time of step 1064 (set t 1064,
> > then 'd t' it`s ok), but when i exceeded from 1064 time of step gives
> > me that error ...
> > >>
> > >> My full GrADS descriptor file ->
> > >>
> > >> dset ^test.dat
> > >> options big_endian
> > >> title MM5 data
> > >> undef -9999.
> > >> pdef 60 60 lcc 32.63 -112.54 1.00 1.00 30.00 60.00
> > -110.34 6000. 6000.
> > >> xdef 149 linear -112.65 0.0270
> > >> ydef 123 linear 32.63 0.0270
> > >> zdef 25 levels
> > >> 0.99689
> > >> 0.99191
> > >> 0.98944
> > >> 0.98818
> > >> 0.97377
> > >> 0.94500
> > >> 0.91000
> > >> 0.87000
> > >> 0.82500
> > >> 0.77500
> > >> 0.72500
> > >> 0.67500
> > >> 0.62500
> > >> 0.57500
> > >> 0.52500
> > >> 0.47500
> > >> 0.42500
> > >> 0.37500
> > >> 0.32500
> > >> 0.27500
> > >> 0.22500
> > >> 0.17500
> > >> 0.12500
> > >> 0.07500
> > >> 0.02500
> > >> tdef 1036 linear 00:00Z02JAN2012 60MN
> > >> vars 33
> > >> rc 0 99 accum conv pcn (cm)
> > >> rn 0 99 accum non-c pcn (cm)
> > >> ter 0 99 ter elevation (m)
> > >> xlat 0 99 cross lat (degree)
> > >> xlon 0 99 cross lon (degree)
> > >> lu 0 99 land use
> > >> t2m 0 99 2 m temperature (K)
> > >> q2m 0 99 2m mix ratio (kg/kg)
> > >> u10 0 99 10 m u wind (m/sec)
> > >> v10 0 99 10 m v wind (m/sec)
> > >> pslv 0 99 sea level prs (mb)
> > >> u 25 99 u wind (m/s)
> > >> v 25 99 v wind (m/s)
> > >> w 25 99 vertical vel (m/s)
> > >> pp 25 99 prs pert (Pa)
> > >> t 25 99 temperature (C)
> > >> q 25 99 mixing ratio (kg/kg)
> > >> clw 25 99 cloud water (kg/kg)
> > >> rnw 25 99 rain water (kg/kg)
> > >> rtnd 25 99 rad tend (K/day)
> > >> z 25 99 height AGL (m)
> > >> h 25 99 geopot height (m)
> > >> td 25 99 dewpoint temp (C)
> > >> rh 25 99 rel humidity (%)
> > >> th 25 99 potential temp (K)
> > >> the 25 99 theta-e (K)
> > >> prs 25 99 pressure (Pa)
> > >> vor 25 99 vorticity (s-1)
> > >> pv 25 99 potential vort (pvu)
> > >> dbz 25 99 reflectivity (dbz)
> > >> div 25 99 divergence (s-1)
> > >> dir 25 99 dir (degrees)
> > >> tadv 25 99 temp advection (K/s)
> > >> endvars
> > >>
> > >>
> > >> El 21/09/13 09:35, Leopoldo Alvarez escribió:
> > >>> Hi Jennifer,
> > >>>
> > >>> My binary data grad.exe file is 210 Kb. and data file .dat weighs
> > 32 Gb.
> > >>> xsize*ysize*zsize*tsize*esize*4 , Where I see these parameters? i
> > don´t know how to fix it, where ?, I need to recompile ?.
> > >>>
> > >>> best regards,
> > >>>
> > >>> Leopoldo ÁLVAREZ
> > >>> Tenerife - Canary Islands
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> El 20/09/13 23:13, Leopoldo Alvarez escribió:
> > >>>> Hi jennifer,
> > >>>>
> > >>>> I am running a script (nuevo.gs) to extract info domain mm5 model
> > data. I use GrADS 2.0.2 version. I use 'options big_endian' but my
> > data file weighs 32 GB and
> > >>>> shows multiple mutiple on screen- ->
> > >>>>
> > >>>> Low Level I/O Error: Seek error on data file
> > >>>> Data file name = test.dat
> > >>>> 650 rc=-1 fpos=-172300 pfi->fhdr=0
> > >>>>
> > >>>> I think the problem is when running the mm5_to_grads.csh and
> > creates the .ctl and .dat and not script. I can see the data well,
> > until the time of step 1062 (set t 1062, then 'd t' it`s ok), but when
> > i exceeded from 1062 time of step gives me that error ... How i can
> > run my script saving this problem???. One clue
> > http://gradsusr.org/pipermail/gradsusr/2008-April/006883.html
> > >>>>
> > >>>> I'm on a machine that works -> uname -a
> > >>>> Linux login1 2.6.16.53-0.16-ppc64 #1 SMP Tue Oct 2 16:57:49 UTC
> > 2007 ppc64 ppc64 ppc64 GNU/Linux
> > >>>>
> > >>>>
> > >>>> nuevo.gs script
> > >>>> ***************
> > >>>> ***************
> > >>>> 'set lon -110.1829'
> > >>>> 'set lat 34.6518'
> > >>>> 'set lev 0.99199'
> > >>>>
> > >>>> #8737
> > >>>> timeend = 4198
> > >>>> i = 1
> > >>>> while(i <= timeend)
> > >>>> 'set t 'i''
> > >>>> 'q time'
> > >>>> timehr = substr(result, 8, 3)
> > >>>> timedy = substr(result, 11, 2)
> > >>>> timemo = substr(result, 13, 3)
> > >>>> timeyr = substr(result, 16, 4)
> > >>>>
> > >>>> #velocidad viento
> > >>>> 'tg1=mag(u,v)'
> > >>>> 'd tg1'
> > >>>> a.nr1 = subwrd(result,4)
> > >>>> #direccion viento
> > >>>> 'd dir'
> > >>>> a.nr11 = subwrd(result,10)
> > >>>> # tv de Jeffrey iowa
> > >>>> 'tv=t+t*0.61*q'
> > >>>> 'rho=prs/(286.9*tv)'
> > >>>> 'd rho'
> > >>>> a.nr = subwrd(result,4)
> > >>>> 'd t-273.15'
> > >>>> a.nr2 = subwrd(result,10)
> > >>>> 'd rh'
> > >>>> a.nr3 = subwrd(result,10)
> > >>>> 'd prs/100'
> > >>>> a.nr4 = subwrd(result,10)
> > >>>>
> > >>>> val = timeyr%"-"%timemo%"-"%timedy"-"%timehr" "%a.nr1" "%a.nr11"
> > "%a.nr" "%a.nr2" "%a.nr3" "%a.nr4
> > >>>> rec = write('drylake_80v.txt',val, append)
> > >>>> i = i + 1
> > >>>> endwhile
> > >>>> *******************
> > >>>> *******************
> > >>>>
> > >>>> best regards,
> > >>>>
> > >>>> Leopoldo ÁLVAREZ
> > >>>> Tenerife - Canary Islands
> > >>>
> > >>
> > >>
> > >>
> > >> _______________________________________________
> > >> gradsusr mailing list
> > >> gradsusr at gradsusr.org
> > >> http://gradsusr.org/mailman/listinfo/gradsusr
> > >
> > > _______________________________________________
> > > gradsusr mailing list
> > > gradsusr at gradsusr.org
> > > http://gradsusr.org/mailman/listinfo/gradsusr
> >
> > --
> > Jennifer M. Adams
> > Center for Ocean-Land-Atmosphere Studies (COLA)
> > 111 Research Hall, Mail Stop 2B3
> > George Mason University
> > 4400 University Drive
> > Fairfax, VA 22030
> >
>
>
>
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20130924/619b5f19/attachment-0001.html
More information about the gradsusr
mailing list