[gradsusr] Low Level I/O Error: Seek error on data file [GrADS] -

Leopoldo Alvarez lap.bolsa at gmail.com
Mon Sep 23 16:39:31 EDT 2013


Hi Matias,

I could do this calculation before without problems. A year or more 
data. I think now is the architecture I use.
Linux login1 2.6.16.53-0.16-ppc64 #1 SMP Tue Oct 2 16:57:49 UTC 2007 
ppc64 ppc64 ppc64 GNU/Linux
The GrADS can handle files larger than 2 GB in size.

Leopoldo




 > Hi Leopoldo,
 >
 > One solution could be to split big MM5 output file into smaller 
files.  You
 > can do this with the following fortran program:

 > ftp://ftp.ucar.edu/mesouser/MM5V3/Util/SplitMMFiles.f

 > Then you execute mm5_to_grads to each smaller files.

 > I hope this helps you.

 > Regards
 > Matias




El 23/09/13 21:35, Leopoldo Alvarez escribió:
> Dear Jennifer and James,
>
> The size file that I generated with my mm5_to_grads.csh  is ->
> [login1 17] /gpfs/projects/XXXXX/XXXXXX/mesoescala/MM5toGrADS > du -k 
> test.dat
> *68926752* 2013-09-22 11:11 test.dat
>
> I think it can be that grad.exe is bad compiled?  Is this file 
> (test.dat) bad generated? Is it really a problem of file size? how I 
> fix it?
> >>>> I'm on a machine that works -> uname -a
> >>>> Linux login1 2.6.16.53-0.16-ppc64 #1 SMP Tue Oct 2 16:57:49 UTC 
> 2007 ppc64 ppc64 ppc64 GNU/Linux
>
> > When you are using PDEF, the xsize and ysize come from the first two 
> args in the PDEF entry. Here is my estimate of what the file size 
> should be: sfc_vars * x * y * t *  4 bytes = 11* 60 * 60 * 1036 * 4 = 
> 164102400
> > plus
> > z-varying-vars * z * x * y * t * 4 bytes = 22 * 25 * 60 *60 * 1036 * 
> 4 = 8205120000
> > equals 8369222400 bytes.
>
> I paste another GrADS descriptor file, sorry.
> I can see the data well, until the time of step 1064 (set t 1064, then 
> 'd t' it`s ok), but when i exceeded from 1064 time of step gives me 
> that error ...
>
> /ga-> set t *1065*
> Time values set: 2012:2:15:8 2012:2:15:8
> ga-> d t
> Notice:  Automatic Grid Interpolation Taking Place
> Low Level I/O Error:  Seek error on data file
>   Data file name = test.dat
> 650 rc=-1 fpos=-2145713296 pfi->fhdr=0
>   Error occurred when seeking to byte 7081408
> Data Request Error:  Error for variable 't'
>   Error ocurred at column 1
> DISPLAY error:  Invalid expression
>   Expression = t/
>
> My full GrADS descriptor file is OK to *8737 *time values (one year) ->
>
> dset ^test.dat
> options big_endian
> title MM5 data
> undef -9999.
> pdef   60   60 lcc   32.63 -112.54    1.00    1.00   30.00 60.00 
> -110.34   6000.   6000.
> xdef  149 linear -112.65  0.0270
> ydef  123 linear   32.63  0.0270
> zdef  25 levels
>  0.99689
>  0.99191
>  0.98944
>  0.98818
>  0.97377
>  0.94500
>  0.91000
>  0.87000
>  0.82500
>  0.77500
>  0.72500
>  0.67500
>  0.62500
>  0.57500
>  0.52500
>  0.47500
>  0.42500
>  0.37500
>  0.32500
>  0.27500
>  0.22500
>  0.17500
>  0.12500
>  0.07500
>  0.02500
> tdef*8737 *linear 00:00Z02JAN2012   60MN
> vars 33
> rc       0 99 accum conv pcn (cm)
> rn       0 99 accum non-c pcn (cm)
> ter      0 99 ter elevation (m)
> xlat     0 99 cross lat (degree)
> xlon     0 99 cross lon (degree)
> lu       0 99 land use
> t2m      0 99 2 m temperature (K)
> q2m      0 99 2m mix ratio (kg/kg)
> u10      0 99 10 m u wind (m/sec)
> v10      0 99 10 m v wind (m/sec)
> pslv     0 99 sea level prs (mb)
> u         25 99 u wind (m/s)
> v         25 99 v wind (m/s)
> w         25 99 vertical vel (m/s)
> pp        25 99 prs pert (Pa)
> t         25 99 temperature (C)
> q         25 99 mixing ratio (kg/kg)
> clw       25 99 cloud water (kg/kg)
> rnw       25 99 rain water (kg/kg)
> rtnd      25 99 rad tend (K/day)
> z         25 99 height AGL (m)
> h         25 99 geopot height (m)
> td        25 99 dewpoint temp (C)
> rh        25 99 rel humidity (%)
> th        25 99 potential temp (K)
> the       25 99 theta-e (K)
> prs       25 99 pressure (Pa)
> vor       25 99 vorticity (s-1)
> pv        25 99 potential vort (pvu)
> dbz       25 99 reflectivity (dbz)
> div       25 99 divergence (s-1)
> dir       25 99 dir (degrees)
> tadv      25 99 temp advection (K/s)
> endvars
>
>
> Leopoldo
>
>
>
> On Sep 23, 2013, at 3:03 PM, James T. Potemra wrote:
>
> >
> > Leopoldo:
> >
> > You have a couple problems here.  First, your descriptor file 
> specifies 1036 time values, but you are trying to access t=1064 (?).
> You should be sure about the number of time steps in your data set.
>
> > Second, as Jennifer suggested, x*y*z*t*4 should be your file size.  
> In this case you have 22 variables with 25 levels, and 11 with 1; 149 
> x-points; 123 y-points and 1036 time points. I think your file should 
> therefore be (22*25+11)*149*123*1036*4 = 426,063,116,368 bytes.
>
> When you are using PDEF, the xsize and ysize come from the first two 
> args in the PDEF entry. Here is my estimate of what the file size 
> should be:
> sfc_vars * x * y * t *  4 bytes = 11* 60 * 60 * 1036 * 4 = 164102400
> plus
> z-varying-vars * z * x * y * t * 4 bytes = 22 * 25 * 60 *60 * 1036 * 4 
> = 8205120000
> equals 8369222400 bytes.
>
> If you have 1064 time steps instead of 1036 then you would have a file 
> size of 8595417600 bytes.
>
> --Jennifer
>
>
> >
> > Jim
> >
> > On 9/23/13 8:24 AM, Leopoldo Alvarez wrote:
> >> Hi Jennifer,
> >>
> >> >> and my data file .dat weighs 32 Gb
> >> >A precise file size is required, down to the last byte.
> >>
> >> Now my data file .dat weighs 66 Gb
> >> 68926752 - 66G 2013-09-22 11:11 test.dat
> >>
> >> I can see the data well, until the time of step 1064 (set t 1064, 
> then 'd t' it`s ok), but when i exceeded from 1064 time of step gives 
> me that error ...
> >>
> >> My full GrADS descriptor file ->
> >>
> >> dset ^test.dat
> >> options big_endian
> >> title MM5 data
> >> undef -9999.
> >> pdef   60   60 lcc   32.63 -112.54    1.00    1.00 30.00   60.00 
> -110.34   6000.   6000.
> >> xdef  149 linear -112.65  0.0270
> >> ydef  123 linear   32.63  0.0270
> >> zdef  25 levels
> >>  0.99689
> >>  0.99191
> >>  0.98944
> >>  0.98818
> >>  0.97377
> >>  0.94500
> >>  0.91000
> >>  0.87000
> >>  0.82500
> >>  0.77500
> >>  0.72500
> >>  0.67500
> >>  0.62500
> >>  0.57500
> >>  0.52500
> >>  0.47500
> >>  0.42500
> >>  0.37500
> >>  0.32500
> >>  0.27500
> >>  0.22500
> >>  0.17500
> >>  0.12500
> >>  0.07500
> >>  0.02500
> >> tdef 1036 linear 00:00Z02JAN2012   60MN
> >> vars 33
> >> rc       0 99 accum conv pcn (cm)
> >> rn       0 99 accum non-c pcn (cm)
> >> ter      0 99 ter elevation (m)
> >> xlat     0 99 cross lat (degree)
> >> xlon     0 99 cross lon (degree)
> >> lu       0 99 land use
> >> t2m      0 99 2 m temperature (K)
> >> q2m      0 99 2m mix ratio (kg/kg)
> >> u10      0 99 10 m u wind (m/sec)
> >> v10      0 99 10 m v wind (m/sec)
> >> pslv     0 99 sea level prs (mb)
> >> u         25 99 u wind (m/s)
> >> v         25 99 v wind (m/s)
> >> w         25 99 vertical vel (m/s)
> >> pp        25 99 prs pert (Pa)
> >> t         25 99 temperature (C)
> >> q         25 99 mixing ratio (kg/kg)
> >> clw       25 99 cloud water (kg/kg)
> >> rnw       25 99 rain water (kg/kg)
> >> rtnd      25 99 rad tend (K/day)
> >> z         25 99 height AGL (m)
> >> h         25 99 geopot height (m)
> >> td        25 99 dewpoint temp (C)
> >> rh        25 99 rel humidity (%)
> >> th        25 99 potential temp (K)
> >> the       25 99 theta-e (K)
> >> prs       25 99 pressure (Pa)
> >> vor       25 99 vorticity (s-1)
> >> pv        25 99 potential vort (pvu)
> >> dbz       25 99 reflectivity (dbz)
> >> div       25 99 divergence (s-1)
> >> dir       25 99 dir (degrees)
> >> tadv      25 99 temp advection (K/s)
> >> endvars
> >>
> >>
> >> El 21/09/13 09:35, Leopoldo Alvarez escribió:
> >>> Hi Jennifer,
> >>>
> >>> My binary data grad.exe file is 210 Kb. and data file .dat weighs 
> 32 Gb.
> >>> xsize*ysize*zsize*tsize*esize*4 , Where I see these parameters? i 
> don´t know how to fix it, where ?, I need to recompile ?.
> >>>
> >>> best regards,
> >>>
> >>> Leopoldo ÁLVAREZ
> >>> Tenerife - Canary Islands
> >>>
> >>>
> >>>
> >>>
> >>> El 20/09/13 23:13, Leopoldo Alvarez escribió:
> >>>> Hi jennifer,
> >>>>
> >>>> I am running a script (nuevo.gs) to extract info domain mm5 model 
> data. I use GrADS 2.0.2 version. I use 'options big_endian' but my 
> data file weighs 32 GB and
> >>>> shows multiple mutiple on screen- ->
> >>>>
> >>>> Low Level I/O Error:  Seek error on data file
> >>>>   Data file name = test.dat
> >>>> 650 rc=-1 fpos=-172300 pfi->fhdr=0
> >>>>
> >>>> I think the problem is when running the mm5_to_grads.csh and 
> creates the .ctl and .dat and not script. I can see the data well, 
> until the time of step 1062 (set t 1062, then 'd t' it`s ok), but when 
> i exceeded from 1062 time of step gives me that error ... How i can 
> run my script saving this problem???. One clue 
> http://gradsusr.org/pipermail/gradsusr/2008-April/006883.html
> >>>>
> >>>> I'm on a machine that works -> uname -a
> >>>> Linux login1 2.6.16.53-0.16-ppc64 #1 SMP Tue Oct 2 16:57:49 UTC 
> 2007 ppc64 ppc64 ppc64 GNU/Linux
> >>>>
> >>>>
> >>>> nuevo.gs script
> >>>> ***************
> >>>> ***************
> >>>> 'set lon -110.1829'
> >>>> 'set lat 34.6518'
> >>>> 'set lev 0.99199'
> >>>>
> >>>> #8737
> >>>> timeend = 4198
> >>>> i = 1
> >>>> while(i <= timeend)
> >>>> 'set t 'i''
> >>>> 'q time'
> >>>> timehr = substr(result, 8, 3)
> >>>> timedy = substr(result, 11, 2)
> >>>> timemo = substr(result, 13, 3)
> >>>> timeyr = substr(result, 16, 4)
> >>>>
> >>>> #velocidad viento
> >>>> 'tg1=mag(u,v)'
> >>>> 'd tg1'
> >>>> a.nr1 = subwrd(result,4)
> >>>> #direccion viento
> >>>> 'd dir'
> >>>> a.nr11 = subwrd(result,10)
> >>>> # tv de Jeffrey iowa
> >>>> 'tv=t+t*0.61*q'
> >>>> 'rho=prs/(286.9*tv)'
> >>>> 'd rho'
> >>>> a.nr = subwrd(result,4)
> >>>> 'd t-273.15'
> >>>> a.nr2 = subwrd(result,10)
> >>>> 'd rh'
> >>>> a.nr3 = subwrd(result,10)
> >>>> 'd prs/100'
> >>>> a.nr4 = subwrd(result,10)
> >>>>
> >>>> val = timeyr%"-"%timemo%"-"%timedy"-"%timehr" "%a.nr1" "%a.nr11" 
> "%a.nr" "%a.nr2" "%a.nr3" "%a.nr4
> >>>> rec = write('drylake_80v.txt',val, append)
> >>>> i = i + 1
> >>>> endwhile
> >>>> *******************
> >>>> *******************
> >>>>
> >>>> best regards,
> >>>>
> >>>> Leopoldo ÁLVAREZ
> >>>> Tenerife - Canary Islands
> >>>
> >>
> >>
> >>
> >> _______________________________________________
> >> gradsusr mailing list
> >> gradsusr at gradsusr.org
> >> http://gradsusr.org/mailman/listinfo/gradsusr
> >
> > _______________________________________________
> > gradsusr mailing list
> > gradsusr at gradsusr.org
> > http://gradsusr.org/mailman/listinfo/gradsusr
>
> --
> Jennifer M. Adams
> Center for Ocean-Land-Atmosphere Studies (COLA)
> 111 Research Hall, Mail Stop 2B3
> George Mason University
> 4400 University Drive
> Fairfax, VA 22030
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20130923/eef8cf9f/attachment-0001.html 


More information about the gradsusr mailing list