[gradsusr] Low level I/O error: read error on data file

[Mark Sponsler]

Hi Jennifer,

Good info..

I don't want to hijack Annas thread but.....

In my instance the error is usually due to the data being corrupted on the Dods server in later time steps in the file. I can download and process the file fine for weeks, then for whatevet reason it will get corrupted and syat that way for most every model run until NOAA realizes the error and fixes it. I suspect it's due to some batch process that actually puts the files on the dods server. 

The file is typically 228 meg total, so relatively small in size. 3 hr NAM data/4 variables.

Was wondering if there is a way to insert a test of a single variable for each timestep that I execute before I try to execute the grads display function.  It seems some of the time step data is fine, but then gets corrupt in later time steps.  But I'm guessing just the test would cause grads to crash. 

 The data I've downloaded this morning looks good. But I'll monitor to see if there is a significant difference in file size between corrupt and 'good' files. Maybe that's another 'safety check' option just checking bulk file size). 

Any other ideas are welcome. 

On February 4, 2016 5:44:43 AM PST, Jennifer M Adams <jadams21 at gmu.edu> wrote:
>The “Low level I/O error” message usually means your flat binary file
>is the wrong size. GrADS makes a calculation of where to offset the
>pointer — i.e. how many bytes deep into the file it needs to go to read
>the desired grid. If the file read command returns an error, you get
>the message that tells you how many bytes it was trying to read from
>the file (in this case 1144) and the offset it was using (in this case
>-2129859208, which isn’t right and looks like some kind of precision
>error).
>
>The first thing you should do is make a calculation of what the file
>size should be:
>xsize * ysize * zsize * tsize * esize * number-of-variables * 4 (bytes)
>If your file has some z-varying variables and some non-z-varying
>variables of if your data is sequential this calculation is a little
>trickier, but you get the idea. If your file size doesn’t match, then
>figure out where you went wrong in describing the grid or in creating
>your file. If it does match, then the file may be corrputed somewhere.
>Try displaying one time step at a time and figure out which T index
>causes it to fail — if that T value is where the file size crosses the
>2GB mark, then you probably have a precision issue. GrADS 2.0.2 should
>easily be able to read a big binary file, but it may depend on the
>system you’re using to read it.
>—Jennifer
>
>
>On Feb 1, 2016, at 7:50 AM, Anna Lukianova
><romashe4ka04 at rambler.ru<mailto:romashe4ka04 at rambler.ru>> wrote:
>
>
>Hi, everyone!
>
>I have 2-D data file with a lot of records in time.
>
>When I want to plot variables for the last records of time, I obtain
>such an error  :
>
>
>Low level I/O error: read error on data file. Error reading 1144 bytes
>at location -2129859208/
>
>
>Is this error linked to some file limits in GrADS (I have 2.0.2
>version) ?
>
>
>Thanks in advance, Lukianova Ann.
>
>_______________________________________________
>gradsusr mailing list
>gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
>http://gradsusr.org/mailman/listinfo/gradsusr
>
>--
>Jennifer Miletta Adams
>Center for Ocean-Land-Atmosphere Studies (COLA)
>George Mason University
>
>
>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>gradsusr mailing list
>gradsusr at gradsusr.org
>http://gradsusr.org/mailman/listinfo/gradsusr

Thanks,
Mark Sponsler
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20160204/9b451d6a/attachment.html