[gradsusr] Low level I/O error: read error on data file

Jennifer M Adams jadams21 at gmu.edu
Thu Feb 4 13:41:43 EST 2016


On Feb 4, 2016, at 11:02 AM, Mark Sponsler <msponsler at COMCAST.NET<mailto:msponsler at COMCAST.NET>> wrote:

Hi Jennifer,

Good info..

I don't want to hijack Annas thread but.....

In my instance the error is usually due to the data being corrupted on the Dods server in later time steps in the file. I can download and process the file fine for weeks, then for whatevet reason it will get corrupted and syat that way for most every model run until NOAA realizes the error and fixes it. I suspect it's due to some batch process that actually puts the files on the dods server.
That sounds like something you should discuss with the administrator of the NOAA GDS.


The file is typically 228 meg total, so relatively small in size. 3 hr NAM data/4 variables.

Was wondering if there is a way to insert a test of a single variable for each timestep that I execute before I try to execute the grads display function. It seems some of the time step data is fine, but then gets corrupt in later time steps. But I'm guessing just the test would cause grads to crash.
Well, the file I/O error shouldn’t crash GrADS. You can try a display with ’set gxout stat’ and check if you get a non-zero return code.


The data I've downloaded this morning looks good. But I'll monitor to see if there is a significant difference in file size between corrupt and 'good' files. Maybe that's another 'safety check' option just checking bulk file size).

Any other ideas are welcome.
It really depends a lot on how you are downloading the data. Your bug report to NOAA should have the exact time when the corrupted file was downloaded, they could (in theory) check the server logs to see if there’s an explanation.
—Jennifer

On February 4, 2016 5:44:43 AM PST, Jennifer M Adams <jadams21 at gmu.edu<mailto:jadams21 at gmu.edu>> wrote:
The “Low level I/O error” message usually means your flat binary file is the wrong size. GrADS makes a calculation of where to offset the pointer — i.e. how many bytes deep into the file it needs to go to read the desired grid. If the file read command returns an error, you get the message that tells you how many bytes it was trying to read from the file (in this case 1144) and the offset it was using (in this case -2129859208, which isn’t right and looks like some kind of precision error).

The first thing you should do is make a calculation of what the file size should be:
xsize * ysize * zsize * tsize * esize * number-of-variables * 4 (bytes)
If your file has some z-varying variables and some non-z-varying variables of if your data is sequential this calculation is a little trickier, but you get the idea. If your file size doesn’t match, then figure out where you went wrong in describing the grid or in creating your file. If it does match, then the file may be corrputed somewhere. Try displaying one time step at a time and figure out which T index causes it to fail — if that T value is where the file size crosses the 2GB mark, then you probably have a precision issue. GrADS 2.0.2 should easily be able to read a big binary file, but it may depend on the system you’re using to read it.
—Jennifer


On Feb 1, 2016, at 7:50 AM, Anna Lukianova <romashe4ka04 at rambler.ru<mailto:romashe4ka04 at rambler.ru>> wrote:


Hi, everyone!

I have 2-D data file with a lot of records in time.

When I want to plot variables for the last records of time, I obtain such an error  :


Low level I/O error: read error on data file. Error reading 1144 bytes at location -2129859208/


Is this error linked to some file limits in GrADS (I have 2.0.2 version) ?


Thanks in advance, Lukianova Ann.

_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr

--
Jennifer Miletta Adams
Center for Ocean-Land-Atmosphere Studies (COLA)
George Mason University





________________________________

gradsusr mailing list
gradsusr at gradsusr.org<mailto:gradsusr at gradsusr.org>
http://gradsusr.org/mailman/listinfo/gradsusr

Thanks,
Mark Sponsler

--
Jennifer Miletta Adams
Center for Ocean-Land-Atmosphere Studies (COLA)
George Mason University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20160204/b6b5b01c/attachment.html 


More information about the gradsusr mailing list