[gradsusr] fwrite gridding issue

Thu May 10 16:56:35 EDT 2012

Tom,

There are three things I'll point out here:

1. You seem to be trying to write out all the data at once, in a huge array dimensioned (144,73,48212). This array may require more memory than your computer has. Instead, try declaring your array as (144,73) and set up a DO loop in your fortran program to write out each time step individually. You'll need to substantially modify how your program reads the input data, too. If you send your complete fortran code, I might be able to modify it to implement my suggested method (no guarantees, though). This will eliminate the need to break up the 48212 time steps into 68*709. 

2. What is the size of the input files to your fortran program (i.e. the files written by fwrite)? This information will indicate if you have a wraparound problem.

3. Your ctl file has the line:

chi 0 x,y,t 1 ** velocity potential 

Since your data is binary, I think this line should be:

chi 0 99 ** velocity potential

Eric

----- Original Message -----
From: "Thomas Robinson" <ter at hawaii.edu>
To: "GrADS Users Forum" <gradsusr at gradsusr.org>
Sent: Thursday, May 10, 2012 2:03:09 PM
Subject: Re: [gradsusr] fwrite gridding issue

The file is the correct size. 144*73*48212*4=2027218176 

-rw-r--r-- 1 ter businger 2027218176 May 9 15:57 chi_33yr.dat 

My computer crashes if I try to do all 48212, so I had to break it up in 68 separate files each with 709 time steps. So: 
PARAMETER (nt=709) !Number of time steps in each file 
PARAMETER (nrec=709) !Number of records in each file 
PARAMETER (files=68) !Number of input files 
The FORTRAN program reads in all of the files and stores the data in one variable chi, then it goes written out to the file chi_33yr.dat. 

My FORTRAN program also calculates a 33 year 6-hourly mean, but the variable chi isn't affected by that and that is written to a separate file which is also having the same problem. 

-Tom 

On Thu, May 10, 2012 at 7:51 AM, Jennifer Adams < jma at cola.iges.org > wrote: 

I'm no fortran expert, but I think "direct" means the 4-byte headers and footers for each record are not written, so you should remove 'options sequential'. According to your descriptor, your data file (chi_33yr.dat) should be 144*73*48212*4 bytes large. Is it? What is 'files'? 
--Jennifer 

On May 10, 2012, at 1:44 PM, Thomas Robinson wrote: 

Here is my code that opens and writes out the data: 

open (unit=291,file="chi_33yr.dat", access="direct", 
& status="unknown",recl=4*nx*ny*nt*files) 
write (291, rec=1) chi 

The variable chi has dimensions nx,ny,nt*files . 

I added options sequential to the descriptor file and the same thing is happening. Here is my descriptor file: 

dset ^chi_33yr.dat 
title Velocity Potential 
options sequential 
undef 9.999e+20 
xdef 144 linear 0 2.5 
ydef 73 linear -90 2.5 
zdef 1 levels 200 
tdef 48212 linear 00Z01JAN1979 360mn 
vars 1 
chi 0 x,y,t 1 ** velocity potential 
endvars 

All of your input is very much appreciated. 

-Tom 

On Fri, May 11, 2012 at 3:16 AM, Jennifer Adams < jma at cola.iges.org > wrote: 

Tom, Did you include 'options sequential' in your descriptor file for your new binary file? --Jennifer 

On May 10, 2012, at 8:49 AM, Raghu Reddy wrote: 

I am not a grads user, just responding to the FORTRAN/C issue. 

If you do a normal sequential FORTRAN write, it writes an end-of-record marker at the beginning * and * at the end of * every * write. Since you mentioned that the data is skewed, I assume you wrote all the 48,000 values with one write statement, and that would result in one EOR marker at the beginning and one EOR marker at the end. 

If you don’t want these EOR markers you could write the file out as direct access files in Fortran. 

Hope this helps. 

Thanks, 

--Raghu 

From: gradsusr-bounces at gradsusr.org [mailto: gradsusr-bounces at gradsusr.org ] On Behalf Of Thomas Robinson 
Sent: Wednesday, May 09, 2012 11:36 PM 
To: GrADS Users Forum 
Subject: [gradsusr] fwrite gridding issue 

Aloha, 

I used fwrite to write out to a binary file in order to create a large file with about 48000 time steps for global data (because I am crazy). I noticed that when I use fortran to create a new data file that has all 48000, it doesn't match with what the original data plotted, instead it seems skewed. No manipulations were done to the data, it was just read it all in and write it out. When I tried to loop the data, I further noticed that the northern border looped around the south and then moved upwards back to the north. It just keeps looping around and it's odd because the top boundary of the map isn't continuous with the bottom boundary (unlike the side boundaries which are continuous). Anyways, I am wondering why this happened and what I can do to make it work out correctly. 

Here is my fwrite code. it is the first time step, nt is the last time step, chi is the velocity potential that was calculated using 'define chi = fish_chi(UGRDprs,VGRDprs)' : 
*# set up global domain 
'set lev 200 ' 
'set lat -90 90' 
'set lon 0 360' 
*#************************************************************** 
say 'write data out to binary file called CHI.200hPa'filenum'.dat' 
'set fwrite -le -sq -cl CHI_200hPa_'filenum'.dat' 
'set gxout fwrite' 
while (it <= nt) 
'set t 'it 
'display chi' 
it=it+1 
endwhile 
'disable fwrite' 

Mahalo for your help! 
-Tom 

-- 
Tom Robinson 
President Graduate Student Organization 
Student Caucus Representative for the Graduate Student Organization 
Graduate Student - Department of Meteorology 
732-718-2323 _______________________________________________ 

gradsusr mailing list 
gradsusr at gradsusr.org 
http://gradsusr.org/mailman/listinfo/gradsusr 

-- 
Jennifer M. Adams 
IGES/COLA 

4041 Powder Mill Road, Suite 302 
Calverton, MD 20705 
jma at cola.iges.org 

_______________________________________________ 
gradsusr mailing list 
gradsusr at gradsusr.org 
http://gradsusr.org/mailman/listinfo/gradsusr 

-- 
Tom Robinson 
President Graduate Student Organization 
Student Caucus Representative for the Graduate Student Organization 
Graduate Student - Department of Meteorology 
732-718-2323 

_______________________________________________ 
gradsusr mailing list 
gradsusr at gradsusr.org 
http://gradsusr.org/mailman/listinfo/gradsusr 

-- 
Jennifer M. Adams 
IGES/COLA 
4041 Powder Mill Road, Suite 302 
Calverton, MD 20705 
jma at cola.iges.org 

_______________________________________________ 
gradsusr mailing list 
gradsusr at gradsusr.org 
http://gradsusr.org/mailman/listinfo/gradsusr 

-- 
Tom Robinson 
President Graduate Student Organization 
Student Caucus Representative for the Graduate Student Organization 
Graduate Student - Department of Meteorology 
732-718-2323 

_______________________________________________
gradsusr mailing list
gradsusr at gradsusr.org
http://gradsusr.org/mailman/listinfo/gradsusr