[gradsusr] Mask/Scripting error
Justin Hicks
jhicks2014 at gmail.com
Mon Jan 9 15:03:54 EST 2017
Jennifer,
Thanks, I never thought about making a list of the mask names and using
'subwrd' to call them. I used the draft you provided and changed a few
lines, and this is what it currently looks like:
'reinit'
outfile='/home/jhicks/Downloads/GLDAS_MASK_DATA.txt'
'!/bin/rm -f 'outfile
masks='00_WLD 01_ART 02_GOM 03_Med 04_blk 05_Cas 06_GLK 07_NPC 08_NAT
09_IND 10_SPC 11_SAT 12_ANT 13_SAm 14_NAm 15_AFR 16_EUA 17_AUS 18_OCN
19_GRN 20_NHL 21_NHO 22_SHL 23_SHO 24_NHm 25_SHm 26_WOL 27_ATL 28_PAC
29_NIO 30_SIO 31_MNP 32_AR2 33_GOB 34_SAH 35_SWM 36_SPD 37_NMB 38_WAS
39_AUD 40_AN2 41_NWP 42_KUR 43_GLF 44_ECS 45_BEN 46_EQP 47_INC 48_GTP
49_WIN 50_AMZ 51_TAL 52_EIO 53_NIG 54_EAF 55_AZ2 56_BOR'
nmasks=57
m=1
while (m<=nmasks)
maskname=subwrd(masks,m)
infile='/data2/control/Region'maskname'_mask_GLDAS.nc'
'open 'infile
if (rc)
say 'open failed for mask file 'm': 'infile', rc='rc
return
endif
'set dfile 1'
'set t 1 756'
'set lon -179.5 179.5'
'set lat -89.5 89.5'
'sdfopen /data2/control/GLDAS_NOAH10_M.020.ctl_1Deg_RainfallRateSfc.nc'
if (rc)
say 'sdfopen failed for GLDAS file with mask number 'm
return
endif
'set dfile 2'
'set lon -179.5 179.5'
'set lat -89.5 89.5'
'set time 00Z01JAN1948'
'set dfile 1'
'q file 1'
region = sublin(result,2)
region = subwrd(region,2)
'q file 2'
dataset = sublin(result,2)
dataset = subwrd(dataset,2)
'q dims'
rectime = sublin(result,5)
recdate = subwrd(rectime,6)
'd
aave(maskout(rainfsfc.2*86400,lterp(mask.1(time=00Z01JAN1948),rainfsfc.2)),lon=-179.5,lon=179.5,lat=-89.5,lat=89.5)'
say result
rainfsfc = sublin(result,1)
rainfsfc = subwrd(rainfsfc,4)
string = dataset' 'region' 'rectime' 'rainfsfc
rc=write(outfile , string ,'append')
'close 2'
'close 1'
m=m+1
endwhile
rc=close(outfile)
*
*
*
When I run this script in GrADS, the result is strange...some results are
printed, but I get this error:
gadsdf: Time increment too large for 'minutes since' time units attribute
SET DFILE error: file 2 not open
sdfopen failed for GLDAS file with mask number 3
*
*
However, if I run this script again, I will get an error pertaining to, for
example mask number 6:
gadsdf: Time increment too large for 'minutes since' time units attribute
SET DFILE error: file 2 not open
sdfopen failed for GLDAS file with mask number 6
I'm starting to think the issue is memory related or possibly with the
masks themselves (which were not produced by me), but I'm not quite sure.
-Justin
On Thu, Jan 5, 2017 at 5:45 PM, Jennifer M Adams <jadams21 at gmu.edu> wrote:
> Hi, Justin —
> I have a couple of suggestions for you.
>
> 0. Please use indents to make your code readable for yourself and others
> helping you.
>
> 1. Rather than duplicate the code block over and over again, I would try
> adding a wrapper around it to loop over the mask names.
>
> 2. You are missing a statement to close your text output file:
> rc=close(“gldas_mask_data.txt”)
> And it looks like your ‘append’ option is never set for your write
> statements. Be careful that you don’t overwrite this output file with every
> write call. I would set the name up front, delete the file with a call to
> the shell, then proceed with all the mask results appending to the file
> every time, then close it when you are done. If you are creating a separate
> output text file for each mask, put this code inside the wrapper loop and
> create a uniqe outfile name that uses the mask name.
>
> 3. You should add a check after every ‘open’ and ‘sdfopen’ call to make
> sure they worked (test if rc!=0), and if not, exit the script with an
> informative message. Then you will be able to see more clearly where things
> are going wrong.
>
> Here’s an untested draft of a re-organization of your script:
>
> ‘reinit’
>
> * set the output file name, delete it if it already exists
> outfile=‘GLDAS_MASK_DATA.txt’
> ‘!/bin/rm -f ‘outfile
>
> * create a list the different mask names for constructing filenames
> masks=’00_WLD 01_ART 02_ABC 03_XYZ’
> nmasks=4
>
> * loop over all the masks
> m=1
> while (m<=nmasks)
> maskname=subwrd(masks,m)
> infile=‘/data2/control/Region’maskname’_mask_GLDAS.nc'
> ‘open ‘infile
> * make sure the file open worked without any errors
> if (rc)
> say ‘open failed for mask file ‘m’: ‘infile’, rc=‘rc
> return
> endif
>
> * add your code here
>
> ‘sdfopen GLDASfile.nc’
> if (rc)
> say ‘sdfopen failed for GLDAS file with mask number ‘m
> return
> endif
>
> * add more of your code here
> string=blah’ ‘blah
> rc=write(outfile,string,append)
>
> * no need to keep calling reinit, just clean up properly, reset if
> necessary
> ‘close 2’
> ‘close 1’
>
> mask=mask+1
> endwhile
>
> * close the text file with all the mask data
> rc=close(“text_output_for_this_mask”)
>
> —Jennifer
>
>
> On Jan 5, 2017, at 12:46 PM, Justin Hicks <jhicks2014 at gmail.com> wrote:
>
> Jeff,
>
> The seventh file exists; this is an issue that pertains to any combination
> of seven masks. Also, I begin each new block of commands with 'reinit',
> which should close any previously open mask files. If I run all 56 masks
> (one at a time, each closing before the next mask is opened), some of the
> masks still output data, and others just output that error statement.
> That's why I don't believe it is a memory issue. Each mask file runs
> perfectly fine when run in its own gradscript; I just get this error when
> grouping them all into one script.
>
> -Justin
>
> On Thu, Jan 5, 2017 at 12:32 PM, Jeff Duda <jeffduda319 at gmail.com> wrote:
>
>> -Does the seventh file exist on your system?
>> -Are you opening too many files? I don't see any 'close' statements in
>> your script, so you may want to consider the ramifications of never closing
>> files if you're opening a large number of them.
>> -Are you running out of memory? Usually Grads just crashes entirely when
>> there's a memory error, but perhaps it's not in this instance.
>>
>> Usually when it comes to debugging, the most comprehensive thing you can
>> do is a line-by-line print statement to see what's going on in the script
>> after each and every line. You can simplify by printing out only specific
>> lines that you think are critical, then subdivide sections of code down to
>> the source of the problem so that you don't have to literally issue a print
>> statement after every line.
>>
>> Jeff Duda
>>
>> On Thu, Jan 5, 2017 at 10:19 AM, Justin Hicks <jhicks2014 at gmail.com>
>> wrote:
>>
>>> GrADS users,
>>>
>>> I have come across a strange problem while running a gradscript. I am
>>> using this script to output average monthly data of different regions to a
>>> text file. Each block of commands contains a different mask, but uses the
>>> same datafile. The first block of commands is shown below:
>>>
>>> 'reinit'
>>> 'open /data2/control/Region00_WLD_mask_GLDAS.nc'
>>> 'set t 1 756'
>>> 'set dfile 1'
>>> 'set lon -179.5 179.5'
>>> 'set lat -89.5 89.5'
>>> 'sdfopen /data2/control/GLDAS_NOAH10_M.020.ctl_1Deg_RainfallRateSfc.nc'
>>> 'set dfile 2'
>>> 'set lon -179.5 179.5'
>>> 'set lat -89.5 89.5'
>>> 'set time 00Z01JAN1948'
>>> 'set dfile 1'
>>> nmonths=756
>>> i=1
>>> while (i<=nmonths)
>>> 'set t 'i
>>> 'q file 1'
>>> region = sublin(result,2)
>>> region = subwrd(region,2)
>>> 'q file 2'
>>> dataset = sublin(result,2)
>>> dataset = subwrd(dataset,2)
>>> 'q dims'
>>> rectime = sublin(result,5)
>>> recdate = subwrd(rectime,6)
>>> 'd aave(maskout(rainfsfc.2*86400,lterp(mask.1(time=00Z01JAN1948
>>> ),rainfsfc.2)),lon=-179.5,lon=179.5,lat=-89.5,lat=89.5)'
>>> say 't='i
>>> say result
>>> rainfsfc = sublin(result,1)
>>> rainfsfc = subwrd(rainfsfc,4)
>>> string = dataset' 'region' 'rectime' 'rainfsfc
>>> rc=write('GLDAS_MASK_DATA.txt' , string ,'append')
>>> i = i + 1
>>> endwhile
>>>
>>> I repeat these commands in the same script over and over, only changing
>>> the second line (ex. the next one would read 'open
>>> /data2/control/Region01_ART_mask_GLDAS.nc'). The data is output
>>> perfectly when I have six masks in the script, but when I include the
>>> seventh, the data output from the third mask / block of commands is
>>> altered, and I get the following error:
>>>
>>> QUERY FILE Error: file 2 not open
>>> Data Request Error: File number out of range
>>> Variable = rainfsfc.2
>>> Operation Error: Error from maskout function
>>> Operation Error: Error from aave function
>>> Error ocurred at column 1
>>> DISPLAY error: Invalid expression
>>> Expression = aave(maskout(rainfsfc.2*86400,
>>> lterp(mask.1(time=00z01jan1948),rainfsfc.2)),lon=-179.5,lon=
>>> 179.5,lat=-89.5,lat=89.5)
>>> t=703
>>> Data Request Error: File number out of range
>>> Variable = rainfsfc.2
>>> Operation Error: Error from maskout function
>>> Operation Error: Error from aave function
>>> Error ocurred at column 1
>>> DISPLAY error: Invalid expression
>>> Expression = aave(maskout(rainfsfc.2*86400,
>>> lterp(mask.1(time=00z01jan1948),rainfsfc.2)),lon=-179.5,lon=
>>> 179.5,lat=-89.5,lat=89.5)
>>>
>>>
>>> I don't believe the error lies within the third mask file itself because
>>> it plots fine when it is run independently from the other masks. I also get
>>> the same output issue when I run, for example, masks 50-56 (one or two of
>>> these will come up with the error, but running masks 50-55 will yield no
>>> problems).
>>>
>>> Any help would be appreciated to me solve this issue, as I'm baffled.
>>>
>>> -Justin
>>>
>>> --
>>> *Justin Hicks*
>>>
>>> *Undergraduate Atmospheric Sciences Major*
>>>
>>> *GIS Minor University of Maryland-College Park*
>>>
>>> _______________________________________________
>>> gradsusr mailing list
>>> gradsusr at gradsusr.org
>>> http://gradsusr.org/mailman/listinfo/gradsusr
>>>
>>>
>>
>>
>> --
>> Jeff Duda
>> Post-doctoral research fellow
>> University of Oklahoma School of Meteorology
>>
>> _______________________________________________
>> gradsusr mailing list
>> gradsusr at gradsusr.org
>> http://gradsusr.org/mailman/listinfo/gradsusr
>>
>>
>
>
> --
> *Justin Hicks*
>
> *Undergraduate Atmospheric Sciences Major*
>
> *GIS Minor University of Maryland-College Park*
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
>
>
> --
> Jennifer Miletta Adams
> Center for Ocean-Land-Atmosphere Studies (COLA)
> George Mason University
>
>
>
>
> _______________________________________________
> gradsusr mailing list
> gradsusr at gradsusr.org
> http://gradsusr.org/mailman/listinfo/gradsusr
>
>
--
*Justin Hicks*
*UndergraduateAtmospheric Sciences Major*
*GIS MinorUniversity of Maryland-College Park*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://gradsusr.org/pipermail/gradsusr/attachments/20170109/5d0a21bf/attachment-0001.html
More information about the gradsusr
mailing list