<!doctype html>
<html>
<head>
<meta charset="UTF-8">
</head>
<body>
<div>
<div class="default-style">
Hi Everyone,
<br>On 4/20/2021 with the 12z run of all models, NOAA implemented a brick wall limiter on the OpenDODs server to prevent some users from over-utilizing computer resources and to enable equitable access to everyone.
</div>
<div class="default-style">
</div>
<div class="default-style">
I thought I'd share some info on that process.
</div>
<div class="default-style">
</div>
<div class="default-style">
The limiter is set to prevent more than 120 connections in a 60 second period.
</div>
<div class="default-style">
</div>
<div class="default-style">
A 'connection' in Grads terminology is a request for a single variable. If you access DODs via the sdfopen command, enable fwrite, and then perform a 'display' on a single variable, that constitutes a single 'connection'.
</div>
<div class="default-style">
</div>
<div class="default-style">
While in fwrite (using sdfopen), if you set time = 1 and then display 5 variables, then start a loop and set time = 2 and display the same 5 variables, that is 10 connections from the limiters perspective.
</div>
<div class="default-style">
</div>
<div class="default-style">
Sdfopen moves pretty quick, and if you're downloading data in loops, it's very easy to exceed the 120 connection limit.
</div>
<div class="default-style">
</div>
<div class="default-style">
If you exceed the limit, the limiter will block your access to the 121st variable.
</div>
<div class="default-style">
</div>
<div class="default-style">
Here's the error you will see in Grads (in this case the limiter tripped on the display of variable hgtprs):
</div>
<div class="default-style">
</div>
<div class="default-style">
gancgrid error: nc_get_vara_double failed; NetCDF: Malformed or inaccessible DAP DATADDS
<br>Data Request Error: Error for variable 'hgtprs'
<br>Error ocurred at column 1
<br>DISPLAY error: Invalid expression
<br>Expression = hgtprs
</div>
<div class="default-style">
</div>
<div class="default-style">
<div class="default-style">
The solution is simple.
</div>
<div class="default-style">
</div>
<div class="default-style">
I added a few lines of code to all my sdfopen/fwrite GRads scripts to pause the script after 100 variables have been displayed/downloaded.
</div>
<div class="default-style">
</div>
<div class="default-style">
That grads command (using the Windows sleep executable sleep.exe) would be something like:
</div>
<div class="default-style">
</div>
<div class="default-style">
say 'Sleep for 60 secs to Prevent Hitting 120 connection limit'
<br>'!sleep.exe 60'
<br> say 'Continue Processing'
</div>
<div class="default-style">
</div>
<div class="default-style">
There's probably a much more eloquent way to do this in UNIX and MacOS.
</div>
<div class="default-style">
</div>
<div class="default-style">
Anyway - thought I'd share this tidbit of info.
</div>
<div class="default-style">
</div>
</div>
</div>
<div class="io-ox-signature">
<div>
<span style="font-size: small;">Thanks,<br>Mark</span>
</div>
</div>
</body>
</html>