[gradsusr] division with small numbers in GrADS versions 2 and 1.9b4
Charles Seman (NOAA Federal)
charles.seman at noaa.gov
Wed Nov 28 15:33:39 EST 2012
Dear GrADS Users,
Using GrADS v2.0.a7.oga.3 to do some division with small numbers, it was
found that for var = 1e-7, var/var = 1 as expected, but with var = 1e-8,
var/var = undef ... and maskout() also sets the result to "undef" (when
var is positive)... shown below are some GrADS v2.0.a7.oga.3 calculations...
ga-> var = 1e-6
Define memory allocation size = 8 bytes
Name already DEFINEd: var. Will be deleted and replaced.
ga-> d var/var
Result value = 1
ga-> var = 1e-7
Define memory allocation size = 8 bytes
Name already DEFINEd: var. Will be deleted and replaced.
ga-> d var/var
Result value = 1
ga-> var = 1e-8
Define memory allocation size = 8 bytes
Name already DEFINEd: var. Will be deleted and replaced.
ga-> d var/var
Result value = -9.99e+08
ga-> d maskout(var,var)
Result value = 1e-08
ga-> d maskout(var/var,var)
Result value = -9.99e+08
ga-> d 1/var
Result value = -9.99e+08
ga-> d var
Result value = 1e-08
Using a GrADS 1.9b4 version, the results are as expected all the way
down to somewhere between 1e-37 and 1e-38:
ga-> var = 1e-7
Define memory allocation size = 4 bytes
ga-> d var
Result value = 1e-07
ga-> d var/var
Result value = 1
ga-> var = 1e-8
Define memory allocation size = 4 bytes
Name already DEFINEd: var. Will be deleted and replaced.
ga-> d var
Result value = 1e-08
ga-> d var/var
Result value = 1
ga-> var = 1e-10
Define memory allocation size = 4 bytes
Name already DEFINEd: var. Will be deleted and replaced.
ga-> d var
Result value = 1e-10
ga-> d var/var
Result value = 1
ga-> var = 1e-20
Define memory allocation size = 4 bytes
Name already DEFINEd: var. Will be deleted and replaced.
ga-> d var
Result value = 1e-20
ga-> d var/var
Result value = 1
...
ga-> var = 1e-36
Define memory allocation size = 4 bytes
Name already DEFINEd: var. Will be deleted and replaced.
ga-> d var/var
Result value = 1
ga-> var = 1e-40
Define memory allocation size = 4 bytes
Name already DEFINEd: var. Will be deleted and replaced.
ga-> d var/var
Result value = -9.99e+33
ga-> var = 1e-38
Define memory allocation size = 4 bytes
Name already DEFINEd: var. Will be deleted and replaced.
ga-> d var/var
Result value = -9.99e+33
ga-> var = 1e-37
Define memory allocation size = 4 bytes
Name already DEFINEd: var. Will be deleted and replaced.
ga-> d var/var
Result value = 1
Has anyone else seen this difference in division results in GrADS
versions 2 and 1.9b4?
Thanks,
Chuck
--
Please note that Charles.Seman at noaa.gov should be considered my NOAA
email address, not cjs at gfdl.noaa.gov.
********************************************************************
Charles Seman Charles.Seman at noaa.gov
U.S. Department of Commerce / NOAA / OAR
Geophysical Fluid Dynamics Laboratory voice: (609) 452-6547
201 Forrestal Road fax: (609) 987-5063
Princeton, NJ 08540-6649 http://www.gfdl.noaa.gov/~cjs/
********************************************************************
"The contents of this message are mine personally and do not reflect any
official or unofficial position of the United States Federal Government,
the United States Department of Commerce, or NOAA."
More information about the gradsusr
mailing list