Hi All,
I'm curious as to what folks think about some choices in the compution of
the remainder function. As an example where different choices can be made
In [2]: -1e-64 % 1.
Out[2]: 1.0
In [3]: float64(-1e-64) % 1.
Out[3]: 0.99999999999999989
The first is Python, the second is in my branch. The first is more accurate
numerically, but the modulus is of the same magnitude as the divisor. The
second maintains the convention that the result must have smaller magnitude
than the divisor. There are other corner cases along the same lines. So the
question is, which is more desirable: maintaining numerical accuracy
or enforcing
mathematical convention? The differences are on the order of an ulp, but
there will be a skew in the distribution of the errors if convention is
maintained.
The Fortran modulo function, which is the same basic function as in my
branch, does not specify any bounds on the result for floating numbers, but
gives only the formula, modulus(a, b) = a - b*floor(a/b), which has the
advantage of being simple and well defined ;)
Chuck