Floating point "close" function?
Is there a numpy function for testing floating point equality that returns a boolean array? I'm aware of np.allclose, but I need a boolean array. Properly handling NaN's and Inf's (as allclose does) would be a nice bonus. I wrote the function below to do this, but I suspect there's a method in numpy that I missed. import numpy as np def close(a, b, rtol=1.e-5, atol=1.e-8, check_invalid=True): """Similar to numpy.allclose, but returns a boolean array. See numpy.allclose for an explanation of *rtol* and *atol*.""" def within_tol(x, y, atol, rtol): return np.less_equal(np.abs(x-y), atol + rtol * np.abs(y)) x = np.array(a, copy=False) y = np.array(b, copy=False) if not check_invalid: return within_tol(x, y, atol, rtol) xfin = np.isfinite(x) yfin = np.isfinite(y) if np.all(xfin) and np.all(yfin): return within_tol(x, y, atol, rtol) else: # Avoid subtraction with infinite/nan values... cond = np.zeros(np.broadcast(x, y).shape, dtype=np.bool) mask = xfin & yfin cond[mask] = within_tol(x[mask], y[mask], atol, rtol) # Inf and -Inf equality... cond[~mask] = (x[~mask] == y[~mask]) # NaN equality... cond[np.isnan(x) & np.isnan(y)] = True return cond # A few quick tests... assert np.any(close(0.300001, np.array([0.1, 0.2, 0.3, 0.4]))) x = np.array([0.1, np.nan, np.inf, -np.inf]) y = np.array([0.1000001, np.nan, np.inf, -np.inf]) assert np.all(close(x, y)) x = np.array([0.1, 0.2, np.inf]) y = np.array([0.101, np.nan, 0.2]) assert not np.all(close(x, y)) Thanks, -Joe
On Thu, Mar 1, 2012 at 11:44 PM, Joe Kington <jkington@wisc.edu> wrote:
Is there a numpy function for testing floating point equality that returns a boolean array?
I'm aware of np.allclose, but I need a boolean array. Properly handling NaN's and Inf's (as allclose does) would be a nice bonus.
I wrote the function below to do this, but I suspect there's a method in numpy that I missed.
I don't think such a function exists, would be nice to have. How about just adding a keyword "return_array" to allclose to do so? Ralf
import numpy as np
def close(a, b, rtol=1.e-5, atol=1.e-8, check_invalid=True): """Similar to numpy.allclose, but returns a boolean array. See numpy.allclose for an explanation of *rtol* and *atol*.""" def within_tol(x, y, atol, rtol): return np.less_equal(np.abs(x-y), atol + rtol * np.abs(y)) x = np.array(a, copy=False) y = np.array(b, copy=False) if not check_invalid: return within_tol(x, y, atol, rtol) xfin = np.isfinite(x) yfin = np.isfinite(y) if np.all(xfin) and np.all(yfin): return within_tol(x, y, atol, rtol) else: # Avoid subtraction with infinite/nan values... cond = np.zeros(np.broadcast(x, y).shape, dtype=np.bool) mask = xfin & yfin cond[mask] = within_tol(x[mask], y[mask], atol, rtol) # Inf and -Inf equality... cond[~mask] = (x[~mask] == y[~mask]) # NaN equality... cond[np.isnan(x) & np.isnan(y)] = True return cond
# A few quick tests... assert np.any(close(0.300001, np.array([0.1, 0.2, 0.3, 0.4])))
x = np.array([0.1, np.nan, np.inf, -np.inf]) y = np.array([0.1000001, np.nan, np.inf, -np.inf]) assert np.all(close(x, y))
x = np.array([0.1, 0.2, np.inf]) y = np.array([0.101, np.nan, 0.2]) assert not np.all(close(x, y))
Thanks, -Joe
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Sat, Mar 3, 2012 at 13:59, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
On Thu, Mar 1, 2012 at 11:44 PM, Joe Kington <jkington@wisc.edu> wrote:
Is there a numpy function for testing floating point equality that returns a boolean array?
I'm aware of np.allclose, but I need a boolean array. Properly handling NaN's and Inf's (as allclose does) would be a nice bonus.
I wrote the function below to do this, but I suspect there's a method in numpy that I missed.
I don't think such a function exists, would be nice to have. How about just adding a keyword "return_array" to allclose to do so?
As a general design principle, adding a boolean flag that changes the return type is worse than making a new function. -- Robert Kern
On Sat, Mar 3, 2012 at 3:05 PM, Robert Kern <robert.kern@gmail.com> wrote:
On Sat, Mar 3, 2012 at 13:59, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
On Thu, Mar 1, 2012 at 11:44 PM, Joe Kington <jkington@wisc.edu> wrote:
Is there a numpy function for testing floating point equality that
returns
a boolean array?
I'm aware of np.allclose, but I need a boolean array. Properly handling NaN's and Inf's (as allclose does) would be a nice bonus.
I wrote the function below to do this, but I suspect there's a method in numpy that I missed.
I don't think such a function exists, would be nice to have. How about just adding a keyword "return_array" to allclose to do so?
As a general design principle, adding a boolean flag that changes the return type is worse than making a new function.
That's certainly true as a general principle. Do you have a concrete suggestion in this case though? Because this is also bad:
np.<TAB> Display all 561 possibilities? (y or n)
Ralf
On Sat, Mar 3, 2012 at 14:31, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
On Sat, Mar 3, 2012 at 3:05 PM, Robert Kern <robert.kern@gmail.com> wrote:
On Sat, Mar 3, 2012 at 13:59, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
On Thu, Mar 1, 2012 at 11:44 PM, Joe Kington <jkington@wisc.edu> wrote:
Is there a numpy function for testing floating point equality that returns a boolean array?
I'm aware of np.allclose, but I need a boolean array. Properly handling NaN's and Inf's (as allclose does) would be a nice bonus.
I wrote the function below to do this, but I suspect there's a method in numpy that I missed.
I don't think such a function exists, would be nice to have. How about just adding a keyword "return_array" to allclose to do so?
As a general design principle, adding a boolean flag that changes the return type is worse than making a new function.
That's certainly true as a general principle. Do you have a concrete suggestion in this case though?
np.close()
Because this is also bad:
np.<TAB> Display all 561 possibilities? (y or n)
Not as bad as overloading np.allclose(x,y,return_array=True). Or deprecating np.allclose() in favor of np.close().all(). -- Robert Kern
On Saturday, March 3, 2012, Robert Kern <robert.kern@gmail.com> wrote:
On Sat, Mar 3, 2012 at 14:31, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
On Sat, Mar 3, 2012 at 3:05 PM, Robert Kern <robert.kern@gmail.com>
wrote:
On Sat, Mar 3, 2012 at 13:59, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
On Thu, Mar 1, 2012 at 11:44 PM, Joe Kington <jkington@wisc.edu>
wrote:
Is there a numpy function for testing floating point equality that returns a boolean array?
I'm aware of np.allclose, but I need a boolean array. Properly handling NaN's and Inf's (as allclose does) would be a nice bonus.
I wrote the function below to do this, but I suspect there's a method in numpy that I missed.
I don't think such a function exists, would be nice to have. How about just adding a keyword "return_array" to allclose to do so?
As a general design principle, adding a boolean flag that changes the return type is worse than making a new function.
That's certainly true as a general principle. Do you have a concrete suggestion in this case though?
np.close()
When I read that, I mentally think of "close" as in closing a file. I think we need a synonym. Ben Root
On Sat, Mar 3, 2012 at 15:22, Benjamin Root <ben.root@ou.edu> wrote:
On Saturday, March 3, 2012, Robert Kern <robert.kern@gmail.com> wrote:
On Sat, Mar 3, 2012 at 14:31, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
On Sat, Mar 3, 2012 at 3:05 PM, Robert Kern <robert.kern@gmail.com> wrote:
On Sat, Mar 3, 2012 at 13:59, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
On Thu, Mar 1, 2012 at 11:44 PM, Joe Kington <jkington@wisc.edu> wrote:
Is there a numpy function for testing floating point equality that returns a boolean array?
I'm aware of np.allclose, but I need a boolean array. Properly handling NaN's and Inf's (as allclose does) would be a nice bonus.
I wrote the function below to do this, but I suspect there's a method in numpy that I missed.
I don't think such a function exists, would be nice to have. How about just adding a keyword "return_array" to allclose to do so?
As a general design principle, adding a boolean flag that changes the return type is worse than making a new function.
That's certainly true as a general principle. Do you have a concrete suggestion in this case though?
np.close()
When I read that, I mentally think of "close" as in closing a file. I think we need a synonym.
np.isclose() -- Robert Kern
On Sat, Mar 3, 2012 at 9:26 AM, Robert Kern <robert.kern@gmail.com> wrote:
On Sat, Mar 3, 2012 at 15:22, Benjamin Root <ben.root@ou.edu> wrote:
On Saturday, March 3, 2012, Robert Kern <robert.kern@gmail.com> wrote:
On Sat, Mar 3, 2012 at 14:31, Ralf Gommers <ralf.gommers@googlemail.com
wrote:
On Sat, Mar 3, 2012 at 3:05 PM, Robert Kern <robert.kern@gmail.com> wrote:
On Sat, Mar 3, 2012 at 13:59, Ralf Gommers <
wrote:
On Thu, Mar 1, 2012 at 11:44 PM, Joe Kington <jkington@wisc.edu> wrote: > > Is there a numpy function for testing floating point equality that > returns > a boolean array? > > I'm aware of np.allclose, but I need a boolean array. Properly > handling > NaN's and Inf's (as allclose does) would be a nice bonus. > > I wrote the function below to do this, but I suspect there's a
method
> in > numpy that I missed.
I don't think such a function exists, would be nice to have. How about just adding a keyword "return_array" to allclose to do so?
As a general design principle, adding a boolean flag that changes the return type is worse than making a new function.
That's certainly true as a general principle. Do you have a concrete suggestion in this case though?
np.close()
When I read that, I mentally think of "close" as in closing a file. I
ralf.gommers@googlemail.com> think
we need a synonym.
np.isclose()
Would it be helpful if I went ahead and submitted a pull request with the function in my original question called "isclose" (along with a complete docstring and a few tests)? One note: At the moment, it deliberately compares NaN's as equal. E.g. isclose([np.nan, np.nan], [np.nan, np.nan]) will return: [True, True] This obviously runs counter to the standard way NaN's are handled (and indeed the definition of NaN). However, in the context of a floating point "close to" function, I think it makes the most sense. I've had this sitting around in a small project for awhile now, and it's been more useful to have it compare NaN's as "approximately equal" than not for my purposes at least. Nonetheless, it's something that needs additional consideration. Thanks, -Joe
-- Robert Kern _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Le 3 mars 2012 13:07, Joe Kington <jkington@wisc.edu> a écrit :
On Sat, Mar 3, 2012 at 9:26 AM, Robert Kern <robert.kern@gmail.com> wrote:
On Sat, Mar 3, 2012 at 15:22, Benjamin Root <ben.root@ou.edu> wrote:
On Saturday, March 3, 2012, Robert Kern <robert.kern@gmail.com> wrote:
On Sat, Mar 3, 2012 at 14:31, Ralf Gommers <
wrote:
On Sat, Mar 3, 2012 at 3:05 PM, Robert Kern <robert.kern@gmail.com> wrote:
On Sat, Mar 3, 2012 at 13:59, Ralf Gommers <
ralf.gommers@googlemail.com>
wrote: > > > On Thu, Mar 1, 2012 at 11:44 PM, Joe Kington <jkington@wisc.edu> > wrote: >> >> Is there a numpy function for testing floating point equality that >> returns >> a boolean array? >> >> I'm aware of np.allclose, but I need a boolean array. Properly >> handling >> NaN's and Inf's (as allclose does) would be a nice bonus. >> >> I wrote the function below to do this, but I suspect there's a method >> in >> numpy that I missed. > > > I don't think such a function exists, would be nice to have. How about > just > adding a keyword "return_array" to allclose to do so?
As a general design principle, adding a boolean flag that changes the return type is worse than making a new function.
That's certainly true as a general principle. Do you have a concrete suggestion in this case though?
np.close()
When I read that, I mentally think of "close" as in closing a file. I
ralf.gommers@googlemail.com> think
we need a synonym.
np.isclose()
Would it be helpful if I went ahead and submitted a pull request with the function in my original question called "isclose" (along with a complete docstring and a few tests)?
One note: At the moment, it deliberately compares NaN's as equal. E.g.
isclose([np.nan, np.nan], [np.nan, np.nan])
will return:
[True, True]
This obviously runs counter to the standard way NaN's are handled (and indeed the definition of NaN).
However, in the context of a floating point "close to" function, I think it makes the most sense.
I've had this sitting around in a small project for awhile now, and it's been more useful to have it compare NaN's as "approximately equal" than not for my purposes at least.
Nonetheless, it's something that needs additional consideration.
Thanks, -Joe
It would be confusing if numpy.isclose().all() was different from numpy.allclose(). That being said, I agree it's useful to have NaNs compare equal in some cases, maybe it could be a new argument to the function? -=- Olivier
On Sat, Mar 3, 2012 at 12:50 PM, Olivier Delalleau <shish@keba.be> wrote:
Would it be helpful if I went ahead and submitted a pull request with the function in my original question called "isclose" (along with a complete docstring and a few tests)?
One note: At the moment, it deliberately compares NaN's as equal. E.g.
isclose([np.nan, np.nan], [np.nan, np.nan])
will return:
[True, True]
This obviously runs counter to the standard way NaN's are handled (and indeed the definition of NaN).
However, in the context of a floating point "close to" function, I think it makes the most sense.
I've had this sitting around in a small project for awhile now, and it's been more useful to have it compare NaN's as "approximately equal" than not for my purposes at least.
Nonetheless, it's something that needs additional consideration.
Thanks, -Joe
It would be confusing if numpy.isclose().all() was different from numpy.allclose(). That being said, I agree it's useful to have NaNs compare equal in some cases, maybe it could be a new argument to the function?
Good point. I went ahead and added an "equal_nan" kwarg and removed the "check_invalid" kwarg I had in before. I also made it mimic what np.equal() does in the case of two scalars (return a scalar instead of an array). I went ahead an make a pull request: https://github.com/numpy/numpy/pull/224 Hope that's alright. Cheers, -Joe
-=- Olivier
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Sat, Mar 3, 2012 at 14:34, Robert Kern <robert.kern@gmail.com> wrote:
On Sat, Mar 3, 2012 at 14:31, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
Because this is also bad:
np.<TAB> Display all 561 possibilities? (y or n)
Not as bad as overloading np.allclose(x,y,return_array=True). Or deprecating np.allclose() in favor of np.close().all().
I screwed up this paragraph. I meant that as "Another alternative would be to deprecate ...". -- Robert Kern
Le 3 mars 2012 10:27, Robert Kern <robert.kern@gmail.com> a écrit :
On Sat, Mar 3, 2012 at 14:34, Robert Kern <robert.kern@gmail.com> wrote:
On Sat, Mar 3, 2012 at 14:31, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
Because this is also bad:
np.<TAB> Display all 561 possibilities? (y or n)
Not as bad as overloading np.allclose(x,y,return_array=True). Or deprecating np.allclose() in favor of np.close().all().
I screwed up this paragraph. I meant that as "Another alternative would be to deprecate ...".
np.close().all() would probably be a lot less efficient in terms of CPU / memory though, wouldn't it? -=- Olivier
On Sat, Mar 3, 2012 at 15:51, Olivier Delalleau <shish@keba.be> wrote:
Le 3 mars 2012 10:27, Robert Kern <robert.kern@gmail.com> a écrit :
On Sat, Mar 3, 2012 at 14:34, Robert Kern <robert.kern@gmail.com> wrote:
On Sat, Mar 3, 2012 at 14:31, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
Because this is also bad:
> np.<TAB> Display all 561 possibilities? (y or n)
Not as bad as overloading np.allclose(x,y,return_array=True). Or deprecating np.allclose() in favor of np.close().all().
I screwed up this paragraph. I meant that as "Another alternative would be to deprecate ...".
np.close().all() would probably be a lot less efficient in terms of CPU / memory though, wouldn't it?
No. np.allclose() is essentially doing exactly this already. -- Robert Kern
Le 3 mars 2012 11:03, Robert Kern <robert.kern@gmail.com> a écrit :
On Sat, Mar 3, 2012 at 15:51, Olivier Delalleau <shish@keba.be> wrote:
Le 3 mars 2012 10:27, Robert Kern <robert.kern@gmail.com> a écrit :
On Sat, Mar 3, 2012 at 14:34, Robert Kern <robert.kern@gmail.com>
wrote:
On Sat, Mar 3, 2012 at 14:31, Ralf Gommers < ralf.gommers@googlemail.com> wrote:
Because this is also bad:
>> np.<TAB> Display all 561 possibilities? (y or n)
Not as bad as overloading np.allclose(x,y,return_array=True). Or deprecating np.allclose() in favor of np.close().all().
I screwed up this paragraph. I meant that as "Another alternative would be to deprecate ...".
np.close().all() would probably be a lot less efficient in terms of CPU / memory though, wouldn't it?
No. np.allclose() is essentially doing exactly this already.
Ok. What about then, np.allclose() could theoretically be a lot more efficient in terms of CPU / memory? ;) -=- Olivier
On Sat, Mar 3, 2012 at 16:06, Olivier Delalleau <shish@keba.be> wrote:
Le 3 mars 2012 11:03, Robert Kern <robert.kern@gmail.com> a écrit :
On Sat, Mar 3, 2012 at 15:51, Olivier Delalleau <shish@keba.be> wrote:
Le 3 mars 2012 10:27, Robert Kern <robert.kern@gmail.com> a écrit :
On Sat, Mar 3, 2012 at 14:34, Robert Kern <robert.kern@gmail.com> wrote:
On Sat, Mar 3, 2012 at 14:31, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
Because this is also bad: >>> np.<TAB> Display all 561 possibilities? (y or n)
Not as bad as overloading np.allclose(x,y,return_array=True). Or deprecating np.allclose() in favor of np.close().all().
I screwed up this paragraph. I meant that as "Another alternative would be to deprecate ...".
np.close().all() would probably be a lot less efficient in terms of CPU / memory though, wouldn't it?
No. np.allclose() is essentially doing exactly this already.
Ok. What about then, np.allclose() could theoretically be a lot more efficient in terms of CPU / memory? ;)
True, but so could a hypothetical np.allequal(), np.allless_equal(), etc. -- Robert Kern
On Sat, Mar 3, 2012 at 10:06 AM, Olivier Delalleau <shish@keba.be> wrote:
Le 3 mars 2012 11:03, Robert Kern <robert.kern@gmail.com> a écrit :
On Sat, Mar 3, 2012 at 15:51, Olivier Delalleau <shish@keba.be> wrote:
Le 3 mars 2012 10:27, Robert Kern <robert.kern@gmail.com> a écrit :
On Sat, Mar 3, 2012 at 14:34, Robert Kern <robert.kern@gmail.com>
wrote:
On Sat, Mar 3, 2012 at 14:31, Ralf Gommers < ralf.gommers@googlemail.com> wrote:
Because this is also bad: >>> np.<TAB> Display all 561 possibilities? (y or n)
Not as bad as overloading np.allclose(x,y,return_array=True). Or deprecating np.allclose() in favor of np.close().all().
I screwed up this paragraph. I meant that as "Another alternative would be to deprecate ...".
np.close().all() would probably be a lot less efficient in terms of CPU / memory though, wouldn't it?
No. np.allclose() is essentially doing exactly this already.
Ok. What about then, np.allclose() could theoretically be a lot more efficient in terms of CPU / memory? ;)
allclose() does short-circuit in a few cases where the pattern of Inf's doesn't match. E.g. if not all(xinf == isinf(y)): return False if not all(x[xinf] == y[xinf]): return False At least for the function I wrote, allclose() would be a bit faster than isclose().all() in those specific cases. It's not likely to be terribly significant, though. -Joe
-=- Olivier
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (5)
-
Benjamin Root -
Joe Kington -
Olivier Delalleau -
Ralf Gommers -
Robert Kern