Re: [Python-ideas] float comparison in doctes
It seems that this didn't reach the list directly (see https://mail.python.org/pipermail/python-ideas/2014-August/028956.html), so I'm resending: Erik Bray (the author of the +FLOAT_CMP extension in Astropy), Bruce Leban, and I had a short off-thread email discussion. Here are the points: - [Bruce]: ALMOST_EQUAL is the best flag name. - [Erik]: If there's agreement on this, Erik will develop a patch as soon as he can. - [Erik]: There's no way to adjust the tolerance because there seems to be no easy way to parameterize doctest flags. Ideas are welcome. - [Erik]: Still, "This +FLOAT_CMP flag enabled removing tons of ellipses from the test outputs [of Astropy], and restoring the full outputs which certainly read better in the docs... For more complete unit tests of course we use assert_almost_equal type functions. - [Erik]: This PR is a better link than the one I gave: https://github.com/astropy/astropy/pull/2087 - [Erik]: Most of the code is from the SymPy project with improvements. Erik had started on a similar feature when he found that their implementation was further developed. Kevin
On Mon, Sep 15, 2014 at 12:02:20PM -0700, Kevin Davies wrote:
It seems that this didn't reach the list directly (see https://mail.python.org/pipermail/python-ideas/2014-August/028956.html), so I'm resending:
Erik Bray (the author of the +FLOAT_CMP extension in Astropy), Bruce Leban, and I had a short off-thread email discussion. Here are the points:
- [Bruce]: ALMOST_EQUAL is the best flag name. - [Erik]: If there's agreement on this, Erik will develop a patch as soon as he can. - [Erik]: There's no way to adjust the tolerance because there seems to be no easy way to parameterize doctest flags. Ideas are welcome.
With no way to choose between (at minimum) *four* different "almost equal" models, and no way to specify a tolerance, I don't think doctest ought to have such a directive. Almost equal can mean: - round and compare with == (as unittest does) - absolute difference - relative difference - ULP difference Given what a blunt instrument doctest is, I think the nicest solution is also the most explicit: just use ellipses.
- [Erik]: Still, "This +FLOAT_CMP flag enabled removing tons of ellipses from the test outputs [of Astropy], and restoring the full outputs which certainly read better in the docs... For more complete unit tests of course we use assert_almost_equal type functions. - [Erik]: This PR is a better link than the one I gave: https://github.com/astropy/astropy/pull/2087 - [Erik]: Most of the code is from the SymPy project with improvements. Erik had started on a similar feature when he found that their implementation was further developed.
On 16 Sep 2014 06:32, "Steven D'Aprano"
On Mon, Sep 15, 2014 at 12:02:20PM -0700, Kevin Davies wrote:
It seems that this didn't reach the list directly (see https://mail.python.org/pipermail/python-ideas/2014-August/028956.html), so I'm resending:
Erik Bray (the author of the +FLOAT_CMP extension in Astropy), Bruce Leban, and I had a short off-thread email discussion. Here are the points:
- [Bruce]: ALMOST_EQUAL is the best flag name. - [Erik]: If there's agreement on this, Erik will develop a patch as
soon as he can.
- [Erik]: There's no way to adjust the tolerance because there seems to be no easy way to parameterize doctest flags. Ideas are welcome.
With no way to choose between (at minimum) *four* different "almost equal" models, and no way to specify a tolerance, I don't think doctest ought to have such a directive.
Almost equal can mean:
- round and compare with == (as unittest does) - absolute difference - relative difference - ULP difference
I think it's OK for a doctest flag to just provide the default behaviour offered by unittest.TestCase.assertAlmostEqual. That aligns well with the originally intended use case of testing examples in documentation. If folks want more precise control, they can then switch to full unit tests. It would be reasonable for the docs for the new flag to point that out explicitly. Cheers, Nick.
On Mon, Sep 15, 2014 at 7:55 PM, Nick Coghlan
On 16 Sep 2014 06:32, "Steven D'Aprano"
wrote: On Mon, Sep 15, 2014 at 12:02:20PM -0700, Kevin Davies wrote:
It seems that this didn't reach the list directly (see https://mail.python.org/pipermail/python-ideas/2014-August/028956.html), so I'm resending:
Erik Bray (the author of the +FLOAT_CMP extension in Astropy), Bruce Leban, and I had a short off-thread email discussion. Here are the points:
- [Bruce]: ALMOST_EQUAL is the best flag name. - [Erik]: If there's agreement on this, Erik will develop a patch as soon as he can. - [Erik]: There's no way to adjust the tolerance because there seems to be no easy way to parameterize doctest flags. Ideas are welcome.
With no way to choose between (at minimum) *four* different "almost equal" models, and no way to specify a tolerance, I don't think doctest ought to have such a directive.
Almost equal can mean:
- round and compare with == (as unittest does) - absolute difference - relative difference - ULP difference
I think it's OK for a doctest flag to just provide the default behaviour offered by unittest.TestCase.assertAlmostEqual. That aligns well with the originally intended use case of testing examples in documentation.
I think that sounds like a perfect compromise. As long as it's documented that "this is equivalent to assertAlmostEqual with the default arguments" it should be clear, and that would handle all the basic use cases this feature was intended for. Naming the flag ALMOST_EQUAL would also help make that clear.
If folks want more precise control, they can then switch to full unit tests. It would be reasonable for the docs for the new flag to point that out explicitly.
Exactly--it's mostly a matter of making doctests more readable. Note, ellipses won't work at all in some cases either. I've had cases where we expected something like 1.0 and got 0.99999999..., for example. Erik
On 9/15/2014 3:02 PM, Kevin Davies wrote:
Erik Bray (the author of the +FLOAT_CMP extension in Astropy), Bruce Leban, and I had a short off-thread email discussion. Here are the points:
- [Bruce]: ALMOST_EQUAL is the best flag name.
Agreed. I matches assertAlmostEqual.
- [Erik]: If there's agreement on this, Erik will develop a patch as soon as he can.
I say go ahead.
- [Erik]: There's no way to adjust the tolerance because there seems to be no easy way to parameterize doctest flags. Ideas are welcome.
After thinking about it more, I think this is OK for the purposes of doctests.
- [Erik]: Still, "This +FLOAT_CMP flag enabled removing tons of ellipses from the test outputs [of Astropy], and restoring the full outputs which certainly read better in the docs... For more complete unit tests of course we use assert_almost_equal type functions.
So I would tune the doctest almost-equal compare algorithm to cover most cases, based on your experience, and leave exact tuning and broad domain coverage to unittests.
- [Erik]: This PR is a better link than the one I gave: https://github.com/astropy/astropy/pull/2087 - [Erik]: Most of the code is from the SymPy project with improvements. Erik had started on a similar feature when he found that their implementation was further developed.
-- Terry Jan Reedy
participants (5)
-
Erik Bray
-
Kevin Davies
-
Nick Coghlan
-
Steven D'Aprano
-
Terry Reedy