More pending test framework changes (please give feedback)
Hi all, I can commit any/all of the following changes as long as they are agreeable to everybody: 1. All doctests in NumPy will have the numpy module available in their execution context as "np". 2. Turn on the normalized whitespace option for all doctests. Having a doctest fail just because there's a space after your result seems like an unnecessary hassle for documenters. 3. Output will be ignored for each doctest expected output line that contains "#random". I figured this can serve both as an ignore flag and indication to the reader that the listed output may differ from what they see if they execute the associated command. So you would be able to do:
random.random() 0.1234567890 #random: output may differ on your system
And have the example executed but not cause a failure. You could also use this to ignore the <SomeObject at 0x1234ABCD> output from plot methods as well. If these get committed, I'll also revert doctests I changed earlier so that they use the provided "np" instead of importing it, and fix any currently failing doctests that can be addressed with the new behavior. I've already got a batch of these changes made locally just to try out the new features, but I'll wait to see if adding the features causes any troubles on the buildbots first. There are a couple of options that I haven't got patches for yet: 4. Doctest commands starting with "plt." or "plot." will automatically have their output ignored. 5. Add a decorator that causes all doctest output for that function/method's docstring to be ignored. (This is for cases with many examples, all of which should be executed but not compared, so you don't have to clutter up each output line with #random or ellipsis) If there is still a big need for these, I'll work on them. As an alternative to 3,4 and 5, you could use the ELLIPSIS directive to ignore the varying bits of the output, but you'd still have to put the directive on each command line where you needed it. Unless I turn on the ellipsis feature for all doctests, that is; it's easy to do, and it doesn't cause me any obvious problems locally. I don't know whether that's something that's generally a good idea, though. Thanks, Alan
On Mon, Jun 30, 2008 at 1:54 PM, Alan McIntyre <alan.mcintyre@gmail.com> wrote:
1. All doctests in NumPy will have the numpy module available in their execution context as "np".
2. Turn on the normalized whitespace option for all doctests. Having a doctest fail just because there's a space after your result seems like an unnecessary hassle for documenters.
3. Output will be ignored for each doctest expected output line that contains "#random". I figured this can serve both as an ignore flag and indication to the reader that the listed output may differ from what they see if they execute the associated command. So you would be able to do:
random.random() 0.1234567890 #random: output may differ on your system
And have the example executed but not cause a failure. You could also use this to ignore the <SomeObject at 0x1234ABCD> output from plot methods as well.
Since I didn't see any objections, these changes are now committed. I'll be updating some doctests to take advantage of them later today. Alan
On Tue, Jul 1, 2008 at 8:26 AM, Alan McIntyre <alan.mcintyre@gmail.com> wrote:
Since I didn't see any objections, these changes are now committed. I'll be updating some doctests to take advantage of them later today.
Excellent. Thanks, -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/
On Tue, Jul 1, 2008 at 9:26 AM, Alan McIntyre <alan.mcintyre@gmail.com> wrote:
On Mon, Jun 30, 2008 at 1:54 PM, Alan McIntyre <alan.mcintyre@gmail.com> wrote:
1. All doctests in NumPy will have the numpy module available in their execution context as "np".
2. Turn on the normalized whitespace option for all doctests. Having a doctest fail just because there's a space after your result seems like an unnecessary hassle for documenters.
3. Output will be ignored for each doctest expected output line that contains "#random". I figured this can serve both as an ignore flag and indication to the reader that the listed output may differ from what they see if they execute the associated command. So you would be able to do:
random.random() 0.1234567890 #random: output may differ on your system
And have the example executed but not cause a failure. You could also use this to ignore the <SomeObject at 0x1234ABCD> output from plot methods as well.
Since I didn't see any objections, these changes are now committed. I'll be updating some doctests to take advantage of them later today.
I note that a lot of unit test files import tons of specific functions, numpy.core, etc., etc. Is there any reason not to fix things up to import numpy as np from numpy.testing import * I fixed one file this way, but I wonder if we shouldn't make all of them work like that. Chuck
On Tue, Jul 1, 2008 at 2:37 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
I note that a lot of unit test files import tons of specific functions, numpy.core, etc., etc. Is there any reason not to fix things up to
import numpy as np from numpy.testing import *
I fixed one file this way, but I wonder if we shouldn't make all of them work like that.
Personally, I prefer the imports to be as simple as possible, but I managed to restrain myself from cleaning up test module imports when I was making my changes. ;) If making them somewhat standardized is desirable, I might as well do it while I'm cleaning up and fixing tests.
On Tue, Jul 1, 2008 at 1:21 PM, Alan McIntyre <alan.mcintyre@gmail.com> wrote:
On Tue, Jul 1, 2008 at 2:37 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
I note that a lot of unit test files import tons of specific functions, numpy.core, etc., etc. Is there any reason not to fix things up to
import numpy as np from numpy.testing import *
I fixed one file this way, but I wonder if we shouldn't make all of them work like that.
Personally, I prefer the imports to be as simple as possible, but I managed to restrain myself from cleaning up test module imports when I was making my changes. ;) If making them somewhat standardized is desirable, I might as well do it while I'm cleaning up and fixing tests.
A lot of the imports seem to have just grown over the years, some even contain duplicates. So I think cleaning up would be a good idea if no one objects. Chuck
On Tue, Jul 1, 2008 at 3:37 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
A lot of the imports seem to have just grown over the years, some even contain duplicates. So I think cleaning up would be a good idea if no one objects.
Ok. As a pre-emptive clarification, I'll only be tweaking imports in unit test files--I don't want to mess with any of the magic that goes on in the package imports. ;)
participants (3)
-
Alan McIntyre -
Charles R Harris -
Jarrod Millman