Howdy, In working on the ipython testing machinery, I looked at the numpy nosetester.py file and found that it works by monkeypatching nose itself. I'm curious as to why this approach was taken rather than constructing a plugin object. In general, monkeypatching should be done as a last-resort trick, because it tends to be brittle and can cause bizarre problems to users who after running numpy.test() find that their normal nose-using code starts doing funny things. Any thoughts/insights? Cheers, f
On Wed, Jul 16, 2008 at 20:42, Fernando Perez <fperez.net@gmail.com> wrote:
Howdy,
In working on the ipython testing machinery, I looked at the numpy nosetester.py file and found that it works by monkeypatching nose itself. I'm curious as to why this approach was taken rather than constructing a plugin object. In general, monkeypatching should be done as a last-resort trick, because it tends to be brittle and can cause bizarre problems to users who after running numpy.test() find that their normal nose-using code starts doing funny things.
Any thoughts/insights?
Is there a way to do it programatically without requiring numpy to be installed with setuptools? -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On Wed, Jul 16, 2008 at 7:00 PM, Robert Kern <robert.kern@gmail.com> wrote:
Is there a way to do it programatically without requiring numpy to be installed with setuptools?
I think so, though I'm not 100% certain because I haven't finished the ipython work. So far what I have for ip is all nose mods done as a nose plugin. Right now that plugin needs to be installed as a true plugin (i.e. via setuptools), but IPython does NOT need to be installed via st. What I need to add is a way to run the testing via a python script (right now I use the command line, hence the requirement for the plugin to be really available to nose) that would correctly load and configure everything needed. I think under this scenario, it should be possible to load this plugin from a private package (IPython.testing.plugin) instead of the nose namespace, but that's the part I have yet to confirm with an actual implementation. Cheers, f
On Wed, Jul 16, 2008 at 10:00 PM, Robert Kern <robert.kern@gmail.com> wrote:
Is there a way to do it programatically without requiring numpy to be installed with setuptools?
There is; you have to pass a list of plugin instances to the constructor of TestProgram--all plugins that you might want to use, even the builtin ones. (As far as I know, that is.) The monkeypatching approach was the first one that I could make to work with the least amount of hassle, but it's definitely not the best way. I only had to monkeypatch a couple of things at first, but as I figured out what the test framework needed to do, it just got worse, so I was beginning to get uncomfortable with it myself. (Honest! :) Once the NumPy and SciPy test suites are mostly fixed up to work under the current rules, I'll go back and use a method that doesn't require monkeypatching. It shouldn't have any effect on the public interface or the tests themselves. Since we're discussing this sort of thing, there's something I've been meaning to ask anyway: do we really need to allow end users to pass in arbitrary extra arguments to nose (via the extra_argv in test())? This seems to lock us in to having a mostly unobstructed path from test() through to an uncustomized nose backend.
On Wed, Jul 16, 2008 at 22:21, Alan McIntyre <alan.mcintyre@gmail.com> wrote:
On Wed, Jul 16, 2008 at 10:00 PM, Robert Kern <robert.kern@gmail.com> wrote:
Is there a way to do it programatically without requiring numpy to be installed with setuptools?
There is; you have to pass a list of plugin instances to the constructor of TestProgram--all plugins that you might want to use, even the builtin ones. (As far as I know, that is.)
The monkeypatching approach was the first one that I could make to work with the least amount of hassle, but it's definitely not the best way. I only had to monkeypatch a couple of things at first, but as I figured out what the test framework needed to do, it just got worse, so I was beginning to get uncomfortable with it myself. (Honest! :) Once the NumPy and SciPy test suites are mostly fixed up to work under the current rules, I'll go back and use a method that doesn't require monkeypatching. It shouldn't have any effect on the public interface or the tests themselves.
Sounds good.
Since we're discussing this sort of thing, there's something I've been meaning to ask anyway: do we really need to allow end users to pass in arbitrary extra arguments to nose (via the extra_argv in test())? This seems to lock us in to having a mostly unobstructed path from test() through to an uncustomized nose backend.
At least with other projects, I occasionally want to do things like run with --pdb-failure or --detailed-errors, etc. What exactly is extra_argv blocking? My preference, actually, is for the nosetests command to be able to run our tests correctly if at all possible. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On Wed, Jul 16, 2008 at 11:32 PM, Robert Kern <robert.kern@gmail.com> wrote:
Since we're discussing this sort of thing, there's something I've been meaning to ask anyway: do we really need to allow end users to pass in arbitrary extra arguments to nose (via the extra_argv in test())? This seems to lock us in to having a mostly unobstructed path from test() through to an uncustomized nose backend.
At least with other projects, I occasionally want to do things like run with --pdb-failure or --detailed-errors, etc. What exactly is extra_argv blocking?
It's not blocking anything; it just feels wrong for some reason. Probably because I've been duck-punching nose and doctest to death to make them act the way I want, and I can't fit all the doctest/nose/unittest behavior in my head all at once to comfortably say that any of those other options will still work correctly. ;) It's probably just a pointless worry that will be moot after all the monkeypatching is removed, since the underlying test libraries will be in an unaltered state.
My preference, actually, is for the nosetests command to be able to run our tests correctly if at all possible.
The unit tests will run just fine via nosetests, but the doctests generally will not, because of the limited execution context NoseTester now enforces on them.
On Wed, Jul 16, 2008 at 22:52, Alan McIntyre <alan.mcintyre@gmail.com> wrote:
On Wed, Jul 16, 2008 at 11:32 PM, Robert Kern <robert.kern@gmail.com> wrote:
Since we're discussing this sort of thing, there's something I've been meaning to ask anyway: do we really need to allow end users to pass in arbitrary extra arguments to nose (via the extra_argv in test())? This seems to lock us in to having a mostly unobstructed path from test() through to an uncustomized nose backend.
At least with other projects, I occasionally want to do things like run with --pdb-failure or --detailed-errors, etc. What exactly is extra_argv blocking?
It's not blocking anything; it just feels wrong for some reason. Probably because I've been duck-punching nose and doctest to death to make them act the way I want, and I can't fit all the doctest/nose/unittest behavior in my head all at once to comfortably say that any of those other options will still work correctly. ;)
It's probably just a pointless worry that will be moot after all the monkeypatching is removed, since the underlying test libraries will be in an unaltered state.
That's what I expect.
My preference, actually, is for the nosetests command to be able to run our tests correctly if at all possible.
The unit tests will run just fine via nosetests, but the doctests generally will not, because of the limited execution context NoseTester now enforces on them.
Personally, I could live with that. I don't see the extra options as very useful for testing examples. However, I would prefer to leave the capability there until a concrete practical problem arises. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On Wed, Jul 16, 2008 at 8:21 PM, Alan McIntyre <alan.mcintyre@gmail.com> wrote:
The monkeypatching approach was the first one that I could make to work with the least amount of hassle, but it's definitely not the best way. I only had to monkeypatch a couple of things at first, but as I figured out what the test framework needed to do, it just got worse, so I was beginning to get uncomfortable with it myself. (Honest! :) Once the NumPy and SciPy test suites are mostly fixed up to work under the current rules, I'll go back and use a method that doesn't require monkeypatching. It shouldn't have any effect on the public interface or the tests themselves.
Great. As I mentioned, the code I sent a few days ago already has a non-monkeypatching plugin you can use as a starting point.
Since we're discussing this sort of thing, there's something I've been meaning to ask anyway: do we really need to allow end users to pass in arbitrary extra arguments to nose (via the extra_argv in test())? This seems to lock us in to having a mostly unobstructed path from test() through to an uncustomized nose backend.
As RK said, it's often handy to be able to run the tests from the plain command line to tweak nose's behavior. What I plan for ipython is to have a little script (installed to $PREFIX/bin) that loads all the necessary machinery but otherwise works like 'nosetests --necessary-options'. This will let us run the ipython tests from the command line (and adding additional nose flags as required) but without users needing to install the plugin. I think it's perfectly doable, I just haven't finished it. If you don't like for numpy the script option, then users would need to have the plugin installed when working at the command line (though a python numpy.test() call wouldn't need that, since it can do the plugin loading itself). Cheers, f
participants (3)
-
Alan McIntyre -
Fernando Perez -
Robert Kern