Hi All, Just looking for opinions and feedback on the need to keep NumPy from having a hard nose/pytest dependency. The options as I see them are: 1. pytest is never imported until the tests are run -- current practice with nose 2. pytest is never imported unless the testfiles are imported -- what I would like 3. pytest is imported together when numpy is -- what we need to avoid. Currently the approach has been 1), but I think 2) makes more sense and allows more flexibility. Thoughts? Chuck
On Tue, 2017-07-11 at 14:49 -0600, Charles R Harris wrote:
Hi All,
Just looking for opinions and feedback on the need to keep NumPy from having a hard nose/pytest dependency. The options as I see them are:
pytest is never imported until the tests are run -- current practice with nose pytest is never imported unless the testfiles are imported -- what I would like pytest is imported together when numpy is -- what we need to avoid. Currently the approach has been 1), but I think 2) makes more sense and allows more flexibility.
I am not quite sure about everything here. My guess is we can do whatever we want when it comes to our own tests, and I don't mind just switching everything to pytest (I for one am happy as long as I can run `runtests.py` ;)). When it comes to the utils we provide, those should keep working without nose/pytest if they worked before without it I think. My guess is that all your options do that, so I think we should take the one that gives the nicest maintainable code :). Though can't say I looked enough into it to really make a well educated decision, that probably means your option 2. - Sebastian
Thoughts? Chuck _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Going with option 2 is probably the best option so that you can use pytest fixtures and parameterization. Might be worth looking at how Matplotlib re-arranged things on our master branch to maintain back-compatibility with nose-specific tools that were used by down-stream projects. Tom On Tue, Jul 11, 2017 at 4:22 PM Sebastian Berg <sebastian@sipsolutions.net> wrote:
On Tue, 2017-07-11 at 14:49 -0600, Charles R Harris wrote:
Hi All,
Just looking for opinions and feedback on the need to keep NumPy from having a hard nose/pytest dependency. The options as I see them are:
pytest is never imported until the tests are run -- current practice with nose pytest is never imported unless the testfiles are imported -- what I would like pytest is imported together when numpy is -- what we need to avoid. Currently the approach has been 1), but I think 2) makes more sense and allows more flexibility.
I am not quite sure about everything here. My guess is we can do whatever we want when it comes to our own tests, and I don't mind just switching everything to pytest (I for one am happy as long as I can run `runtests.py` ;)). When it comes to the utils we provide, those should keep working without nose/pytest if they worked before without it I think.
My guess is that all your options do that, so I think we should take the one that gives the nicest maintainable code :). Though can't say I looked enough into it to really make a well educated decision, that probably means your option 2.
- Sebastian
Thoughts? Chuck _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
On Tue, Jul 11, 2017 at 5:04 PM, Thomas Caswell <tcaswell@gmail.com> wrote:
Going with option 2 is probably the best option so that you can use pytest fixtures and parameterization.
I agree -- those are worth a lot! -CHB
Might be worth looking at how Matplotlib re-arranged things on our master branch to maintain back-compatibility with nose-specific tools that were used by down-stream projects.
Tom
On Tue, Jul 11, 2017 at 4:22 PM Sebastian Berg <sebastian@sipsolutions.net> wrote:
On Tue, 2017-07-11 at 14:49 -0600, Charles R Harris wrote:
Hi All,
Just looking for opinions and feedback on the need to keep NumPy from having a hard nose/pytest dependency. The options as I see them are:
pytest is never imported until the tests are run -- current practice with nose pytest is never imported unless the testfiles are imported -- what I would like pytest is imported together when numpy is -- what we need to avoid. Currently the approach has been 1), but I think 2) makes more sense and allows more flexibility.
I am not quite sure about everything here. My guess is we can do whatever we want when it comes to our own tests, and I don't mind just switching everything to pytest (I for one am happy as long as I can run `runtests.py` ;)). When it comes to the utils we provide, those should keep working without nose/pytest if they worked before without it I think.
My guess is that all your options do that, so I think we should take the one that gives the nicest maintainable code :). Though can't say I looked enough into it to really make a well educated decision, that probably means your option 2.
- Sebastian
Thoughts? Chuck _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
-- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Wed, Jul 12, 2017 at 11:06 AM, Chris Barker <chris.barker@noaa.gov> wrote:
On Tue, Jul 11, 2017 at 5:04 PM, Thomas Caswell <tcaswell@gmail.com> wrote:
Going with option 2 is probably the best option so that you can use pytest fixtures and parameterization.
I agree -- those are worth a lot!
Maybe I'm dense, but I don't quite see the difference between 1 and 2. Test files should never be imported unless tests are run, they're not part of any public API nor do they currently have __init__.py files. Ralf
-CHB
Might be worth looking at how Matplotlib re-arranged things on our master branch to maintain back-compatibility with nose-specific tools that were used by down-stream projects.
Tom
On Tue, Jul 11, 2017 at 4:22 PM Sebastian Berg < sebastian@sipsolutions.net> wrote:
On Tue, 2017-07-11 at 14:49 -0600, Charles R Harris wrote:
Hi All,
Just looking for opinions and feedback on the need to keep NumPy from having a hard nose/pytest dependency. The options as I see them are:
pytest is never imported until the tests are run -- current practice with nose pytest is never imported unless the testfiles are imported -- what I would like pytest is imported together when numpy is -- what we need to avoid. Currently the approach has been 1), but I think 2) makes more sense and allows more flexibility.
I am not quite sure about everything here. My guess is we can do whatever we want when it comes to our own tests, and I don't mind just switching everything to pytest (I for one am happy as long as I can run `runtests.py` ;)). When it comes to the utils we provide, those should keep working without nose/pytest if they worked before without it I think.
My guess is that all your options do that, so I think we should take the one that gives the nicest maintainable code :). Though can't say I looked enough into it to really make a well educated decision, that probably means your option 2.
- Sebastian
Thoughts? Chuck _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion___
NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
--
Christopher Barker, Ph.D. Oceanographer
Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@noaa.gov
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
On Wed, Jul 12, 2017 at 1:26 AM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Wed, Jul 12, 2017 at 11:06 AM, Chris Barker <chris.barker@noaa.gov> wrote:
On Tue, Jul 11, 2017 at 5:04 PM, Thomas Caswell <tcaswell@gmail.com> wrote:
Going with option 2 is probably the best option so that you can use pytest fixtures and parameterization.
I agree -- those are worth a lot!
Maybe I'm dense, but I don't quite see the difference between 1 and 2. Test files should never be imported unless tests are run, they're not part of any public API nor do they currently have __init__.py files.
Ralf
In practice, that would generally be true, but the nose testing tools were 1, all nose imports were buried in functions that ran during testing. Whether or not that was by intent I don't know. But having an explicit consensus on 2, which seems to be the case here, is helpful because it allows better use of pytest fixtures. <snip> Chuck
Charles R Harris kirjoitti 12.07.2017 klo 13:53:
In practice, that would generally be true, but the nose testing tools were 1, all nose imports were buried in functions that ran during testing. Whether or not that was by intent I don't know. But having an explicit consensus on 2, which seems to be the case here, is helpful because it allows better use of pytest fixtures.
I guess the question is about shipping new pytest fixtures as a part of the public API of numpy.testing, for use by 3rd party projects. If the issue is only with Numpy's own tests, they can import stuff from a private submodule that's not imported by "import numpy.testing", so it does not introduce a dependency. (Similar thing for the public API might also be possible e.g. "import numpy.testing.pytest_fixtures" but it comes at the cost of a new submodule.) So I guess a main question actually is: how much of the public API in numpy.testing should be ported to pytest for use by 3rd projects? The numerical assert functions are obviously useful. The warnings suppression (pytest warning stuff IIRC doesn't deal with warning registries nor work around the bugs in warnings.catch_warnings) similarly --- it could make sense to actually upstream it... But I'm not so clear about the rest. Pauli
Charles R Harris kirjoitti 12.07.2017 klo 13:53:
In practice, that would generally be true, but the nose testing tools were 1, all nose imports were buried in functions that ran during testing. Whether or not that was by intent I don't know. But having an explicit consensus on 2, which seems to be the case here, is helpful because it allows better use of pytest fixtures.
I guess the question is about shipping new pytest fixtures as a part of the public API of numpy.testing, for use by 3rd party projects. If the issue is only with Numpy's own tests, they can import stuff from a private submodule that's not imported by "import numpy.testing", so it does not introduce a dependency. (Similar thing for the public API might also be possible e.g. "import numpy.testing.pytest_fixtures" but it comes at the cost of a new submodule.) So I guess a main question actually is: how much of the public API in numpy.testing should be ported to pytest for use by 3rd projects? The numerical assert functions are obviously useful. The warnings suppression (pytest warning stuff IIRC doesn't deal with warning registries nor work around the bugs in warnings.catch_warnings) similarly --- it could make sense to actually upstream it... But I'm not so clear about the rest. Pauli
On Thu, Jul 13, 2017 at 8:14 AM, Pauli Virtanen <pav@iki.fi> wrote:
Charles R Harris kirjoitti 12.07.2017 klo 13:53:
In practice, that would generally be true, but the nose testing tools were 1, all nose imports were buried in functions that ran during testing. Whether or not that was by intent I don't know. But having an explicit consensus on 2, which seems to be the case here, is helpful because it allows better use of pytest fixtures.
I guess the question is about shipping new pytest fixtures as a part of the public API of numpy.testing, for use by 3rd party projects.
Agreed. That's a different question, and I'd prefer to keep things as they are in that respect. Otherwise it's basically a hard dependency of numpy itself on pytest.
If the issue is only with Numpy's own tests, they can import stuff from a private submodule that's not imported by "import numpy.testing", so it does not introduce a dependency.
(Similar thing for the public API might also be possible e.g. "import numpy.testing.pytest_fixtures" but it comes at the cost of a new submodule.)
So I guess a main question actually is: how much of the public API in numpy.testing should be ported to pytest for use by 3rd projects?
The numerical assert functions are obviously useful.
The warnings suppression (pytest warning stuff IIRC doesn't deal with warning registries nor work around the bugs in warnings.catch_warnings) similarly --- it could make sense to actually upstream it...
But I'm not so clear about the rest.
Agreed, nothing in the decorators that obviously needs a pytest-based implementation. The Tester class may be the one thing. Ralf
Charles R Harris kirjoitti 12.07.2017 klo 13:53:
In practice, that would generally be true, but the nose testing tools were 1, all nose imports were buried in functions that ran during testing. Whether or not that was by intent I don't know. But having an explicit consensus on 2, which seems to be the case here, is helpful because it allows better use of pytest fixtures.
I guess the question is about shipping new pytest fixtures as a part of the public API of numpy.testing, for use by 3rd party projects. If the issue is only with Numpy's own tests, they can import stuff from a private submodule that's not imported by "import numpy.testing", so it does not introduce a dependency. (Similar thing for the public API might also be possible e.g. "import numpy.testing.pytest_fixtures" but it comes at the cost of a new submodule.) So I guess a main question actually is: how much of the public API in numpy.testing should be ported to pytest for use by 3rd projects? The numerical assert functions are obviously useful. The warnings suppression (pytest warning stuff IIRC doesn't deal with warning registries nor work around the bugs in warnings.catch_warnings) similarly --- it could make sense to actually upstream it... But I'm not so clear about the rest. Pauli
participants (6)
-
Charles R Harris -
Chris Barker -
Pauli Virtanen -
Ralf Gommers -
Sebastian Berg -
Thomas Caswell