namespace for backported stdlib modules?
I want to make the improvements in unittest (which transitively includes mock and traceback and perhaps inspect ...) available for folk that can't use 3.5 yet. We've currently got the unittest2 and mock as modules on pypi, and clearly we can add another e.g. traceback2 as a way to get the traceback changes out there. I'm wondering though if perhaps putting up a namespace package on pypi for things in the stdlib - e.g. 'stdlib' (or stdlib2 or something) would make sense. In particular that might permit relative imports to automatically pick up the other also backported modules without source changes. So we'd have e.g. stdlib.unittest stdlib.traceback stdlib.mock And then using that would be done through the normal try: from stdlib import unitest except ImportError: import unittest dance. -Rob -- Robert Collins <rbtcollins@hp.com> Distinguished Technologist HP Converged Cloud
On 21 October 2014 14:40, Terry Reedy <tjreedy@udel.edu> wrote:
I intend it to be used on still-supported earlier versions of 3.x. What do you think of the idea itself? E.g. is the cost of transition from unittest2 -> $NAMESPACE.unittest worth doing? -Rob -- Robert Collins <rbtcollins@hp.com> Distinguished Technologist HP Converged Cloud
On Oct 20, 2014 8:25 PM, "Robert Collins" <robertc@robertcollins.net> wrote:
There are several packages already named backports.{{module}}. I think they all use the same namespace too. They're also all maintained by people not working on the stdlib. For example, backports.ssl is an attempt to backport python 3's ssl features to python 2 using pyOpenSSL
Hi, 2014-10-21 3:18 GMT+02:00 Robert Collins <robertc@robertcollins.net>:
Why not contributing to the unittest2 module instead of creating yet another project? Would it be possible to work on unittest2 and mock to make them "compatible"?
In my experience, namespaces with Python < 3.3 only means pain. I only got issues with the "oslo" namespace created with a hack in a .pth file for oslo.config, oslo.rootwrap, etc. FYI there a list on the Python wiki: https://wiki.python.org/moin/StandardLibraryBackports Victor
On 21 October 2014 21:09, Victor Stinner <victor.stinner@gmail.com> wrote:
Huh, thats exactly what I'm doing. Remember: unittest2 *is* unittest, just broken out and shipped on PyPI for older Python releases. I'm talking purely about the mechanics of doing that distribution and examining ways I might make it cheaper: unittest2 hasn't been updated in ~2 years and if we're going to keep it up to date it needs to be low overhead.
Would it be possible to work on unittest2 and mock to make them "compatible"?
Not sure what you mean here.
Yeah, 3.3 makes it lovely, and I put some patches into importlib2 (same as unittest2 - a backport of importlib for older pythons) to make that better, but its got a bit of a thorny issue to resolve w.r.t. relative imports before it can be used as the default importer.
FYI there a list on the Python wiki: https://wiki.python.org/moin/StandardLibraryBackports
Thanks. Turns out thats incomplete, I'll figure out a login for that later and see about adding importlib2. -Rob -- Robert Collins <rbtcollins@hp.com> Distinguished Technologist HP Converged Cloud
https://pypi.python.org/pypi/backports/1.0 Question now is whether its active :) On 22 October 2014 04:43, Ethan Furman <ethan@stoneleaf.us> wrote:
-- Robert Collins <rbtcollins@hp.com> Distinguished Technologist HP Converged Cloud
I think the decentralized namespace idea is doomed, for several reasons: - The namespace is not private and anyone can create a backport there without adhering to any standards. This has the problem of false security for potential users and code that is not thoroughly tested. - Some projects flatten the name of the backport, specially when they just want to backport one function. Which result in awfully long names. - You need to import all the namespace boilerplate in your backport project. - Licensing: Not all people put their backports with the PSF license. - it is not clear from which version/revision things were backported. - at the end, one still needs to have this conditional imports that force you to have this compatibility layer, so naming is not that important. What I would do is instead of having a namespace is to actually have just one project (no strong opinions about the name) with community curated backports. The project should/could: - enforce the PSF license - define different namespaces for the different backports, like backport27, backport33, backport34 - keep track of revision used in the latest backport and notify to maintainers of changes - be tested in all platforms specified - have guidelines for backporters - provide diffs of what was changed and limitations of the backport - generate different packages for different python versions or maybe even allow people to select their own bundle. - it could even provide some import magic like: import backports33 backports33.install('shutil.wich') import shutil shutil.wich(...) Note: I am maintaining two backports: https://pypi.python.org/pypi/backport_ipaddress and https://pypi.python.org/pypi/backport_collections. On Tue, Oct 21, 2014 at 9:48 PM, Robert Collins <robertc@robertcollins.net> wrote:
-- Sebastian Kreft
On 21 October 2014 14:40, Terry Reedy <tjreedy@udel.edu> wrote:
I intend it to be used on still-supported earlier versions of 3.x. What do you think of the idea itself? E.g. is the cost of transition from unittest2 -> $NAMESPACE.unittest worth doing? -Rob -- Robert Collins <rbtcollins@hp.com> Distinguished Technologist HP Converged Cloud
On Oct 20, 2014 8:25 PM, "Robert Collins" <robertc@robertcollins.net> wrote:
There are several packages already named backports.{{module}}. I think they all use the same namespace too. They're also all maintained by people not working on the stdlib. For example, backports.ssl is an attempt to backport python 3's ssl features to python 2 using pyOpenSSL
Hi, 2014-10-21 3:18 GMT+02:00 Robert Collins <robertc@robertcollins.net>:
Why not contributing to the unittest2 module instead of creating yet another project? Would it be possible to work on unittest2 and mock to make them "compatible"?
In my experience, namespaces with Python < 3.3 only means pain. I only got issues with the "oslo" namespace created with a hack in a .pth file for oslo.config, oslo.rootwrap, etc. FYI there a list on the Python wiki: https://wiki.python.org/moin/StandardLibraryBackports Victor
On 21 October 2014 21:09, Victor Stinner <victor.stinner@gmail.com> wrote:
Huh, thats exactly what I'm doing. Remember: unittest2 *is* unittest, just broken out and shipped on PyPI for older Python releases. I'm talking purely about the mechanics of doing that distribution and examining ways I might make it cheaper: unittest2 hasn't been updated in ~2 years and if we're going to keep it up to date it needs to be low overhead.
Would it be possible to work on unittest2 and mock to make them "compatible"?
Not sure what you mean here.
Yeah, 3.3 makes it lovely, and I put some patches into importlib2 (same as unittest2 - a backport of importlib for older pythons) to make that better, but its got a bit of a thorny issue to resolve w.r.t. relative imports before it can be used as the default importer.
FYI there a list on the Python wiki: https://wiki.python.org/moin/StandardLibraryBackports
Thanks. Turns out thats incomplete, I'll figure out a login for that later and see about adding importlib2. -Rob -- Robert Collins <rbtcollins@hp.com> Distinguished Technologist HP Converged Cloud
https://pypi.python.org/pypi/backports/1.0 Question now is whether its active :) On 22 October 2014 04:43, Ethan Furman <ethan@stoneleaf.us> wrote:
-- Robert Collins <rbtcollins@hp.com> Distinguished Technologist HP Converged Cloud
I think the decentralized namespace idea is doomed, for several reasons: - The namespace is not private and anyone can create a backport there without adhering to any standards. This has the problem of false security for potential users and code that is not thoroughly tested. - Some projects flatten the name of the backport, specially when they just want to backport one function. Which result in awfully long names. - You need to import all the namespace boilerplate in your backport project. - Licensing: Not all people put their backports with the PSF license. - it is not clear from which version/revision things were backported. - at the end, one still needs to have this conditional imports that force you to have this compatibility layer, so naming is not that important. What I would do is instead of having a namespace is to actually have just one project (no strong opinions about the name) with community curated backports. The project should/could: - enforce the PSF license - define different namespaces for the different backports, like backport27, backport33, backport34 - keep track of revision used in the latest backport and notify to maintainers of changes - be tested in all platforms specified - have guidelines for backporters - provide diffs of what was changed and limitations of the backport - generate different packages for different python versions or maybe even allow people to select their own bundle. - it could even provide some import magic like: import backports33 backports33.install('shutil.wich') import shutil shutil.wich(...) Note: I am maintaining two backports: https://pypi.python.org/pypi/backport_ipaddress and https://pypi.python.org/pypi/backport_collections. On Tue, Oct 21, 2014 at 9:48 PM, Robert Collins <robertc@robertcollins.net> wrote:
-- Sebastian Kreft
participants (9)
-
Antoine Pitrou -
Barry Warsaw -
Ethan Furman -
Georg Brandl -
Ian Cordasco -
Robert Collins -
Sebastian Kreft -
Terry Reedy -
Victor Stinner