PEP 517: Python 3 finder?
Build systems should be able to run under a different version of Python than the one that is running 'pip install'. Does PEP 517 have anything to say about that? Then a flit back end could have a small amount of Python 2.7 compatible interface code and create the wheel with Python 3 anyway.
I'm wondering about this one as well. Even when it is not a different interpreter version it could also be a different environment. Who "owns" environment variables like PYTHONHOME and PYTHONPATH? On Fri, Jun 16, 2017 at 9:13 PM, Daniel Holth <dholth@gmail.com> wrote:
Build systems should be able to run under a different version of Python than the one that is running 'pip install'. Does PEP 517 have anything to say about that? Then a flit back end could have a small amount of Python 2.7 compatible interface code and create the wheel with Python 3 anyway.
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
There's a canonical way to say "I want to run another python process in the same environment as the one I'm already running in". You look at sys.executable and run the binary named there. On Jun 16, 2017 12:32 PM, "Freddy Rietdijk" <freddyrietdijk@fridh.nl> wrote:
I'm wondering about this one as well. Even when it is not a different interpreter version it could also be a different environment. Who "owns" environment variables like PYTHONHOME and PYTHONPATH?
On Fri, Jun 16, 2017 at 9:13 PM, Daniel Holth <dholth@gmail.com> wrote:
Build systems should be able to run under a different version of Python than the one that is running 'pip install'. Does PEP 517 have anything to say about that? Then a flit back end could have a small amount of Python 2.7 compatible interface code and create the wheel with Python 3 anyway.
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
On Jun 16, 2017 12:13 PM, "Daniel Holth" <dholth@gmail.com> wrote: Build systems should be able to run under a different version of Python than the one that is running 'pip install'. Does PEP 517 have anything to say about that? No, and I don't think it should. At the start of the bootstrap process, the *only* thing we know about the system doing the build is that the python being used to run pip is installed. Even if the build system could declare a requirement for some other version of python, then what could we even do with that information? pip3 install python==2.7? I guess I can imagine building some system that tries to find other interpreters, or where you can configure a database of installed interpreters that pip reads, or even making it so that you actually can do the equivalent of "pip3 install python==2.7". But that's all *way* out of scope for this initial PEP. Then a flit back end could have a small amount of Python 2.7 compatible interface code and create the wheel with Python 3 anyway. If someone wants to experiment with this, then it's possible within the PEP 517 framework to write a 2.7-compatible backend that searches the system for a python 3 install and then uses it. I'm not sure it's a good idea, but you can do it :-). For flit in particular I suspect this is unnecessary though. I see three cases: 1. Developers building wheels to release: they can use python 3, no big deal. The resulting wheels will be tagged as py2 or py2.py3 as appropriate. 2. End users installing from pypi: there are wheels on pypi so they never need to run flit. (Key observation here: flit wheels are always pure python, so they work everywhere the project is supported all.) 3. End users who have for whatever reason decided to manually get a source tree (via git checkout or unpacking an sdist), and want to install it: if they can manually get a source tree, they can also manually read and follow directions to use python 3 to build it :-) -n
On Fri, Jun 16, 2017, at 10:01 PM, Nathaniel Smith wrote:
On Jun 16, 2017 12:13 PM, "Daniel Holth" <dholth@gmail.com> wrote:
Build systems should be able to run under a different version of Python than the one that is running 'pip install'. Does PEP 517 have anything to say about that?> No, and I don't think it should.
I agree. This would add considerable complexity to the spec, and then create a headache for frontend implementors - how would pip organise running a backend on another version of Python?
On Jun 16, 2017, at 5:01 PM, Nathaniel Smith <njs@pobox.com> wrote:
If someone wants to experiment with this, then it's possible within the PEP 517 framework to write a 2.7-compatible backend that searches the system for a python 3 install and then uses it. I'm not sure it's a good idea, but you can do it :-).
I just want to hammer this specific point home a bit more. The only real requirement on Python version PEP 517 puts into place is that you write the API that the frontend will call in the version of Python the frontend is running under. What you do beyond that is entirely up to you, including running a different version of Python or something that’s not written in Python at all. The only “downside" to this really is that your options if that build system isn’t available is to have some sort of fallback (in which case why have the other thing at all?) or just error out. However I don’t think this is a solvable downside, because in either case if you’re running on py3 and the build backend needs py2, there’s nothing really for a frontend to do here except bail out. — Donald Stufft
On 17 June 2017 at 10:25, Donald Stufft <donald@stufft.io> wrote:
On Jun 16, 2017, at 5:01 PM, Nathaniel Smith <njs@pobox.com> wrote:
If someone wants to experiment with this, then it's possible within the PEP 517 framework to write a 2.7-compatible backend that searches the system for a python 3 install and then uses it. I'm not sure it's a good idea, but you can do it :-).
I just want to hammer this specific point home a bit more. The only real requirement on Python version PEP 517 puts into place is that you write the API that the frontend will call in the version of Python the frontend is running under. What you do beyond that is entirely up to you, including running a different version of Python or something that’s not written in Python at all.
The only “downside" to this really is that your options if that build system isn’t available is to have some sort of fallback (in which case why have the other thing at all?) or just error out. However I don’t think this is a solvable downside, because in either case if you’re running on py3 and the build backend needs py2, there’s nothing really for a frontend to do here except bail out.
Right, from PEP 517's point of view, running an external build backend like scons/waf/meson under Python 3 to produce Python 2.7 wheels (or even vice-versa) is the same as running any other external binary as part of the build process. The *downside* of doing it that way is that it forces wrappers around those tools (like enscons) to choose between the following two options for 2.7 support: - retain 2.7 compatibility in the backend tool, declare it as a dependency in build-system.requires - run the backend as an external application, losing the ability to automatically install it However, even though I acknowledge that as an unfortunate limitation, I think it would still be better to tackle doing something about it as a separate proposal that improves support for external dependencies. It just highlights that if/when we move forward with that idea, we should have a "pycli" category of external dependencies that specifically handles scripts that can be installed via a tool like `pipsi` based on a PyPI dependency specification (as opposed to arbitrary applications handled by the system package manager). The semantics of a "pycli" external dependency would then be "Install into a dedicated virtual environment using the latest installed Python", with a separate "py2cli" for tools that are currently still Python-2 only. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (6)
-
Daniel Holth -
Donald Stufft -
Freddy Rietdijk -
Nathaniel Smith -
Nick Coghlan -
Thomas Kluyver