PEP 477: selected ensurepip backports for Python 2.7
Earlier versions of PEP 453 proposed bootstrapping pip into a Python 2.7 maintenance release in addition to including it with Python 3.4. That part of the proposal proved to be controversial, so we dropped it from the original PEP in order to focus on meeting the Python 3.4 specific release deadlines. This also had the benefit of working out the kinks in the bootstrapping processing as part of the Python 3.4 release cycle. However, we still think we should start providing pip by default to Python 2.7 users as well, at least as part of the Windows and Mac OS X installers. One notable difference from PEP 453 is that because there is no venv module in 2.7, and hence no integration between venv and ensurepip, we can give redistributors the option of just disabling ensurepip entirely and redirecting users to platform specific installation tools. Regards, Nick. ====================== PEP: 477 Title: Backport ensurepip (PEP 453) to Python 2.7 Version: $Revision$ Last-Modified: $Date$ Author: Donald Stufft <donald@stufft.io> Nick Coghlan <ncoghlan@gmail.com> Status: Active Type: Process Content-Type: text/x-rst Created: 26-Aug-2014 Post-History: 1-Sep-2014 Abstract ======== This PEP proposes that the ``ensurepip`` module, added to Python 3.4 by PEP 453, be backported to Python 2.7. It also proposes that automatic invocation of ``ensurepip`` be added to the Python 2.7 Windows and OSX installers. However it does **not** propose that automatic invocation be added to the ``Makefile``. It also proposes that the documentation changes for the package distribution and installation guides be updated to match that in 3.4, which references using the ``ensurepip`` module to bootstrap the installer. Rationale ========= Python 2.7 is effectively a LTS release of Python which represents the end of the 2.x series and there is still a very large contingent of users whom are still using Python 2.7 as their primary version. These users, in order to participate in the wider Python ecosystem, must manually attempt to go out and find the correct way to bootstrap the packaging tools. It is the opinion of this PEP that making it as easy as possible for end users to participate in the wider Python ecosystem is important for 3 primary reasons: 1. The Python 2.x to 3.x migration has a number of painpoints that are eased by a number of third party modules such as six [#six]_, modernize [#modernize]_, or future [#future]_. However relying on these tools requires that everyone who uses the project have a tool to install these packages. 2. In addition to tooling to aid in migration from Python 2.x to 3.x, there are also a number of modules that are *new* in Python 3 for which there are backports available on PyPI. This can also aid in the ability for people to write 2.x and 3.x compatible software as well as enable them to use some of the newer features of Python 3 on Python 2. 3. Users also will need a number of tools in order to create python packages that conform to the newer standards that are being proposed. Things like setuptools [#setuptools]_, Wheel [#wheel]_, and twine [#twine]_ are enabling a safer, faster, and more reliable packaging tool chain. These tools can be difficult for people to use if first they must be told how to go out and install the package manager. 4. One of Pythons biggest strengths is in the huge ecosystem of libraries and projects that have been built on top of it, most of which are distributed through PyPI. However in order to benefit from this wide ecosystem meaningfully requires end users, some of which are going to be new, to make a decision on which package manager they should get, how to get it, and then finally actually installing it first. Furthermore, alternative implementations of Python are recognizing the benefits of PEP 453 and both PyPy and Jython have plans to backport ensurepip to their 2.7 runtimes. Automatic Invocation ==================== PEP 453 has ``ensurepip`` automatically invoked by default in the ``Makefile`` and the Windows and OSX Installers. This allowed it to ensure that, by default, all users would get Python with pip already installed. This PEP however believes that while this is fine for the Python 2.7 Windows and Mac OS X installers it is *not* ok for the Python 2.7 ``Makefile`` in general. The primary consumers of the ``Makefile`` are downstream package managers which distribute Python themselves. These downstream distributors typically do not want pip to be installed via ``ensurepip`` and would prefer that end users install it with their own package manager. Not invoking ``ensurepip`` automatically from the ``Makefile`` would allow these distributors to simply ignore the fact that ``ensurepip`` has been backported and still not end up with pip installed via it. The primary consumers of the OSX and Windows installers are end users who are attempting to install Python on their own machine. There is not a package manager available where these users can install pip into their Python through a more supported mechanism. For this reason it is the belief of this PEP that installing by default on OSX and Windows is the best course of action. Documentation ============= As part of this PEP, the updated packaging distribution and installation guides for Python 3.4 would be backported to Python 2.7. Disabling ensurepip by Downstream Distributors ============================================== Due to its use in the ``venv`` module, downstream distributors cannot disable the ``ensurepip`` module in Python 3.4. However since Python 2.7 has no such module it is explicitly allowed for downstream distributors to patch the ``ensurepip`` module to prevent it from installing anything. If a downstream distributor wishes to disable ``ensurepip`` completely in Python 2.7, they should still at least provide the module and allow `python -m ensurepip` style invocation. However it should raise errors or otherwise exit with a non-zero exit code and print out an error on stderr directing users to what they can/should use instead of ``ensurepip``. References ========== .. [#six] `six.py <https://pypi.python.org/pypi/six>`__ .. [#modernize] `modernize <https://pypi.python.org/pypi/modernize>`__ .. [#future] `python-future <https://pypi.python.org/pypi/future>`__ .. [#setuptools] `setuptools <https://pypi.python.org/pypi/setuptools>`__ .. [#wheel] `Wheel <https://pypi.python.org/pypi/wheel>`__ .. [#twine] `twine <https://pypi.python.org/pypi/twine>`__ Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:
On 8/31/2014 6:00 PM, Nick Coghlan wrote:
Earlier versions of PEP 453 proposed bootstrapping pip into a Python 2.7 maintenance release in addition to including it with Python 3.4.
That part of the proposal proved to be controversial, so we dropped it from the original PEP in order to focus on meeting the Python 3.4 specific release deadlines. This also had the benefit of working out the kinks in the bootstrapping processing as part of the Python 3.4 release cycle.
However, we still think we should start providing pip by default to Python 2.7 users as well, at least as part of the Windows and Mac OS X installers.
Having used pip for 3.4 on two windows computers, the only reason I see to *not* install it with Python would be 'push' people to 3.4+ by keeping 2.7 harder to use.
One notable difference from PEP 453 is that because there is no venv module in 2.7, and hence no integration between venv and ensurepip, we can give redistributors the option of just disabling ensurepip entirely and redirecting users to platform specific installation tools.
This also suggests that ensurepip does not have to be left on a Win/OS install either, after installation is done. Perhaps after ensuring that pip is installed, ensurepip could remove itself. This would remove any objection that its presence in /lib constitutes a new feature (which it is, even if not very useful). It would also keep /lib uniform across platforms. -- Terry Jan Reedy
On Mon, 1 Sep 2014 08:00:14 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
That part of the proposal proved to be controversial, so we dropped it from the original PEP in order to focus on meeting the Python 3.4 specific release deadlines. This also had the benefit of working out the kinks in the bootstrapping processing as part of the Python 3.4 release cycle.
However, we still think we should start providing pip by default to Python 2.7 users as well, at least as part of the Windows and Mac OS X installers.
I don't agree with this. pip is simply not part of the 2.7 feature set. If you add pip to a bugfix version, then you have bugfix versions which are more featureful than others, which makes things more complicated to explain. Regards Antoine.
On Sun, Aug 31, 2014, at 16:17, Antoine Pitrou wrote:
On Mon, 1 Sep 2014 08:00:14 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
That part of the proposal proved to be controversial, so we dropped it from the original PEP in order to focus on meeting the Python 3.4 specific release deadlines. This also had the benefit of working out the kinks in the bootstrapping processing as part of the Python 3.4 release cycle.
However, we still think we should start providing pip by default to Python 2.7 users as well, at least as part of the Windows and Mac OS X installers.
I don't agree with this. pip is simply not part of the 2.7 feature set. If you add pip to a bugfix version, then you have bugfix versions which are more featureful than others, which makes things more complicated to explain.
2.7.x has been and will be alive for so long that will already have to explain that sort thing; i.e. PEP 466 and why different bugfix releases support different versions of dependency libraries.
On 1 Sep 2014 09:23, "Benjamin Peterson" <benjamin@python.org> wrote:
On Sun, Aug 31, 2014, at 16:17, Antoine Pitrou wrote:
On Mon, 1 Sep 2014 08:00:14 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
That part of the proposal proved to be controversial, so we dropped
it from
the original PEP in order to focus on meeting the Python 3.4 specific release deadlines. This also had the benefit of working out the kinks in the bootstrapping processing as part of the Python 3.4 release cycle.
However, we still think we should start providing pip by default to Python 2.7 users as well, at least as part of the Windows and Mac OS X installers.
I don't agree with this. pip is simply not part of the 2.7 feature set. If you add pip to a bugfix version, then you have bugfix versions which are more featureful than others, which makes things more complicated to explain.
2.7.x has been and will be alive for so long that will already have to explain that sort thing; i.e. PEP 466 and why different bugfix releases support different versions of dependency libraries.
Exactly. LTS is genuinely different from stopping maintenance after the next feature release - it requires considering the "stability risk" and "user experience improvement" questions separately. In this case, the problem is that the Python 2 platform *is* still evolving, but the centre of that evolution has moved to PyPI. For "standard library only" users, Python 2 stopped evolving back in 2010. For PyPI users, by contrast, it's still evolving at a rapid pace. For our Python 3 transition story to be coherent, we need to ensure tools like six, modernize and future are readily available, while still remaining free to evolve independently of the standard library. That means providing a package management utility as easily and as painlessly as possible. Embracing pip upstream for Python 2 as well as Python 3 also sends a powerful signal to redistributors that says "your users are going to need this" and makes them do additional work to *avoid* providing it. Some of them *will* choose that path, but that becomes a matter for discussion between them and their user base, rather than a problem we need to worry about upstream. Cheers, Nick.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
In article <CADiSq7e4zachW4b_NmwEwPwtxG6DavNdC-ZwHAOAJkQ4sa3Qtg@mail.gmail.com>, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 1 Sep 2014 09:23, "Benjamin Peterson" <benjamin@python.org> wrote:
On Sun, Aug 31, 2014, at 16:17, Antoine Pitrou wrote:
On Mon, 1 Sep 2014 08:00:14 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
That part of the proposal proved to be controversial, so we dropped
it from
the original PEP in order to focus on meeting the Python 3.4 specific release deadlines. This also had the benefit of working out the kinks in the bootstrapping processing as part of the Python 3.4 release cycle.
However, we still think we should start providing pip by default to Python 2.7 users as well, at least as part of the Windows and Mac OS X installers.
A much bigger than average +1
I don't agree with this. pip is simply not part of the 2.7 feature set. If you add pip to a bugfix version, then you have bugfix versions which are more featureful than others, which makes things more complicated to explain. 2.7.x has been and will be alive for so long that will already have to explain that sort thing; i.e. PEP 466 and why different bugfix releases support different versions of dependency libraries.
And that is a minor complication compared with the confusion and difficulty of trying to explain to users (stuck with 2.7 for the time being) of how to install third-party packages on each platform (especially Windows) versus the simplicity of the 3.4.x story, thanks to ensurepip. Having pip available as a documented, batteries-included tool for all current releases would be a huge usability improvement.
Exactly. LTS is genuinely different from stopping maintenance after the next feature release - it requires considering the "stability risk" and "user experience improvement" questions separately.
In this case, the problem is that the Python 2 platform *is* still evolving, but the centre of that evolution has moved to PyPI. For "standard library only" users, Python 2 stopped evolving back in 2010. For PyPI users, by contrast, it's still evolving at a rapid pace.
For our Python 3 transition story to be coherent, we need to ensure tools like six, modernize and future are readily available, while still remaining free to evolve independently of the standard library. That means providing a package management utility as easily and as painlessly as possible.
Embracing pip upstream for Python 2 as well as Python 3 also sends a powerful signal to redistributors that says "your users are going to need this" and makes them do additional work to *avoid* providing it. Some of them *will* choose that path, but that becomes a matter for discussion between them and their user base, rather than a problem we need to worry about upstream.
FTR, I'm willing to backport the pieces I did for 3.4 and I could do the ensurepip backport, as well. I'll leave the Windows installer changes for someone else, though. -- Ned Deily, nad@acm.org
On Sep 1, 2014, at 2:22 AM, Ned Deily <nad@acm.org> wrote:
In article <CADiSq7e4zachW4b_NmwEwPwtxG6DavNdC-ZwHAOAJkQ4sa3Qtg@mail.gmail.com <mailto:CADiSq7e4zachW4b_NmwEwPwtxG6DavNdC-ZwHAOAJkQ4sa3Qtg@mail.gmail.com>>, Nick Coghlan <ncoghlan@gmail.com <mailto:ncoghlan@gmail.com>> wrote:
On 1 Sep 2014 09:23, "Benjamin Peterson" <benjamin@python.org> wrote:
On Sun, Aug 31, 2014, at 16:17, Antoine Pitrou wrote:
On Mon, 1 Sep 2014 08:00:14 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
That part of the proposal proved to be controversial, so we dropped
it from
the original PEP in order to focus on meeting the Python 3.4 specific release deadlines. This also had the benefit of working out the kinks in the bootstrapping processing as part of the Python 3.4 release cycle.
However, we still think we should start providing pip by default to Python 2.7 users as well, at least as part of the Windows and Mac OS X installers.
A much bigger than average +1
I don't agree with this. pip is simply not part of the 2.7 feature set. If you add pip to a bugfix version, then you have bugfix versions which are more featureful than others, which makes things more complicated to explain. 2.7.x has been and will be alive for so long that will already have to explain that sort thing; i.e. PEP 466 and why different bugfix releases support different versions of dependency libraries.
And that is a minor complication compared with the confusion and difficulty of trying to explain to users (stuck with 2.7 for the time being) of how to install third-party packages on each platform (especially Windows) versus the simplicity of the 3.4.x story, thanks to ensurepip. Having pip available as a documented, batteries-included tool for all current releases would be a huge usability improvement.
Yes this is a major driver. I mean I think I probably have an above average knowledge of how to bootstrap pip, and if you dump me on a Windows box I struggle to actually do it the first time around without stumbling around and doing things in the wrong order and the like. (Getting a compiler toolchain is worse, but yay for Wheels).
Exactly. LTS is genuinely different from stopping maintenance after the next feature release - it requires considering the "stability risk" and "user experience improvement" questions separately.
In this case, the problem is that the Python 2 platform *is* still evolving, but the centre of that evolution has moved to PyPI. For "standard library only" users, Python 2 stopped evolving back in 2010. For PyPI users, by contrast, it's still evolving at a rapid pace.
For our Python 3 transition story to be coherent, we need to ensure tools like six, modernize and future are readily available, while still remaining free to evolve independently of the standard library. That means providing a package management utility as easily and as painlessly as possible.
Embracing pip upstream for Python 2 as well as Python 3 also sends a powerful signal to redistributors that says "your users are going to need this" and makes them do additional work to *avoid* providing it. Some of them *will* choose that path, but that becomes a matter for discussion between them and their user base, rather than a problem we need to worry about upstream.
FTR, I'm willing to backport the pieces I did for 3.4 and I could do the ensurepip backport, as well. I'll leave the Windows installer changes for someone else, though.
Awesome, I’m of course willing to back port ensure pip itself as well. Truthfully it shouldn’t be a very difficult backport. It’s only ~200 SLOC or so and the only real things would be removing a Python3ism here or there. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On 1 Sep 2014 17:31, "Donald Stufft" <donald@stufft.io> wrote:
On Sep 1, 2014, at 2:22 AM, Ned Deily <nad@acm.org> wrote:
And that is a minor complication compared with the confusion and difficulty of trying to explain to users (stuck with 2.7 for the time being) of how to install third-party packages on each platform (especially Windows) versus the simplicity of the 3.4.x story, thanks to ensurepip. Having pip available as a documented, batteries-included tool for all current releases would be a huge usability improvement.
Yes this is a major driver. I mean I think I probably have an above
average
knowledge of how to bootstrap pip, and if you dump me on a Windows box I struggle to actually do it the first time around without stumbling around and doing things in the wrong order and the like. (Getting a compiler toolchain is worse, but yay for Wheels).
FTR, I'm willing to backport the pieces I did for 3.4 and I could do the ensurepip backport, as well. I'll leave the Windows installer changes for someone else, though.
Awesome, I’m of course willing to back port ensure pip itself as well. Truthfully it shouldn’t be a very difficult backport. It’s only ~200 SLOC or so and
Yeah. I've mentioned it before, but I think it bears repeating that trying to install pip on Windows with both Python 2 & 3 installed was one of the key things that convinced me to write PEP 453 in the first place. The default settings in both Internet and Windows explorer make it tricky regardless, but parallel installs make it even worse. the only
real things would be removing a Python3ism here or there.
Backporting meaningful tests will actually be the annoying part: the current unit tests use unittest.mock, while the current functional tests use pyvenv :) Both of those can be dealt with, but the tests will be a bit of an ugly hack by comparison with their Py3 counterparts :) Cheers, Nick.
On Sun, Aug 31, 2014 at 3:00 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
However, we still think we should start providing pip by default to Python 2.7 users as well, at least as part of the Windows and Mac OS X installers.
serious +1 here. Just last night I was writing up notes for an intro to python class on getting started. (py2, :-( ) The "go download get-pip.py and run it with python" step(s) is a serious hiccup. Better than it's been for years, but sure would be nice to just "install python" then "use pip to install packages" As for the feature freeze of py2.7 -- this is really an addition to teh installers, not to python itself (or it can be seen that way, anyway) Thanks Ned for offering to back port for OS-X ! -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On 1 September 2014 08:00, Nick Coghlan <ncoghlan@gmail.com> wrote:
Earlier versions of PEP 453 proposed bootstrapping pip into a Python 2.7 maintenance release in addition to including it with Python 3.4.
That part of the proposal proved to be controversial, so we dropped it from the original PEP in order to focus on meeting the Python 3.4 specific release deadlines. This also had the benefit of working out the kinks in the bootstrapping processing as part of the Python 3.4 release cycle.
However, we still think we should start providing pip by default to Python 2.7 users as well, at least as part of the Windows and Mac OS X installers.
I suggested off-list that, as Python 2.7 release manager, Benjamin would be a good candidate for BDFL-Delegate for this proposal, and both Guido and Benjamin agreed. The PEP has been updated accordingly: http://hg.python.org/peps/rev/909954e4ba43 (I also fixed a copy-and-paste error in the PEP status!) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Wed, Sep 3, 2014, at 17:03, Nick Coghlan wrote:
On 1 September 2014 08:00, Nick Coghlan <ncoghlan@gmail.com> wrote:
Earlier versions of PEP 453 proposed bootstrapping pip into a Python 2.7 maintenance release in addition to including it with Python 3.4.
That part of the proposal proved to be controversial, so we dropped it from the original PEP in order to focus on meeting the Python 3.4 specific release deadlines. This also had the benefit of working out the kinks in the bootstrapping processing as part of the Python 3.4 release cycle.
However, we still think we should start providing pip by default to Python 2.7 users as well, at least as part of the Windows and Mac OS X installers.
I suggested off-list that, as Python 2.7 release manager, Benjamin would be a good candidate for BDFL-Delegate for this proposal, and both Guido and Benjamin agreed. The PEP has been updated accordingly: http://hg.python.org/peps/rev/909954e4ba43 (I also fixed a copy-and-paste error in the PEP status!)
... and accepted. Let's make our users' lives' better!.
On 4 September 2014 12:21, Benjamin Peterson <benjamin@python.org> wrote:
On Wed, Sep 3, 2014, at 17:03, Nick Coghlan wrote:
On 1 September 2014 08:00, Nick Coghlan <ncoghlan@gmail.com> wrote:
Earlier versions of PEP 453 proposed bootstrapping pip into a Python 2.7 maintenance release in addition to including it with Python 3.4.
That part of the proposal proved to be controversial, so we dropped it from the original PEP in order to focus on meeting the Python 3.4 specific release deadlines. This also had the benefit of working out the kinks in the bootstrapping processing as part of the Python 3.4 release cycle.
However, we still think we should start providing pip by default to Python 2.7 users as well, at least as part of the Windows and Mac OS X installers.
I suggested off-list that, as Python 2.7 release manager, Benjamin would be a good candidate for BDFL-Delegate for this proposal, and both Guido and Benjamin agreed. The PEP has been updated accordingly: http://hg.python.org/peps/rev/909954e4ba43 (I also fixed a copy-and-paste error in the PEP status!)
... and accepted. Let's make our users' lives' better!.
Huzzah! Over to you, Ned & Donald. We can give Steve the good news for the Windows side of things as well :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (7)
-
Antoine Pitrou -
Benjamin Peterson -
Chris Barker -
Donald Stufft -
Ned Deily -
Nick Coghlan -
Terry Reedy