[Python-Dev] PEP 453 (pip bootstrapping) ready for pronouncement?
nad at acm.org
Tue Sep 24 01:34:54 CEST 2013
In general, I think this is a very important usability feature and I
am in favor of the general approach. Good work, all! I do have some
comments, primarily about items that are not currently addressed.
> ``ensurepip`` itself (including the private copy of ``pip`` and its
> dependencies) will still be installed normally (as it is a regular
> part of the standard library), only the implicit installation of pip and
> its dependencies will be skipped.
The PEP should address the license implications of this since the PEP will
add the source of pip and setuptools to all new Python releases, both
source and binary installers releases. For Python itself, we go to some
trouble to ensure that all contributions are contributed under a suitable
license. What steps are needed, initially and on-going, to ensure that
these third-party items to be pulled in at each release are compatible
with the license of Python itself? Who will do this: release managers?
Are updates needed in the internal and external documentation for releases
(README, LICENSE)? I took a quick look at the current source releases for
both pip and setuptools. I noted a reference to the LGPL for pip's CA
certs bundle; I didn't see a license file for setuptools.
> This PEP proposes that, if pip still requires it as a dependency,
> ``ensurepip`` will include a private copy of ``setuptools`` (in addition
> to the private copy of ``ensurepip``). ``python -m ensurepip`` will then
> install the private copy in addition to installing ``pip`` itself.
> However, this behavior is officially considered an implementation
> detail. Other projects which explicitly require ``setuptools`` must still
> provide an appropriate dependency declaration, rather than assuming
> ``setuptools`` will always be installed alongside ``pip``.
> Once pip is able to run ``pip install --upgrade pip`` without needing
> ``setuptools`` installed first, then the private copy of ``setuptools``
> will be removed from ``ensurepip`` in subsequent CPython releases.
The PEP does not sufficiently address the issue of the setuptools-provided
easy_install command. I think it is important to do that for several
1. The PEP should make clear whether the easy_install scripts are or
are not installed, globally or in a venv, as a side effect of running
ensurepip, at least while pip has the implicit private dependency
on setuptools. By default, the scripts are installed so ensurepip
would need to take action to prevent this if that is not desired.
2. Despite the current strong preference for using pip as a command
line installer, the fact is that much third-party documentation still refers
to and recommends using easy_install for their projects. That is not
going to change overnight. The PEP should identify some steps needed to
facilitate the migration, like perhaps adding something to the "Installing
Python Modules" documentation explicitly deprecating easy_install use
and/or giving pip equivalents for the most common easy_install
scenarios (or linking to other documentation that does so).
3. A specific issue for OS X users is that, as part of OS X, Apple ships
versions of Python 2 and various third-party packages, including
setuptools (older, pre-reunification versions, at that) but not pip.
That means that OS X ships with easy_install and easy_install-2.n
commands in /usr/bin, including 2.7 in recent OS X releases. This has
been an ongoing source of confusion for OS X users who install a newer
version of Python. Depending on the distributor and/or source build
configure options, the new version of Python will be installed to a
different path, like /usr/local/bin. Which instance is used
is generally managed by direct PATH environment manipulations (or by
launcher programs similar to the Windows py launcher). If setuptools
is not installed using the new instance, users invoking the easy_install
command will by default end up with distributions being installed to a
system Python rather than to the new Python. This leads to the
unfortunately common scenario of:
sudo easy_install blah # --> installs to system Python 2.7
python -c 'import blah' # --> fails because using newer Python 2.7
This may very well be the most common current usability problem today
with python.org (and other third-party) Pythons on OS X.
With this PEP, we have an opportunity to fix this problem in one of
two ways. As long as pip requires setuptools and we do not inhibit
the installation of the easy_install scripts is to the same directory
as the python command itself, the new easy_install scripts will
shadow the Apple-supplied ones just as the python commands do. If it
is decided to not install the easy_install scripts now or in the
future (e.g. when pip no longer depends on setuptools),
ensurepip could install dummy easy_install scripts (if there
are none already installed there) that merely emit a message to the user
to encourage migration to pip and suggest, if needed, to use pip to install
setuptools to replace the placeholder easy_install. Either approach
would be a huge usability improvement over the situation today.
(The actual implementation details for either approach are slightly more
complicated for OS X framework builds like the python.org installers
as the actual scripts directory is a ``bin`` directory within the
framework itself with optional symlinks from ``/usr/local/bin`` to
the fw bin directory for specific commands. The installer will now need
to also install symlinks in ``/usr/local/bin`` for ``pip`` and, as
proposed above, for ``easy_install``.)
> A common source of Python installations are through downstream distributors
> such as the various Linux Distributions [#ubuntu]_ [#debian]_ [#fedora]_, OSX
> package managers [#homebrew]_, or Python-specific tools [#conda]_. In order
If you are going to call out Homebrew, you should include the other major
OS X package managers, MacPorts and Fink.
> .. [#homebrew] `Homebrew <http://brew.sh/>`
.. [#macports] `MacPorts <http://macports.org>`
.. [#fink] `Fink <http://finkproject.org>`
One final comment is that the PEP does not go into any detail on how it
will be implemented. As it stands, there is a fair amount of one-time
work, including implementing ensurepip, changes to the Windows installer,
changes to the OS X installer, documentation changes, all with testing and
backporting to 2.7.x and 3.3.x. Then there are the on-going process changes
for all future releases, impacting all release team members, which will need
to be documented in PEP 101. Do we have an understanding of who is do the
big pieces and by when? For the record, I am willing to do the extra
one-time and ongoing work for the OS X installers but it would be helpful
to know how we are going to get it all done in time for upcoming releases.
nad at acm.org
More information about the Python-Dev