[Python-checkins] peps: Assorted PEP 453 (pip bootstrapping) updates
nick.coghlan
python-checkins at python.org
Sun Sep 15 08:20:30 CEST 2013
http://hg.python.org/peps/rev/0522bee830a6
changeset: 5116:0522bee830a6
user: Nick Coghlan <ncoghlan at gmail.com>
date: Sun Sep 15 16:20:11 2013 +1000
summary:
Assorted PEP 453 (pip bootstrapping) updates
- revert to being explicitly 3.4 only (with rationale)
- open question re uninstallation
- open question re script execution on Windows
- explicit cover handling of the setuptools dependency
- spell out the proposed CLI options for getpip
- note that venv will also support --no-download
- explicitly note that pip may gain new features in CPython
maintenance releases and continues using its own release
cycle
- explicitly note that bundling pip doesn't preclude the
use of alternate installers, but instead better *enables*
them by making them easier to bootstrap
- note we should update to new pip version when they're
released, so the release process just checks they're up to
date.
files:
pep-0453.txt | 201 +++++++++++++++++++++++++++++++-------
1 files changed, 160 insertions(+), 41 deletions(-)
diff --git a/pep-0453.txt b/pep-0453.txt
--- a/pep-0453.txt
+++ b/pep-0453.txt
@@ -21,29 +21,31 @@
redistributors of Python to also provide pip by default (in a way reasonable
for their distributions).
-This PEP does *not* propose the inclusion of pip itself in the standard
-library.
-
Proposal
========
This PEP proposes the inclusion of a ``getpip`` bootstrapping module in
-Python 3.4, as well as in the upcoming maintenance releases of Python 2.7
-and Python 3.3.
+Python 3.4.
+
+This PEP does *not* propose making pip (or any dependencies) part of the
+standard library. Instead, pip will be a bundled application provided
+along with CPython for the convenience of Python users, but subject to
+its own development life cycle and able to be upgraded independently of
+the core interpreter and standard library.
Rationale
=========
-Installing a third party package into a freshly installed Python requires first
-installing the package manager. This requires users ahead of time to know what
-the package manager is, where to get them from, and how to install them. The
-effect of this is that these external projects are required to either blindly
-assume the user already has the package manager installed, needs to duplicate
-the instructions and tell their users how to install the package manager, or
-completely forgo the use of dependencies to ease installation concerns for
-their users.
+Installing a third party package into a freshly installed Python requires
+first installing the package manager. This requires users ahead of time to
+know what the package manager is, where to get them from, and how to install
+them. The effect of this is that these external projects are required to
+either blindly assume the user already has the package manager installed,
+needs to duplicate the instructions and tell their users how to install
+the package manager, or completely forgo the use of dependencies to ease
+installation concerns for their users.
All of the available options have their own drawbacks.
@@ -68,9 +70,11 @@
experience involved still isn't good (especially on Windows, where
downloading and running a Python script with the default OS configuration is
significantly more painful than downloading and running a binary executable
-or installer).
+or installer). The situation becomes even more complicated when multiple
+Python versions are involved (for example, parallel installations of Python
+2 and Python 3).
-The projects that have decided to forgo dependencies all together are forced
+The projects that have decided to forgo dependencies altogether are forced
to either duplicate the efforts of other projects by inventing their own
solutions to problems or are required to simply include the other projects
in their own source trees. Both of these options present their own problems
@@ -92,6 +96,14 @@
have a particular tool out of the box instead of needing to use the difficulty
in installing a package as justification for inclusion.
+Providing a standard installation system also helps with bootstrapping
+alternate build and installer systems, such as ``setuptools``,
+``zc.buildout`` and the ``hashdist``/``conda`` combination that is aimed
+specifically at the scientific community. So long as
+``pip install <tool>`` works, then a standard Python-specific installer
+provides a reasonably secure, cross platform mechanism to get access to
+these utilities.
+
Explicit Bootstrapping
======================
@@ -113,30 +125,79 @@
right, and evolves along with the rest of packaging.
Instead of attempting to maintain a "mini pip" for the sole purpose of
-installing pip the ``getpip`` module will, as an implementation detail, include
-a private copy of pip which will be used to discover and install pip from PyPI.
-It is important to stress that this private copy of pip is *only* an
-implementation detail and it should *not* be relied on or assumed to exist.
+installing pip the ``getpip`` module will, as an implementation detail,
+include a private copy of pip and its dependencies which will be used to
+discover and install pip from PyPI. It is important to stress that this
+private copy of pip is *only* an implementation detail and it should *not*
+be relied on or assumed to exist.
-Not all users will have network access to PyPI whenever they run the bootstrap.
-In order to ensure that these users will still be able to bootstrap pip the
-bootstrap will fallback to simply installing the included copy of pip.
+Not all users will have network access to PyPI whenever they run the
+bootstrap. In order to ensure that these users will still be able to
+bootstrap pip the bootstrap will fallback to simply installing the included
+copy of pip. The pip ``--no-download`` command line option will be supported
+to force installation of the bundled version, without even attempting to
+contact PyPI.
-This presents a balance between giving users the latest version of pip, saving
-them from needing to immediately upgrade pip after bootstrapping it, and
-allowing the bootstrap to work offline in situations where users might already
-have packages downloaded that they wish to install.
+This presents a balance between giving users the latest version of pip,
+saving them from needing to immediately upgrade pip after bootstrapping it,
+and allowing the bootstrap to work offline in situations where users might
+already have packages downloaded that they wish to install.
-Updating the Bundled pip
+Proposed CLI
+------------
+
+The proposed CLI is based on a subset of the existing ``pip install``
+options::
+
+ Usage:
+ python -m getpip [options]
+
+ Download Options:
+ --no-download Install the bundled version, don't attempt to download
+ -i, --index-url <url> Base URL of Python Package Index (default https://pypi.python.org/simple/).
+ --proxy <proxy> Specify a proxy in the form [user:passwd@]proxy.server:port.
+ --timeout <sec> Set the socket timeout (default 15 seconds).
+ --cert <path> Path to alternate CA bundle.
+
+ Installation Options:
+ -U, --upgrade Upgrade pip and dependencies, even if already installed
+ --user Install using the user scheme.
+ --root <dir> Install everything relative to this alternate root directory.
+
+Additional options (such as verbosity and logging options) may also
+be supported.
+
+
+Automatic installation of setuptools
+------------------------------------
+
+``pip`` currently depends on ``setuptools`` to handle metadata generation
+during the build process, along with some other features. While work is
+ongoing to reduce or eliminate this dependency, it is not clear if that
+work will be complete for pip 1.5 (which is the version likely to be
+bundled with Python 3.4.0).
+
+This PEP proposes that, if pip still requires it, ``setuptools`` will be
+bundled along with pip itself, and thus installed when running
+``python -m getpip``.
+
+However, this behaviour will be officially declared an implementation
+detail. Other projects which explicitly require setuptools should still
+provide an appropriate dependency declaration, rather than assuming
+``setuptools`` will always be installed alongside ``pip``.
+
+
+Updating the bundled pip
------------------------
In order to keep up with evolutions in packaging as well as providing users
who are using the offline installation method with as recent version as
-possible the ``getpip`` module should be updates to the latest versions of
-everything it bootstraps. During the preparation for any release of Python, a
-script, provided as part of this PEP, should be run to update the bundled
-packages to the latest versions.
+possible the ``getpip`` module should be updated to the latest versions of
+everything it bootstraps. After each new pip release, and again during the
+preparation for any release of Python, a script, provided as part of this
+PEP, should be run to ensure the bundled packages have been updated to the
+latest versions.
This means that maintenance releases of the CPython installers will include
an updated version of the ``getpip`` bootstrap module.
@@ -153,16 +214,54 @@
The Windows and OSX installers distributed by Python.org will automatically
attempt to run ``python -m getpip`` by default however the ``make install``
-and ``make altinstall`` commands of the source distribution will not.
+and ``make altinstall`` commands of the source distribution will not. Note
+that ``getpip`` itself will still be installed normally (as it is a regular
+part of the standard library), only
Keeping the pip bootstrapping as a separate step for make based
installations should minimize the changes CPython redistributors need to
make to their build processes. Avoiding the layer of indirection through
-make for the getpip invocation also ensures those installing from a custom
+``make`` for the getpip invocation also ensures those installing from a custom
source build can easily force an offline installation of pip, install it
from a private index server, or skip installing pip entirely.
+Open Question: Uninstallation
+=============================
+
+No changes are currently proposed to the uninstallation process. The
+bootstrapped pip will be installed the same way as any other pip
+installed packages, and will be handled in the same way as any other
+post-install additions to the Python environment.
+
+At least on Windows, that means the bootstrapped files will be
+left behind after uninstallation, since those files won't be associated
+with the Python MSI installer.
+
+.. note::
+
+ Perhaps the installer needs to be updated to clobber everything in
+ site-packages and the Scripts directory, but I would prefer not to make
+ this PEP conditional on that change.
+
+
+Open Question: Script Execution on Windows
+==========================================
+
+While the Windows installer was updated in Python 3.3 to make ``python``
+available on the PATH, no such change was made to include the scripts
+directory. This PEP proposes that this be changed to also add the scripts
+directory.
+
+Without this change, the most reliable way to invoke pip on Windows (without
+tinkering with paths) is actually be ``py -m pip`` (or ``py -3 -m pip``
+if both Python 2 and 3 are installed) rather than simply calling ``pip``.
+
+Adding the scripts directory to the system PATH would mean that ``pip``
+works reliably in the "only one Python installation" case, with
+``py -m pip`` needed only for the parallel installation case.
+
+
Python Virtual Environments
===========================
@@ -178,9 +277,11 @@
will allow people the same convenience inside of the virtual environment as
this PEP provides outside of it as well as bringing the ``venv`` module closer
to feature parity with the external ``virtualenv`` package making it a more
-suitable replacement. In the case that a user does not wish to have pip
+suitable replacement. To handles cases where a user does not wish to have pip
bootstrapped into their virtual environment a ``--without-pip`` option will be
-added.
+added. The ``--no-download`` option will also be supported, to force the
+use of the bundled ``pip`` rather than retrieving the latest version from
+PyPI.
Recommendations for Downstream Distributors
@@ -195,9 +296,9 @@
* Ensure that whenever Python is installed pip is also installed.
- * This may take the form of separate with dependencies on each either so that
- installing the python package installs the pip package and installing the
- pip package installs the Python package.
+ * This may take the form of separate packages with dependencies on each
+ other so that installing the Python package installs the pip package
+ and installing the pip package installs the Python package.
* Do not remove the bundled copy of pip.
@@ -211,8 +312,8 @@
* Migrate build systems to utilize `pip`_ and `Wheel`_ instead of directly
using ``setup.py``.
- * This will ensure that downstream packages can utilize the new formats which
- will not have a ``setup.py`` easier.
+ * This will ensure that downstream packages can more easily utilize the
+ new metadata formats which may not have a ``setup.py``.
* Ensure that all features of this PEP continue to work with any modifications
made.
@@ -247,6 +348,10 @@
backwards compatibility policy of Python for its standard library. The
externally developed software that this PEP bundles does not.
+Most importantly, this means that the bundled version of pip may gain new
+features in CPython maintenance releases, and pip continues to operate on
+its own 6 month release cycle rather than CPython's 18-24 month cycle.
+
Security Releases
-----------------
@@ -260,10 +365,24 @@
============================
+Bundling the installer in Python 2.7 and 3.3 Maintenance Releases
+-----------------------------------------------------------------
+
+Unlike earlier Python versions, Python 3.4 provides access to the system
+certificate store on Windows systems. This allows ``getpip`` to create a
+verified connection to PyPI without needing to include a custom certificate
+bundle with CPython.
+
+Rather than trying to come up with a secure bootstrapping alternative for
+earlier Python versions, the existing manual bootstrapping mechanism (which
+relies on SSL verification in other tools like curl, wget and web browsers)
+will continue to be used.
+
+
Implicit Bootstrap
------------------
-`PEP439`_, the predecessor for this PEP, proposes it's own solution. Its
+`PEP439`_, the predecessor for this PEP, proposes its own solution. Its
solution involves shipping a fake ``pip`` command that when executed would
implicitly bootstrap and install pip if it does not already exist. This has
been rejected because it is too "magical". It hides from the end user when
--
Repository URL: http://hg.python.org/peps
More information about the Python-checkins
mailing list