[Python-checkins] peps: Add PEP 453: explicit pip bootstrapping

nick.coghlan python-checkins at python.org
Fri Aug 30 15:10:22 CEST 2013 

http://hg.python.org/peps/rev/e726ddffb7cf
changeset:   5084:e726ddffb7cf
user:        Nick Coghlan <ncoghlan at gmail.com>
date:        Fri Aug 30 23:10:06 2013 +1000
summary:
  Add PEP 453: explicit pip bootstrapping

files:
  pep-0453.txt |  323 +++++++++++++++++++++++++++++++++++++++
  1 files changed, 323 insertions(+), 0 deletions(-)


diff --git a/pep-0453.txt b/pep-0453.txt
new file mode 100644
--- /dev/null
+++ b/pep-0453.txt
@@ -0,0 +1,323 @@
+PEP: 453
+Title: Explicit bootstrapping of pip in Python installations
+Version: $Revision$
+Last-Modified: $Date$
+Author: Donald Stufft <donald at stufft.io>,
+        Nick Coghlan <ncoghlan at gmail.com>
+Status: Draft
+Type: Process
+Content-Type: text/x-rst
+Created: 10-Aug-2013
+Post-History: 30-Aug-2013
+
+
+Abstract
+========
+
+This PEP proposes the inclusion of a method for explicitly bootstrapping
+`pip`_ as the default package manager for Python. It also proposes that the
+distributions of Python available via Python.org will automatically run this
+explicit bootstrapping method and a recommendation to third party
+redistributors of Python to also provide pip by default (in a way reasonable
+for their distributions).
+
+This PEP does *not* proposes the inclusion of pip itslef in the standard
+library.
+
+
+Proposal
+========
+
+This PEP proposes the inclusion of a ``getpip`` bootstrapping module in
+Python 3.4, as well as in the upcoming maintenance releases of Python 2.7
+and Python 3.3.
+
+
+Rationale
+=========
+
+Installing a third party package into a freshly installed Python requires first
+installing the package manager. This requires users ahead of time to know what
+the package manager is, where to get them from, and how to install them. The
+effect of this is that these external projects are required to either blindly
+assume the user already has the package manager installed, needs to duplicate
+the instructions and tell their users how to install the package manager, or
+completely forgo the use of dependencies to ease installation concerns for
+their users.
+
+All of the available options have their own drawbacks.
+
+If a project simply assumes a user already has the tooling then they get a
+confusing error message when the installation command doesn't work. Some
+operating may ease this pain by providing a global hook that looks for commands
+that don't exist and suggest an OS package they can install to make the command
+work.
+
+If a project chooses to duplicate the installation instructions and tell their
+users how to install the package manager before telling them how to install
+their own project then whenever these instructions need updates they need
+updating by every project that has duplicated them. This will inevitably not
+happen in every case leaving many different instructions on how to install it
+many of them broken or less than optimal. These additional instructions might
+also confuse users who try to install the package manager a second time
+thinking that it's part of the instructions of installing the project.
+
+The problem of stale instructions can be alleviated by referencing `pip's
+own bootstrapping instructions
+<http://www.pip-installer.org/en/latest/installing.html>`__, but the user
+experience involved still isn't good (especially on Windows, where
+downloading and running a Python script with the default OS configuration is
+significantly more painful than downloading and running a binary executable
+or installer).
+
+The projects that have decided to forgo dependencies all together are forced
+to either duplicate the efforts of other projects by inventing their own
+solutions to problems or are required to simply include the other projects
+in their own source trees. Both of these options present their own problems
+either in duplicating maintenance work across the ecosystem or potentially
+leaving users vulnerable to security issues because the included code or
+duplicated efforts are not automatically updated when upstream releases a new
+version.
+
+By providing the package manager by default it will be easier for users trying
+to install these third party packages as well as easier for the people
+distributing them as they no longer need to pick the lesser evil. This will
+become more important in the future as the Wheel_ package format does not have
+a built in "installer" in the form of ``setup.py`` so users wishing to install
+a Wheel package will need an installer even in the simple case.
+
+Reducing the burden of actually installing a third party package should also
+decrease the pressure to add every useful module to the standard library. This
+will allow additions to the standard library to focus more on why Python should
+have a particular tool out of the box instead of needing to use the difficulty
+in installing a package as justification for inclusion.
+
+
+Explicit Bootstrapping
+======================
+
+An additional module called ``getpip`` will be added to the standard library
+whose purpose is to install pip and any of its dependencies into the
+appropriate location (most commonly site-packages). It will expose a single
+callable named ``bootstrap()`` as well as offer direct execution via
+``python -m getpip``. Options for installing it such as index server,
+installation location (``--user``, ``--root``, etc) will also be available
+to enable different installation schemes.
+
+It is believed that users will want the most recent versions available to be
+installed so that they can take advantage of the new advances in packaging.
+Since any particular version of Python has a much longer staying power than
+a version of pip in order to satisfy a user's desire to have the most recent
+version the bootstrap will contact PyPI, find the latest version, download it,
+and then install it. This process is security sensitive, difficult to get
+right, and evolves along with the rest of packaging.
+
+Instead of attempting to maintain a "mini pip" for the sole purpose of
+installing pip the ``getpip`` module will, as an implementation detail, include
+a private copy of pip which will be used to discover and install pip from PyPI.
+It is important to stress that this private copy of pip is *only* an
+implementation detail and it should *not* be relied on or assumed to exist.
+
+Not all users will have network access to PyPI whenever they run the bootstrap.
+In order to ensure that these users will still be able to bootstrap pip the
+bootstrap will fallback to simply installing the included copy of pip.
+
+This presents a balance between giving users the latest version of pip, saving
+them from needing to immediately upgrade pip after bootstrapping it, and
+allowing the bootstrap to work offline in situations where users might already
+have packages downloaded that they wish to install.
+
+
+Updating the Bundled pip
+------------------------
+
+In order to keep up with evolutions in packaging as well as providing users
+who are using the offline installation method with as recent version as
+possible the ``getpip`` module should be updates to the latest versions of
+everything it bootstraps. During the preparation for any release of Python, a
+script, provided as part of this PEP, should be run to update the bundled
+packages to the latest versions.
+
+This means that maintenance releases of the CPython installers will include
+an updated version of the ``getpip`` bootstrap module.
+
+
+Pre-installation
+================
+
+During the installation of Python from Python.org ``python -m getpip`` should
+be executed. Leaving people using the Windows or OSX installers with a working
+copy of pip once the installation has completed. The exact method of this is
+left up to the maintainers of the installers however if the bootstrapping is
+optional it should be opt out rather than opt in.
+
+The Windows and OSX installers distributed by Python.org will automatically
+attempt to run ``python -m getpip`` by default however the ``make install``
+and ``make altinstall`` commands of the source distribution will not.
+
+Keeping the pip bootstrapping as a separate step for make based
+installations should minimise the changes CPython redistributors need to
+make to their build processes. Avoiding the layer of indirection through
+make for the getpip invocation also ensures those installing from a custom
+source build can easily force an offline installation of pip, install it
+from a private index server, or skip installing pip entirely.
+
+
+Python Virtual Environments
+===========================
+
+Python 3.3 included a standard library approach to virtual Python environments
+through the ``venv`` module. Since it's release it has become clear that very
+few users have been willing to use this feature in part due to the lack of
+an installer present by default inside of the virtual environment. They have
+instead opted to continue using the ``virtualenv`` package which *does* include
+pip installed by default.
+
+To make the ``venv`` more useful to users it will be modified to issue the
+pip bootstrap by default inside of the new environment while creating it. This
+will allow people the same convenience inside of the virtual environment as
+this PEP provides outside of it as well as bringing the ``venv`` module closer
+to feature parity with the external ``virtualenv`` package making it a more
+suitable replacement.
+
+
+Recommendations for Downstream Distributors
+===========================================
+
+A common source of Python installations are through downstream distributors
+such as the various Linux Distributions [#ubuntu]_ [#debian]_ [#fedora]_, OSX
+package managers [#homebrew]_, or python specific tools [#conda]_. In order to
+provide a consistent, user friendly experience to all users of Python
+regardless of how they attained Python this PEP recommends and asks that
+downstream distributors:
+
+* Ensure that whenever Python is installed pip is also installed.
+
+  * This may take the form of separate with dependencies on each either so that
+    installing the python package installs the pip package and installing the
+    pip package installs the Python package.
+
+* Do not remove the bundled copy of pip.
+
+  * This is required for offline installation of pip into a virtual environment.
+  * This is similar to the existing ``virtualenv`` package for which many
+    downstream distributors have already made exception to the common
+    "debundling" policy.
+  * This does mean that if ``pip`` needs to be updated due to a security
+    issue, so does the bundled version in the ``getpip`` bootstrap module
+
+* Migrate build systems to utilize `pip`_ and `Wheel`_ instead of directly
+  using ``setup.py``.
+
+  * This will ensure that downstream packages can utilize the new formats which
+    will not have a ``setup.py`` easier.
+
+* Ensure that all features of this PEP continue to work with any modifications
+  made.
+
+  * Online installation of the latest version of pip into a global or virtual
+    python environment using ``python -m getpip``.
+  * Offline installation of the bundled version of pip into a global or virtual
+    python environment using ``python -m getpip``.
+  * ``pip install --upgrade pip`` in a global installation should not affect
+    any already created virtual environments.
+  * ``pip install --upgrade pip`` in a virtual environment should not affect
+    the global installation.
+
+
+Policies & Governance
+=====================
+
+The maintainers of the bundled software and the CPython core team will work
+together in order to address the needs of both. The bundled software will still
+remain external to CPython and this PEP does not include CPython subsuming the
+responsibilities or decisions of the bundled software. This PEP aims to
+decrease the burden on end users wanting to use third party packages and the
+decisions inside it are pragmatic ones that represent the trust that the
+Python community has placed in the authors and maintainers of the bundled
+software.
+
+
+Backwards Compatibility
+-----------------------
+
+The public API of the ``getpip`` module itself will fall under the typical
+backwards compatibility policy of Python for its standard library. The
+externally developed software that this PEP bundles does not.
+
+
+Security Releases
+-----------------
+
+Any security update that affects the ``getpip`` module will be shared prior to
+release with the PSRT. The PSRT will then decide if the issue inside warrants
+a security release of Python.
+
+
+Appendix: Rejected Proposals
+============================
+
+
+Implicit Bootstrap
+------------------
+
+`PEP439`_, the predecessor for this PEP, proposes it's own solution. Its
+solution involves shipping a fake ``pip`` command that when executed would
+implicitly bootstrap and install pip if it does not already exist. This has
+been rejected because it is too "magical". It hides from the end user when
+exactly the pip command will be installed or that it is being installed at all.
+It also does not provide any recommendations or considerations towards
+downstream packagers who wish to manage the globally installed pip through the
+mechanisms typical for their system.
+
+
+Including pip In the Standard Library
+-------------------------------------
+
+Similar to this PEP is the proposal of just including pip in the standard
+library. This would ensure that Python always includes pip and fixes all of the
+end user facing problems with not having pip present by default. This has been
+rejected because we've learned through the inclusion and history of
+``distutils`` in the standard library that losing the ability to update the
+packaging tools independently can leave the tooling in a state of constant
+limbo. Making it unable to ever reasonably evolve in a timeframe that actually
+affects users as any new features will not be available to the general
+population for *years*.
+
+Allowing the packaging tools to progress separately from the Python release
+and adoption schedules allows the improvements to be used by *all* members
+of the Python community and not just those able to live on the bleeding edge
+of Python releases.
+
+
+.. _Wheel: http://www.python.org/dev/peps/pep-0427/
+.. _pip: http://www.pip-installer.org
+.. _setuptools: https://pypi.python.org/pypi/setuptools
+.. _PEP439: http://www.python.org/dev/peps/pep-0439/
+
+
+References
+==========
+
+.. [#ubuntu] `Ubuntu <http://www.ubuntu.com/>`
+.. [#debian] `Debian <http://www.debian.org>`
+.. [#fedora] `Fedora <https://fedoraproject.org/>`
+.. [#homebrew] `Homebrew  <http://brew.sh/>`
+.. [#conda] `Conda <http://www.continuum.io/blog/conda>`
+
+
+Copyright
+=========
+
+This document has been placed in the public domain.
+
+
+

+..
+   Local Variables:
+   mode: indented-text
+   indent-tabs-mode: nil
+   sentence-end-double-space: t
+   fill-column: 70
+   coding: utf-8
+   End:

-- 
Repository URL: http://hg.python.org/peps

More information about the Python-checkins mailing list