<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi!<br>
We have written a draft PEP entitled "Distributing a Subset of the
Standard Library" that aims to standardize and improve how Python
handles omissions from its standard library. This is relevant both
to Python itself as well as to Linux and other distributions that
are packaging it, as they already separate out parts of the standard
library into optionally installable packages.<br>
<br>
Ideas leading up to this PEP were discussed on the python-dev
mailing list:<br>
<a class="moz-txt-link-freetext" href="https://mail.python.org/pipermail/python-dev/2016-July/145534.html">https://mail.python.org/pipermail/python-dev/2016-July/145534.html</a><br>
<br>
Rendered PEP: <a class="moz-txt-link-freetext" href="https://fedora-python.github.io/pep-drafts/pep-A.html">https://fedora-python.github.io/pep-drafts/pep-A.html</a><br>
<br>
<hr size="2" width="100%"><br>
Abstract<br>
========<br>
<br>
Python is sometimes being distributed without its full standard
library.<br>
However, there is as of yet no standardized way of dealing with
importing a<br>
missing standard library module. This PEP proposes a mechanism for
identifying<br>
which standard library modules are missing and puts forth a method
of how<br>
attempts to import a missing standard library module should be
handled.<br>
<br>
<br>
Motivation<br>
==========<br>
<br>
There are several use cases for including only a subset of Python's
standard<br>
library. However, there is so far no formal specification of how to
properly<br>
implement distribution of a subset of the standard library. Namely,
how to<br>
safely handle attempts to import a missing *stdlib* module, and
display an<br>
informative error message.<br>
<br>
<br>
CPython<br>
-------<br>
<br>
When one of Python standard library modules (such as ``_sqlite3``)
cannot be<br>
compiled during a Python build because of missing dependencies (e.g.
SQLite<br>
header files), the module is simply skipped.<br>
<br>
If you then install this compiled Python and use it to try to import
one of the<br>
missing modules, Python will go through the ``sys.path`` entries
looking for<br>
it. It won't find it among the *stdlib* modules and thus it will
continue onto<br>
``site-packages`` and fail with a ModuleNotFoundError_ if it doesn't
find it.<br>
<br>
.. _ModuleNotFoundError:<br>
<a class="moz-txt-link-freetext" href="https://docs.python.org/3.7/library/exceptions.html#ModuleNotFoundError">https://docs.python.org/3.7/library/exceptions.html#ModuleNotFoundError</a><br>
<br>
This can confuse users who may not understand why a cleanly built
Python is<br>
missing standard library modules.<br>
<br>
<br>
Linux and other distributions<br>
-----------------------------<br>
<br>
Many Linux and other distributions are already separating out parts
of the<br>
standard library to standalone packages. Among the most commonly
excluded<br>
modules are the ``tkinter`` module, since it draws in a dependency
on the<br>
graphical environment, and the ``test`` package, as it only serves
to test<br>
Python internally and is about as big as the rest of the standard
library put<br>
together.<br>
<br>
The methods of omission of these modules differ. For example,
Debian patches<br>
the file ``Lib/tkinter/__init__.py`` to envelop the line ``import
_tkinter`` in<br>
a *try-except* block and upon encountering an ``ImportError`` it
simply adds<br>
the following to the error message: ``please install the python3-tk
package``<br>
[#debian-patch]_. Fedora and other distributions simply don't
include the<br>
omitted modules, potentially leaving users baffled as to where to
find them.<br>
<br>
<br>
Specification<br>
=============<br>
<br>
When, for any reason, a standard library module is not to be
included with the<br>
rest, a file with its name and the extension ``.missing.py`` shall
be created<br>
and placed in the directory the module itself would have occupied.
This file<br>
can contain any Python code, however, it *should* raise a
ModuleNotFoundError_<br>
with a helpful error message.<br>
<br>
Currently, when Python tries to import a module ``XYZ``, the
``FileFinder``<br>
path hook goes through the entries in ``sys.path``, and in each
location looks<br>
for a file whose name is ``XYZ`` with one of the valid suffixes
(e.g. ``.so``,<br>
..., ``.py``, ..., ``.pyc``). The suffixes are tried in order. If
none of<br>
them are found, Python goes on to try the next directory in
``sys.path``.<br>
<br>
The ``.missing.py`` extension will be added to the end of the list,
and<br>
configured to be handled by ``SourceFileLoader``. Thus, if a module
is not<br>
found in its proper location, the ``XYZ.missing.py`` file is found
and<br>
executed, and further locations are not searched.<br>
<br>
The CPython build system will be modified to generate
``.missing.py`` files for<br>
optional modules that were not built.<br>
<br>
<br>
Rationale<br>
=========<br>
<br>
The mechanism of handling missing standard library modules through
the use of<br>
the ``.missing.py`` files was chosen due to its advantages both for
CPython<br>
itself and for Linux and other distributions that are packaging it.<br>
<br>
The missing pieces of the standard library can be subsequently
installed simply<br>
by putting the module files in their appropriate location. They
will then take<br>
precedence over the corresponding ``.missing.py`` files. This makes<br>
installation simple for Linux package managers.<br>
<br>
This mechanism also solves the minor issue of importing a module
from<br>
``site-packages`` with the same name as a missing standard library
module.<br>
Now, Python will import the ``.missing.py`` file and won't ever look
for a<br>
*stdlib* module in ``site-packages``.<br>
<br>
In addition, this method of handling missing *stdlib* modules can be<br>
implemented in a succinct, non-intrusive way in CPython, and thus
won't add to<br>
the complexity of the existing code base.<br>
<br>
The ``.missing.py`` file can be customized by the packager to
provide any<br>
desirable behaviour. While we strongly recommend that these files
only raise a<br>
ModuleNotFoundError_ with an appropriate message, there is no reason
to limit<br>
customization options.<br>
<br>
Ideas leading up to this PEP were discussed on the `python-dev
mailing list`_.<br>
<br>
.. _`python-dev mailing list`:<br>
<a class="moz-txt-link-freetext" href="https://mail.python.org/pipermail/python-dev/2016-July/145534.html">https://mail.python.org/pipermail/python-dev/2016-July/145534.html</a><br>
<br>
<br>
Backwards Compatibility<br>
=======================<br>
<br>
No problems with backwards compatibility are expected.
Distributions that are<br>
already patching Python modules to provide custom handling of
missing<br>
dependencies can continue to do so unhindered.<br>
<br>
<br>
Reference Implementation<br>
========================<br>
<br>
Reference implementation can be found on `GitHub`_ and is also
accessible in<br>
the form of a `patch`_.<br>
<br>
.. _`GitHub`: <a class="moz-txt-link-freetext" href="https://github.com/torsava/cpython/pull/1">https://github.com/torsava/cpython/pull/1</a><br>
.. _`patch`: <a class="moz-txt-link-freetext" href="https://github.com/torsava/cpython/pull/1.patch">https://github.com/torsava/cpython/pull/1.patch</a><br>
<br>
<br>
References<br>
==========<br>
<br>
.. [#debian-patch]<br>
<a class="moz-txt-link-freetext" href="http://bazaar.launchpad.net/~doko/python/pkg3.5-debian/view/head:/patches/tkinter-import.diff">http://bazaar.launchpad.net/~doko/python/pkg3.5-debian/view/head:/patches/tkinter-import.diff</a><br>
<br>
<br>
Copyright<br>
=========<br>
<br>
This document has been placed in the public domain.<br>
<br>
<hr size="2" width="100%">Regards,<br>
Tomas Orsava
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</body>
</html>