[Python-ideas] PEP: Distributing a Subset of the Standard Library

[Steven D'Aprano]

On Tue, Nov 29, 2016 at 10:55:14AM -0500, Todd wrote:
> On Tue, Nov 29, 2016 at 4:13 AM, M.-A. Lemburg <mal at egenix.com> wrote:

> > Just as with .pth files, the possibility to hook arbitrary code
> > execution into the module search path will get abused for
> > all kinds of weird things, esp. if the whole sys.path is
> > scanned for the .missing.py module and not only the part
> > where the stdlib lives (as was suggested in the thread).
> >
> > So why not limit the PEP to just the intended use case ?

> I think a better question is why should we artificially limit the PEP?

Because YAGNI.

Overly complex, complicated systems which do more than is needed 
"because it might be useful one day" is an anti-pattern. The intended 
use-case is to allow Linux distributions to customize the error message 
on ImportError. From there, it is a small step to allow *other* people 
to do the same thing. But it is a BIG step to go from that to a solution 
that executes arbitrary code.

Before we take that big step, we ought to have a good reason.


> These is something that could be useful outside of the stdlib.

Sure. I don't think there is any proposal to prevent people outside of 
Linux package distributors from using this mechanism. I'm not sure how 
this would even be possible: if Red Hat or Debian can create a .missing 
file, so can anyone else.


> At least
> for Linux packages it is common to split out optional components of a
> python package into separate linux packages to limit the size and
> dependencies of the main package.  This could help a lot in that situation.

Well... I'm not sure how common that it. But it doesn't really matter.

This is a good argument for having a separate .missing file for each 
module, rather than a single flat registry of custom error messages. 
Separate .missing files will allow any Python package to easily install 
their own message. But either way, whether there's a single registry or 
an import hook that searches for .missing files if and only if the 
import failed, I haven't seen a strong argument for allowing arbitrary 
Python code.

(Earlier I suggested such a flat registry -- I now withdraw that 
suggestion. I'm satisfied that a separate spam.missing file containing 
the custom error message when spam.py cannot be found is a better way to 
handle this.)


-- 
Steve