[Python-ideas] PEP: Distributing a Subset of the Standard Library

Tomas Orsava torsava at redhat.com
Fri Dec 2 11:56:29 EST 2016

On 11/30/2016 03:56 AM, Nick Coghlan wrote:
> Really, I think the ideal solution from a distro perspective would be
> to enable something closer to what bash and other shells support for
> failed CLI calls:
>      $ blender
>      bash: blender: command not found...
>      Install package 'blender' to provide command 'blender'? [N/y] n
> This would allow redistributors to point folks towards platform
> packages (via apt/yum/dnf/PyPM/conda/Canopy/etc) for the components
> they provide, and towards pip/PyPI for everything else (and while we
> don't have a dist-lookup-by-module-name service for PyPI *today*, it's
> something I hope we'll find a way to provide sometime in the next few
> years).
> I didn't suggest that during the Fedora-level discussions of this PEP
> because it didn't occur to me - the elegant simplicity of the new
> import suffix as a tactical solution to the immediate "splitting the
> standard library" problem [1] meant I missed that it was really a
> special case of the general "provide guidance on obtaining missing
> modules from the system package manager" concept.
> The problem with that idea however is that while it provides the best
> possible interactive user experience, it's potentially really slow,
> and hence too expensive to do for every import error - we would
> instead need to find a way to run with Wolfgang Maier's suggestion of
> only doing this for *unhandled* import errors.
> Fortunately, we do have the appropriate mechanisms in place to support
> that approach:
> 1. For interactive use, we have sys.excepthook
> 2. For non-interactive use, we have the atexit module
> As a simple example of the former:
>      >>> def module_missing(modname):
>      ...     return f"Module not found: {modname}"
>      >>> def my_except_hook(exc_type, exc_value, exc_tb):
>      ...     if isinstance(exc_value, ModuleNotFoundError):
>      ...         print(module_missing(exc_value.name))
>      ...
>      >>> sys.excepthook = my_except_hook
>      >>> import foo
>      Module not found: foo
>      >>> import foo.bar
>      Module not found: foo
>      >>> import sys.bar
>      Module not found: sys.bar
> For the atexit handler, that could be installed by the `site` module,
> so the existing mechanisms for disabling site module processing would
> also disable any default exception reporting hooks. Folks could also
> register their own handlers via either `sitecustomize.py` or
> `usercustomize.py`.

Is there some reason not to use sys.excepthook for both interactive and 
non-interactive use? From the docs:

    "When an exception is raised and uncaught, the interpreter
    calls|sys.excepthook|with three arguments, the exception class,
    exception instance, and a traceback object. In an interactive
    session this happens just before control is returned to the prompt;
    in a Python program this happens just before the program exits. The
    handling of such top-level exceptions can be customized by assigning
    another three-argument function to|sys.excepthook|."

Though I believe the default sys.excepthook function is currently 
written in C, so it wouldn't be very easy for distributors to customize 
it. Maybe it could be made to read module=error_message pairs from some 
external file, which would be easier to modify?

Yours aye,

> And at that point the problem starts looking less like "Customise the
> handling of missing modules" and more like "Customise the rendering
> and reporting of particular types of unhandled exceptions". For
> example, a custom handler for subprocess.CalledProcessError could
> introspect the original command and use `shutil.which` to see if the
> requested command was even visible from the current process (and, in a
> redistributor provided Python, indicate which system packages to
> install to obtain the requested command).
>> My personal vote is a callback called at
>> https://github.com/python/cpython/blob/master/Lib/importlib/_bootstrap.py#L948
>> with a default implementation that raises ModuleNotFoundError just like the
>> current line does.
> Ethan's observation about try/except import chains has got me think
> that limiting this to handling errors within the context of single
> import statement will be problematic, especially given that folks can
> already write their own metapath hook for that case if they really
> want to.
> Cheers,
> Nick.
> [1] For folks wondering "This problem has existed for years, why
> suddenly worry about it now?", Fedora's in the process of splitting
> out an even more restricted subset of the standard library for system
> tools to use: https://fedoraproject.org/wiki/Changes/System_Python
> That means "You're relying on a missing stdlib module" is going to
> come up more often for system tools developers trying to stick within
> that restricted subset.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20161202/600bed9a/attachment-0001.html>

More information about the Python-ideas mailing list