[Python-ideas] PEP: Distributing a Subset of the Standard Library

[Tomas Orsava]

On 12/05/2016 01:42 PM, Nick Coghlan wrote:
> On 5 December 2016 at 19:56, Tomas Orsava <torsava at redhat.com> wrote:
>> On 12/03/2016 05:08 AM, Nick Coghlan wrote:
>>>> Though I believe the default sys.excepthook function is currently written
>>>> in
>>>> C, so it wouldn't be very easy for distributors to customize it. Maybe it
>>>> could be made to read module=error_message pairs from some external file,
>>>> which would be easier to modify?
>>> The default implementation is written in C, but distributors could
>>> patch site.py to replace it with a custom one written in Python. For
>>> example, publish a "fedora-hooks" module to PyPI (so non-system Python
>>> installations or applications regularly run without the site module
>>> can readily use the same hooks if they choose to do so), and then
>>> patch site.py in the system Python to do:
>>>
>>>       import fedora_hooks
>>>       fedora_hooks.install_excepthook()
>>>
>>> The nice thing about that approach is it wouldn't need a new switch to
>>> turn it off - it would get turned off with all the other site-specific
>>> customisations when -S or -I is used. It would also better open things
>>> up to redistributor experimentation in existing releases (2.7, 3.5,
>>> etc) before we commit to a specific approach in the reference
>>> interpreter (such as adding an optional 'platform.hooks' submodule
>>> that vendors may provide, and relevant stdlib APIs will then call
>>> automatically to override the default upstream provided processing).
>> Ah, but of course! That leaves us with only one part of the PEP unresolved:
>> When the build process is unable to compile some modules when building
>> Python from source (such as _sqlite3 due to missing sqlite headers), it
>> would be great to provide a custom message when one then tries to import
>> such module when using the compiled Python.
>>
>> Do you see a 'pretty' solution for that within this framework?
> I'm not sure it qualifies as 'pretty', but one approach would be to
> have a './Modules/missing/' directory that gets pre-populated with
> checked in "<name>.py" files for extension modules that aren't always
> built. When getpath.c detects it's running from a development
> checkout, it would add that directory to sys.path (just before
> site-packages), while 'make install' and 'make altinstall' would only
> copy files from that directory into the installation target if the
> corresponding extension modules were missing.
>
> Essentially, that would be the "name.missing.py" part of the draft
> proposal for optional standard library modules, just with a regular
> "name.py" module name and a tweak to getpath.c.

To my eye that looks like a complicated mechanism necessitating changes 
to several parts of the codebase. Have you considered modifying the 
default sys.excepthook implementation to read a list of modules and 
error messages from a file that was generated during the build process? 
To me that seems simpler, and the implementation will be only in one place.

In addition, distributors could just populate that file with their data, 
thus we would have one mechanism for both use cases.

Tomas