[Python-ideas] PEP: Distributing a Subset of the Standard Library

[M.-A. Lemburg]

On 07.12.2016 13:57, Nick Coghlan wrote:
> On 7 December 2016 at 18:33, M.-A. Lemburg <mal at egenix.com> wrote:
>> I know that you started this thread focusing on the stdlib,
>> but for the purpose of distributors, the scope goes far
>> beyond just the stdlib.
>>
>> Basically any Python module or package which the distribution can
>> provide should be usable as basis for a nice error message pointing to
>> the package to install.
> 
> The PEP draft covered two questions:
> 
> - experienced redistributors breaking the standard library up into pieces
> - optional modules for folks building their own Python (even if
> they're new to that)
> 
>> Now, it's the distribution which knows which modules/packages
>> are available, so we don't need a list of stdlib modules
>> in Python to help with this.
> 
> Right, that's the case that we realised can be covered entirely by the
> suggestion "patch site.py to install a different default
> sys.excepthook()"
> 
>> A list of stdlib modules may still be useful, but it comes
>> with it's own set of problems, which should be irrelevant
>> for this use case: some stdlib modules are optional and
>> only available if the system provides (and Python can find)
>> certain libs (or header files during compilation).
> 
> While upstream changes turned out not to be necessary for the
> "distributor breaking up the standard library" use case, they may
> still prove worthwhile in making import errors more informative in the
> case of "I just built my own Python from upstream sources and didn't
> notice (or didn't read) the build message indicating that some modules
> weren't built".
> 
> Given the precedent of the sysconfig metadata generation, providing
> some form of machine-readable build-time-generated module manifest
> should be pretty feasible if someone was motivated to implement it,
> and we already have the logic to track which optional modules weren't
> built in order to generate the message at the end of the build
> process.

True, but the build process only covers C extensions. Writing
the information somewhere for Python to pick up would be easy,
though (just dump the .failed* lists somewhere).

For pure Python modules, I suppose the install process could
record all installed modules.

Put all this info into a generated "_sysconfigstdlib" module,
import this into sysconfig and you're set.

Still, in all the years I've been using Python I never ran
into a situation where I was interested in such information.

For cases where a module is optional, you usually write
a try...except and handle this on a case-by-case basis.

That's safer than relying on some build time generated
list, since the Python binary may well have been built
on a different machine than the one the application is
currently running on and so, even if an optional module
is listed as built successfully, it may still fail to
import.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Dec 07 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...           http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
                      http://www.malemburg.com/