[Python-ideas] PEP: Distributing a Subset of the Standard Library

[M.-A. Lemburg]

I know that you started this thread focusing on the stdlib,
but for the purpose of distributors, the scope goes far
beyond just the stdlib.

Basically any Python module or package which the distribution can
provide should be usable as basis for a nice error message pointing to
the package to install.

Now, it's the distribution which knows which modules/packages
are available, so we don't need a list of stdlib modules
in Python to help with this.

The helper function (whether called via sys.excepthook() or perhaps
a new sys.importerrorhook()) would then check the imported
module name against this list and write out the message
pointing the user to the missing package.

A list of stdlib modules may still be useful, but it comes
with it's own set of problems, which should be irrelevant
for this use case: some stdlib modules are optional and
only available if the system provides (and Python can find)
certain libs (or header files during compilation).

For a distribution there are no optional stdlib modules,
since the distributor will know the complete list of available
modules in the distribution, including their external
dependencies.

In other words: Python already provides all the necessary
logic to enable implementing the suggested use case.


On 07.12.2016 06:24, Nick Coghlan wrote:
> On 7 December 2016 at 02:50, Tomas Orsava <torsava at redhat.com> wrote:
>> So using _sysconfigdata as inspiration, it would likely be possible to
>> provide a "sysconfig.get_missing_modules()" API that the default
>> sys.excepthook() could use to report that a particular import didn't
>> work because an optional standard library module hadn't been built.
>>
>> Quite interesting. And sysconfig.get_missing_modules() wouldn't even have to
>> be generated during the build process, because it would be called only when
>> the import has failed, at which point it is obvious Python was built without
>> said component (like _sqlite3). So do you see that as an acceptable
>> solution?
> 
> Oh, I'd missed that - yes, the sysconfig API could potentially be
> something like `sysconfig.get_stdlib_modules()` and
> `sysconfig.get_optional_modules()` instead of specifically reporting
> which ones were missed by the build process. There'd still be some
> work around generating the manifests backing those APIs at build time
> (including getting them right for Windows as well), but it would make
> some other questions that are currently annoying to answer relatively
> straightforward (see
> http://stackoverflow.com/questions/6463918/how-can-i-get-a-list-of-all-the-python-standard-library-modules
> for more on that)
> 
>> Do you prefer the one you suggested previously?
> 
> The only strong preference I have around how this is implemented is
> that I don't want to add complex single-purpose runtime infrastructure
> for the task. For all of the other specifics, I think it makes sense
> to err on the side of "What will be easiest to maintain over time?"
> 
>> Alternatively, can the contents of site.py be generated during the build
>> process? Because if some modules couldn't be built, a custom implementation
>> of sys.excepthook might be generated there with the data for the modules
>> that failed to be built.
> 
> We don't really want site.py itself to be auto-generated (although it
> could be updated to use Argument Clinic selectively if we deemed that
> to be an appropriate thing to do), but there's no problem with
> generating either data modules or normal importable modules that get
> accessed from site.py.
> 
> Cheers,
> Nick.
> 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Dec 07 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...           http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
                      http://www.malemburg.com/