[Python-ideas] PEP: Distributing a Subset of the Standard Library

[Eric V. Smith]

On 11/29/2016 1:33 PM, Brett Cannon wrote:
>
>
> On Tue, 29 Nov 2016 at 06:49 Nick Coghlan <ncoghlan at gmail.com
> <mailto:ncoghlan at gmail.com>> wrote:
>
>     On 29 November 2016 at 20:54, Tomas Orsava <torsava at redhat.com
>     <mailto:torsava at redhat.com>> wrote:
>     > With a metapath hook, .missing.py files are probably overkill, and
>     the hook
>     > can just look at one file (or a static compiled-in list) of
>     > ModuleNotFound/ImportError messages for all missing modules, as M.-A.
>     > Lemburg and others are suggesting. We'll just need to think about
>     > coordinating how the list is generated/updated: the current PEP
>     implicitly
>     > allows other parties, besides Python and the distributors, to step in
>     > cleanly if they need to—needing to update a single list could lead
>     to messy
>     > hacks.
>
>     What if, rather than using an explicitly file-based solution, this was
>     instead defined as a new protocol module, where the new metapath hook
>     imported a "__missing__" module and called a particular function in it
>     (e.g. "__missing__.module_not_found(modname)")?
>
>
> You can answer this question the best, Nick, but would it be worth
> defining a _stdlib.py that acts as both a marker for where the stdlib is
> installed -- instead of os.py which is the current marker -- and which
> also stores metadata like an attribute called `missing` which is a dict
> that maps modules to ModuleNotFoundError messages? Although maybe this
> is too specific of a solution (or still too general and we use an e.g.
> missing.json off of sys.path which contains the same mapping).
>
> Otherwise MAL touched on the solution I always had in the back of my
> head where we let people register a callback that gets passed the name
> of any module that wasn't found through sys.meta_path. We could either
> have the return value mean nothing and by default raise
> ModuleNotFoundError, have the return value be what to set the module to
> and raise an exception as expected, or have it be more error-specific
> and return an exception to raise (all of these options also ask whether
> a default callback doing what is normal is provided or if it's None by
> default and import continues to provide the default semantics). The perk
> of the callback is it removes the order sensitivity of any sys.meta_path
> or sys.path_hooks solution where people might be doing
> sys.meta_path.append(custom_finder) and thus won't necessarily trigger
> if a new hook for missing modules is put at the end by default.

How about having a sys.meta_path_last_chance (or whatever), which is 
identical to anything else on sys.meta_path, but is always guaranteed to 
be called last.

Then instead of:
meta_path = sys.meta_path

it would be:
meta_path = sys.meta_path + ([sys.meta_path_last_chance] if 
sys.meta_path_last_chance else [])

There's no need to invent a new callback signature, or any new logic 
anywhere: it's literally just another metapath importer, but always 
guaranteed to be last.

Eric.