Re: [Python-ideas] metamodules (was: Re: Idea to support lazy loaded names.)

On Thu, Oct 9, 2014 at 11:21 PM, Nick Coghlan wrote:

On 10 Oct 2014 00:05, "Nathaniel Smith" wrote:

...

...

Valid reasons would be to make it easier for alternative interpreters, or to catch edge cases (e.g. make it work nicely with custom importers, zip files, etc). But at that point, we can just do it the "right way"*, have the backcompat story to a third-party shim.

* i.e. "not paying attention to older Pythons" – I'm not saying __new__.py is necessarily the right way, I'm criticizing this reason against it

My point is just that it matters whether a backcompat shim is doable. Probably there is some way to implement a backcompat shim for __new__.py, but I don't immediately know how to do it.

Eric Snow already backported the Python 3.4 importlib to Python 2.7 as importlib2, and I know of at least one large company that is planning to deploy that in order to benefit from the directory caching feature. It's a reasonably safe assumption that any future Python 3 import system changes will be made available for Python 2.7 the same way.

I don't really see how importlib2 can help here. I mean, we can't reasonably tell people "okay, this is very important -- before you run 'import numpy' (or whatever) you first have to run 'import importlib2; importlib2.hook.install()' or else it won't work". The bootstrapping problem here is nasty, because the whole idea is that __new__.py would be executed before *any* other code distributed with the package.

That said...

...

And again, this is to be compared to the __class__ assignment approach, where we know very well that a backcompat shim is doable because it is done :-).

I actually agree making modules a proxy type that allows setting __class__ on instances is likely the simpler solution here.

You won't get full magic method support, but it *will* enable use of the descriptor protocol for module level attributes.

Like Andrew, I'm not sure what you mean here. Note that there's various code (e.g. reload()) which insists that module objects must be instances of the built-in ModuleType class. -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org