[Python-ideas] metamodules (was: Re: Idea to support lazy loaded names.)

[Nick Coghlan]

On 10 Oct 2014 10:16, "Nathaniel Smith" <njs at pobox.com> wrote:
>
> On Thu, Oct 9, 2014 at 11:21 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> >
> > On 10 Oct 2014 00:05, "Nathaniel Smith" <njs at pobox.com> wrote:
> >>
> >> > Valid reasons would be to make it easier for alternative
interpreters,
> >> > or to catch edge cases (e.g. make it work nicely with custom
> >> > importers, zip files, etc). But at that point, we can just do it the
> >> > "right way"*, have the backcompat story to a third-party shim.
> >> >
> >> > * i.e. "not paying attention to older Pythons" – I'm not saying
> >> > __new__.py is necessarily the right way, I'm criticizing this reason
> >> > against it
> >>
> >> My point is just that it matters whether a backcompat shim is doable.
> >> Probably there is some way to implement a backcompat shim for
__new__.py,
> >> but I don't immediately know how to do it.
> >
> > Eric Snow already backported the Python 3.4 importlib to Python 2.7 as
> > importlib2, and I know of at least one large company that is planning to
> > deploy that in order to benefit from the directory caching feature.
It's a
> > reasonably safe assumption that any future Python 3 import system
changes
> > will be made available for Python 2.7 the same way.
>
> I don't really see how importlib2 can help here. I mean, we can't
> reasonably tell people "okay, this is very important -- before you run
> 'import numpy' (or whatever) you first have to run 'import importlib2;
> importlib2.hook.install()' or else it won't work".

Yes, that's exactly what the Python 2 usage model would be if relying on an
import system feature.

> The bootstrapping
> problem here is nasty, because the whole idea is that __new__.py would
> be executed before *any* other code distributed with the package.

If folks want Python 3 import system features in Python 2.7, importlib2 is
how to get them. Most single source packages won't want to do that because
of the bootstrapping challenge, but that's not the same thing as being
entirely incompatible with Python 2.

> > That said...
> >
> >> And again, this is to be compared to the __class__ assignment approach,
> >> where we know very well that a backcompat shim is doable because it is
done
> >> :-).
> >
> > I actually agree making modules a proxy type that allows setting
__class__
> > on instances is likely the simpler solution here.
> >
> > You won't get full magic method support, but it *will* enable use of the
> > descriptor protocol for module level attributes.
>
> Like Andrew, I'm not sure what you mean here.
>
> Note that there's various code (e.g. reload()) which insists that
> module objects must be instances of the built-in ModuleType class.

I'm agreeing with you. One of the main characteristics of proxy types is
that type(x) and x.__class__ give different answers, which is how I
interpreted your metamodule proposal.

Cheers,
Nick.

>
> -n
>
> --
> Nathaniel J. Smith
> Postdoctoral researcher - Informatics - University of Edinburgh
> http://vorpus.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20141010/f818f94e/attachment-0001.html>