[Python-Dev] advice needed: best approach to enabling "metamodules"?

[Guido van Rossum]

On Sun, Nov 30, 2014 at 11:29 AM, Nathaniel Smith <njs at pobox.com> wrote:

> On Sun, Nov 30, 2014 at 2:54 AM, Guido van Rossum <guido at python.org>
> wrote:
> > All the use cases seem to be about adding some kind of getattr hook to
> > modules. They all seem to involve modifying the CPython C code anyway. So
> > why not tackle that problem head-on and modify module_getattro() to look
> for
> > a global named __getattr__ and if it exists, call that instead of raising
> > AttributeError?
>
> You need to allow overriding __dir__ as well for tab-completion, and
> some people wanted to use the properties API instead of raw
> __getattr__, etc. Maybe someone will want __getattribute__ semantics,
> I dunno.


Hm... I agree about __dir__ but the other things feel too speculative.


> So since we're *so close* to being able to just use the
> subclassing machinery, it seemed cleaner to try and get that working
> instead of reimplementing bits of it piecewise.
>

That would really be option 1, right? It's the one that looks cleanest from
the user's POV (or at least from the POV of a developer who wants to build
a framework using this feature -- for a simple one-off use case,
__getattr__ sounds pretty attractive). I think that if we really want
option 1, the issue of PyModuleType not being a heap type can be dealt with.


> That said, __getattr__ + __dir__ would be enough for my immediate use
> cases.


 Perhaps it would be a good exercise to try and write the "lazy submodule
import"(*) use case three ways: (a) using only CPython 3.4; (b) using
__class__ assignment; (c) using customizable __getattr__ and __dir__. I
think we can learn a lot about the alternatives from this exercise. I
presume there's already a version of (a) floating around, but if it's been
used in practice at all, it's probably too gnarly to serve as a useful
comparison (though its essence may be extracted to serve as such).

FWIW I believe all proposals here have a big limitation: the module
*itself* cannot benefit much from all these shenanigans, because references
to globals from within the module's own code are just dictionary accesses,
and we don't want to change that.

(*) I originally wrote "lazy import", but I realized that messing with the
module class object probably isn't the best way to implement that -- it
requires a proxy for the module that's managed by an import hook. But if
you think it's possible, feel free to use this example, as "lazy import"
seems a pretty useful thing to have in many situations. (At least that's
how I would do it. And I would probably add some atrocious hack to patch up
the importing module's globals once the module is actually loaded, to
reduce the cost of using the proxy over the lifetime of the process.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141130/206921a5/attachment-0001.html>