<div dir="ltr">All the use cases seem to be about adding some kind of getattr hook to modules. They all seem to involve modifying the CPython C code anyway. So why not tackle that problem head-on and modify module_getattro() to look for a global named __getattr__ and if it exists, call that instead of raising AttributeError?<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Nov 29, 2014 at 11:37 AM, Nathaniel Smith <span dir="ltr"><<a href="mailto:njs@pobox.com" target="_blank">njs@pobox.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Sat, Nov 29, 2014 at 4:21 AM, Guido van Rossum <<a href="mailto:guido@python.org">guido@python.org</a>> wrote:<br>
> Are these really all our options? All of them sound like hacks, none of them<br>
> sound like anything the language (or even the CPython implementation) should<br>
> sanction. Have I missed the discussion where the use cases and constraints<br>
> were analyzed and all other approaches were rejected? (I might have some<br>
> half-baked ideas, but I feel I should read up on the past discussion first,<br>
> and they are probably more fit for python-ideas than for python-dev. Plus<br>
> I'm just writing this email because I'm procrastinating on the type hinting<br>
> PEP. :-)<br>
<br>
</span>The previous discussions I was referring to are here:<br>
<a href="http://thread.gmane.org/gmane.comp.python.ideas/29487/focus=29555" target="_blank">http://thread.gmane.org/gmane.comp.python.ideas/29487/focus=29555</a><br>
<a href="http://thread.gmane.org/gmane.comp.python.ideas/29788" target="_blank">http://thread.gmane.org/gmane.comp.python.ideas/29788</a><br>
<br>
There might well be other options; these are just the best ones I<br>
could think of :-). The constraints are pretty tight, though:<br>
- The "new module" object (whatever it is) should have a __dict__ that<br>
aliases the original module globals(). I can elaborate on this if my<br>
original email wasn't enough, but hopefully it's obvious that making<br>
two copies of the same namespace and then trying to keep them in sync<br>
at the very least smells bad :-).<br>
- The "new module" object has to be a subtype of ModuleType, b/c there<br>
are lots of places that do isinstance(x, ModuleType) checks (notably<br>
-- but not only -- reload()). Since a major goal here is to make it<br>
possible to do cleaner deprecations, it would be really unfortunate if<br>
switching an existing package to use the metamodule support itself<br>
broke things :-).<br>
- Lookups in the normal case should have no additional performance<br>
overhead, because module lookups are extremely extremely common. (So<br>
this rules out dict proxies and tricks like that -- we really need<br>
'new_module.__dict__ is globals()' to be true.)<br>
<br>
AFAICT there are three logically possible strategies for satisfying<br>
that first constraint:<br>
(a) convert the original module object into the type we want, in-place<br>
(b) create a new module object that acts like the original module object<br>
(c) somehow arrange for our special type to be used from the start<br>
<br>
My options 1 and 2 are means of accomplishing (a), and my options 3<br>
and 4 are means of accomplishing (b) while working around the<br>
behavioural quirks of module objects (as required by the second<br>
constraint).<br>
<br>
The python-ideas thread did also consider several methods of<br>
implementing strategy (c), but they're messy enough that I left them<br>
out here. The problem is that somehow we have to execute code to<br>
create the new subtype *before* we have an entry in sys.modules for<br>
the package that contains the code for the subtype. So one option<br>
would be to add a new rule, that if a file pkgname/__new__.py exists,<br>
then this is executed first and is required to set up<br>
sys.modules["pkgname"] before we exec pkgname/__init__.py. So<br>
pkgname/__new__.py might look like:<br>
<br>
import sys<br>
from pkgname._metamodule import MyModuleSubtype<br>
sys.modules[__name__] = MyModuleSubtype(__name__, docstring)<br>
<br>
This runs into a lot of problems though. To start with, the 'from<br>
pkgname._metamodule ...' line is an infinite loop, b/c this is the<br>
code used to create sys.modules["pkgname"]. It's not clear where the<br>
globals dict for executing __new__.py comes from (who defines<br>
__name__? Currently that's done by ModuleType.__init__). It only works<br>
for packages, not modules. The need to provide the docstring here,<br>
before __init__.py is even read, is weird. It adds extra stat() calls<br>
to every package lookup. And, the biggest showstopper IMHO: AFAICT<br>
it's impossible to write a polyfill to support this code on old python<br>
versions, so it's useless to any package which needs to keep<br>
compatibility with 2.7 (or even 3.4). Sure, you can backport the whole<br>
import system like importlib2, but telling everyone that they need to<br>
replace every 'import numpy' with 'import importlib2; import numpy' is<br>
a total non-starter.<br>
<br>
So, yeah, those 4 options are really the only plausible ones I know of.<br>
<br>
Option 1 and option 3 are pretty nice at the language level! Most<br>
Python objects allow assignment to __class__ and __dict__, and both<br>
PyPy and Jython at least do support __class__ assignment. Really the<br>
only downside with Option 1 is that actually implementing it requires<br>
attention from someone with deep knowledge of typeobject.c.<br>
<span class="im HOEnZb"><br>
-n<br>
<br>
--<br>
Nathaniel J. Smith<br>
Postdoctoral researcher - Informatics - University of Edinburgh<br>
<a href="http://vorpus.org" target="_blank">http://vorpus.org</a><br>
</span><div class="HOEnZb"><div class="h5">_______________________________________________<br>
Python-Dev mailing list<br>
<a href="mailto:Python-Dev@python.org">Python-Dev@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/python-dev" target="_blank">https://mail.python.org/mailman/listinfo/python-dev</a><br>
Unsubscribe: <a href="https://mail.python.org/mailman/options/python-dev/guido%40python.org" target="_blank">https://mail.python.org/mailman/options/python-dev/guido%40python.org</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature">--Guido van Rossum (<a href="http://python.org/~guido">python.org/~guido</a>)</div>
</div>