__getattr__ bouncer for modules
Every now and then there's been talk of making it easier to subclass modules, and the most common use case that I can remember hearing about is descriptor protocol requiring code on the type. (For instance, you can't change a module-level constant into a property without moving away from the default ModuleType.)
How bad would it be for the default ModuleType to have a __getattr__ function which defers to the instance? Something like this:
def __getattr__(self, name): if '__getattr__' in self.__dict__: return self.__dict__['__getattr__'](name) raise AttributeError
The biggest downside I'm seeing is that module attributes double as global names, which might mean this would get checked for every global name that ends up being resolved from the builtins (which is going to be a LOT). But I'm not sure if that's even true.
Dumb idea? Already been thought of and rejected?
ChrisA
On Sat, Apr 16, 2016, at 22:50, Chris Angelico wrote:
def __getattr__(self, name): if '__getattr__' in self.__dict__: return self.__dict__['__getattr__'](name) raise AttributeError
The biggest downside I'm seeing is that module attributes double as global names, which might mean this would get checked for every global name that ends up being resolved from the builtins (which is going to be a LOT). But I'm not sure if that's even true.
It is not. (Also, incidentally, defining a global called __class__ does not set the module's class.)
I don't think this would be enough alone to let you use property decorators on a module - you'd have to explicitly define a __getattr__ (and __setattr__). And of course make sure that the names you're using as properties don't exist as real members of the module, since you're using __getattr__ instead of __getattribute__.
On Sun, Apr 17, 2016 at 1:26 PM, Random832 random832@fastmail.com wrote:
On Sat, Apr 16, 2016, at 22:50, Chris Angelico wrote:
def __getattr__(self, name): if '__getattr__' in self.__dict__: return self.__dict__['__getattr__'](name) raise AttributeError
The biggest downside I'm seeing is that module attributes double as global names, which might mean this would get checked for every global name that ends up being resolved from the builtins (which is going to be a LOT). But I'm not sure if that's even true.
It is not. (Also, incidentally, defining a global called __class__ does not set the module's class.)
I don't think this would be enough alone to let you use property decorators on a module - you'd have to explicitly define a __getattr__ (and __setattr__). And of course make sure that the names you're using as properties don't exist as real members of the module, since you're using __getattr__ instead of __getattribute__.
Right, it wouldn't automatically allow the use of properties *as such*, but you would be able to achieve most of the same goal.
# Version 1 BITS_PER_BYTE = 8 BITS_PER_WORD = 32
# Version 2 - doesn't work BITS_PER_BYTE = 8 @property def BITS_PER_WORD(): return 32 or 64
# Version 3 - could work BITS_PER_BYTE = 8 def __getattr__(name): if name == 'BITS_PER_WORD': return 32 or 64 raise AttributeError
It's not as clean as actually supporting @property, but it could be done without the "bootstrap problem" of trying to have a module contain the class that it's to be an instance of. All you have to do is define __getattr__ as a regular top-level function, and it'll get called. You can then dispatch to property functions if you wish (eg """return globals()['_property_'+name]()"""), or just put all the code straight into __getattr__.
ChrisA
On Sat, Apr 16, 2016, at 23:42, Chris Angelico wrote:
It's not as clean as actually supporting @property, but it could be done without the "bootstrap problem" of trying to have a module contain the class that it's to be an instance of. All you have to do is define __getattr__ as a regular top-level function, and it'll get called. You can then dispatch to property functions if you wish (eg """return globals()['_property_'+name]()"""), or just put all the code straight into __getattr__.
Or you could have ModuleType.__getattr(ibute?)__ search through some object other than the module itself for descriptors.
from types import ModuleType, SimpleNamespace import sys
# __magic__ could simply be a class, but just to prove it doesn't have to be:
@property def foo(self): return eval(input("get foo?"))
@foo.setter def foo(self, value): print("set foo=" + repr(value))
def __call__(self, v1, v2): print("modcall" + repr((v1, v2)))
__magic__ = SimpleNamespace() for name in 'foo __call__'.split(): setattr(__magic__, name, globals()[name]) del globals()[name]
class MagicModule(ModuleType): def __getattr__(self, name): descriptor = getattr(self.__magic__, name) return descriptor.__get__(self, None)
def __setattr__(self, name, value): try: descriptor = getattr(self.__magic__, name) except AttributeError: return super().__setattr__(name, value) return descriptor.__set__(self, value)
def __call__(self, *args, **kwargs): # Because why not? try: call = self.__magic__.__call__ except AttributeError: raise TypeError("Could not access " + self.__name__ + "__magic__.__call__ method.") return call(self, *args, **kwargs)
sys.modules[__name__].__class__ = MagicModule
On Sun, Apr 17, 2016 at 7:06 AM, Random832 random832@fastmail.com wrote:
On Sat, Apr 16, 2016, at 23:42, Chris Angelico wrote:
It's not as clean as actually supporting @property, but it could be done without the "bootstrap problem" of trying to have a module contain the class that it's to be an instance of. All you have to do is define __getattr__ as a regular top-level function, and it'll get called. You can then dispatch to property functions if you wish (eg """return globals()['_property_'+name]()"""), or just put all the code straight into __getattr__.
Or you could have ModuleType.__getattr(ibute?)__ search through some object other than the module itself for descriptors.
from types import ModuleType, SimpleNamespace import sys [...]
Not for a module, but for a *package*, this was reasonably easy (well, at least after figuring out how ;-) to do in pre-3.5 Python: One would import a submodule in __init__.py and then, within that submodule, reconstruct the package using a subclass of ModuleType and replace the original package in sys.modules.
-Koos
On Sat, Apr 16, 2016 at 7:50 PM, Chris Angelico rosuav@gmail.com wrote:
Every now and then there's been talk of making it easier to subclass modules, and the most common use case that I can remember hearing about is descriptor protocol requiring code on the type. (For instance, you can't change a module-level constant into a property without moving away from the default ModuleType.)
How bad would it be for the default ModuleType to have a __getattr__ function which defers to the instance? Something like this:
def __getattr__(self, name): if '__getattr__' in self.__dict__: return self.__dict__['__getattr__'](name) raise AttributeError
The biggest downside I'm seeing is that module attributes double as global names, which might mean this would get checked for every global name that ends up being resolved from the builtins (which is going to be a LOT). But I'm not sure if that's even true.
It's not true :-). Code executing inside the module has 'globals() is mod.__dict__', so lookups go directly to mod.__dict__ and skip mod.__getattr__.
However, starting in 3.5 cpython allows __class__ assignment on modules, so you can implement custom __getattr__ on a module with:
class ModuleWithMyGetattr(types.ModuleType): def __getattr__(self, name): # .. whatever you want ...
sys.modules[__name__].__class__ = ModuleWithMyGetattr
The advantage of doing it this way is that you can also implement other things like __dir__ (so tab completion on your new attributes will work).
This package backports the functionality to earlier versions of CPython:
https://pypi.python.org/pypi/metamodule https://github.com/njsmith/metamodule/
Basically just replace the explicit __class__ assignment with
import metamodule metamodule.install(__name__, ModuleWithMyGetattr)
(See the links above for more details and a worked example.)
-n
On Sun, Apr 17, 2016 at 1:40 PM, Nathaniel Smith njs@pobox.com wrote:
However, starting in 3.5 cpython allows __class__ assignment on modules, so you can implement custom __getattr__ on a module with:
class ModuleWithMyGetattr(types.ModuleType): def __getattr__(self, name): # .. whatever you want ...
sys.modules[__name__].__class__ = ModuleWithMyGetattr
The advantage of doing it this way is that you can also implement other things like __dir__ (so tab completion on your new attributes will work).
Oooh! I did not know that. That's pretty much what I was thinking of - you can have the class in the module that it's affecting. Coolness!
And inside that class, you can toss in @property and everything, so it's exactly as clean as it wants to be. I like! Thanks Nathaniel.
ChrisA
participants (4)
-
Chris Angelico -
Koos Zevenhoven -
Nathaniel Smith -
Random832