new super redux (better late than never?)
I was looking at the reference implementation in PEP 3135 (New Super), and I was inspired to put together a slightly different implementation that doesn't fiddle with bytecode. I know that the new super() in python 3000 doesn't follow the reference implementation in the PEP, but the code intrigued me enough to offer up this little tidbit, which can be easily be used in python 2.5. What I did was borrow the idea of using a metaclass to do a post-definition fix-up on the methods, but added a new function decorator called autosuper_method. Like staticmethod or classmethod, the decorator wraps the function using the non-data descriptor protocol. The method wrapped by the decorator will receive an extra implicit argument (super) inserted before the instance argument (self). One caveat about the decorator: it must be the first decorator in the list (i.e. the outermost wrapper), or else the metaclass will not recognize the wrapped function as an instance of the decorator class. I think this implementation strikes me as more pythonic than the spooky behavior of the new python 3000 super() built-in, and it is more flexible because of the implicit argument design. This allows things like the ability to use the super argument in inner functions without worrying about the 'first argument' assumption of python 3000's super(). The implementation follows, which is also called autosuper in deference to the original reference implementation. It includes a demonstration of some of its flexibility: ------------------------------------------------------------ #!/usr/bin/env python # # autosuper.py class autosuper_method(object): def __init__(self, func, cls=None): self.func = func self.cls = cls def __get__(self, obj, type=None): # return self if self.cls is not set yet if self.cls is None: return self if obj is None: # class binding - assume first argument is instance, # and insert superclass before it def newfunc(*args, **kwargs): if not len(args): raise TypeError('instance argument missing') return self.func(super(self.cls, args[0]), *args, **kwargs) else: # instance binding - insert superclass as first # argument, and instance as second def newfunc(*args, **kwargs): return self.func(super(self.cls, obj), obj, *args, **kwargs) return newfunc class autosuper_meta(type): def __init__(cls, name, bases, clsdict): # set cls attribute of all instances of autosuper_method for v in clsdict: o = getattr(cls, v) if isinstance(o, autosuper_method): o.cls = cls class autosuper(object): __metaclass__ = autosuper_meta if __name__ == '__main__': class A(autosuper): def f(self): return 'A' # Demo - standard use class B(A): @autosuper_method def f(super, self): return 'B' + super.f() # Demo - reference super in inner function class C(A): @autosuper_method def f(super, self): def inner(): return 'C' + super.f() return inner() # Demo - define function before class definition @autosuper_method def D_f(super, self): return 'D' + super.f() class D(B, C): f = D_f # Demo - define function after class definition class E(B, C): pass # don't use @autosuper_method here! The metaclass has already # processed E, so it won't be able to set the cls attribute def E_f(super, self): return 'E' + super.f() # instead, use the extended version of the decorator E.f = autosuper_method(E_f, E) d = D() assert d.f() == 'DBCA' # Instance binding assert D.f(d) == 'DBCA' # Class binding e = E() assert e.f() == 'EBCA' # Instance binding assert E.f(e) == 'EBCA' # Class binding ------------------------------------------------------------ P.S. I know that using the word 'super' as an argument name might be frowned upon, but I'm just copying what I've seen done in the standard python library (e.g. using 'list' as a local variable name :). Anyway, it doesn't really hurt anything unless you wanted to call the original super() built-in from the decorated method, which would kind of defeat the purpose. P.P.S. Something like this might have been offered up already. I've been searching the mail list archives for a while, and found a few reference to using decorators, but didn't find any full implementations. This implementation also has the advantage of being compatible with existing code.
Ehhh! The PEP's "reference implementation" is useless and probably doesn't even work. The actual implementation is completely different. If you want to help, a rewrite of the PEP to match reality would be most welcome! On Tue, Mar 4, 2008 at 6:36 PM, Anthony Tolle <artomegus@gmail.com> wrote:
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Tue, Mar 4, 2008 at 10:24 PM, Guido van Rossum <guido@python.org> wrote:
Yep, I knew the actual implementation was completely different from the reference implementation. I was really just trying to offer a different take on 'fixing' super, even though I know it is too late to suggest this type of change for python 3000. That's one reason I refrained from posting in the python-3000 list. I was enamored with the idea of passing the super object as an actual parameter to the method that needs it. Using a decorator with descriptor behavior (like staticmethod or classmethod) seemed the best way to do this. The only downside is that my implementation depends on using a metaclass to fix up the decorator objects after the class definition is completed (or catching assignment to class attributes after the fact). It would be nice if the decorator class could be self-contained without depending on an associated metaclass. However, the __get__ method of the decorator would have to dynamically determine the class that the wrapped function belongs to. Since functions can be defined outside of a class and then arbitrarily assigned to a class attribute (or even multiple classes!), this seems to be difficult. In fact, the code in my previous post has a bug related to this. Which brings me to posting a new version of my code: -- Defined __setattr__ in the metaclass to make demo code more consistent (and less ugly). -- Modified __init__ function in the metaclass so it doesn't generate __get__ calls. -- Fix-ups now create new instance of autosuper_method object instead of modifying cls attribute of existing object. Reason: assigning a decorated function to multiple classes would modify the original object, breaking functionality for all classes but one. -- Known issue: cases such as E.f = D.f are not caught, because __get__ on D.f doesn't return an instance of autosuper_method. Can be resolved by having autosuper_method.__get__ return a callable sublass of autosuper_method. However, it makes me wonder if my idea isn't so hot after all. :/ Here's the new version: ------------------------------------------------------------ #!/usr/bin/env python # # autosuper.py class autosuper_method(object): def __init__(self, func, cls=None): self.func = func self.cls = cls def __get__(self, obj, type=None): # return self if self.cls is not set - prevents use # by methods of classes that don't subclass autosuper if self.cls is None: return self if obj is None: # class binding - assume first argument is instance, # and insert superclass before it def newfunc(*args, **kwargs): if not len(args): raise TypeError('instance argument missing') return self.func(super(self.cls, args[0]), *args, **kwargs) else: # instance binding - insert superclass as first # argument, and instance as second def newfunc(*args, **kwargs): return self.func(super(self.cls, obj), obj, *args, **kwargs) return newfunc class autosuper_meta(type): def __init__(cls, name, bases, clsdict): # fix up all autosuper_method instances in class for attr in clsdict: value = clsdict[attr] if isinstance(value, autosuper_method): setattr(cls, attr, autosuper_method(value.func, cls)) def __setattr__(cls, attr, value): # catch assignment after class definition if isinstance(value, autosuper_method): value = autosuper_method(value.func, cls) type.__setattr__(cls, attr, value) class autosuper(object): __metaclass__ = autosuper_meta if __name__ == '__main__': class A(autosuper): def f(self): return 'A' # Demo - standard use class B(A): @autosuper_method def f(super, self): return 'B' + super.f() # Demo - reference super in inner function class C(A): @autosuper_method def f(super, self): def inner(): return 'C' + super.f() return inner() # Demo - define function before class definition @autosuper_method def D_f(super, self): return 'D' + super.f() class D(B, C): f = D_f # Demo - define function after class definition class E(B, C): pass @autosuper_method def E_f(super, self): return 'E' + super.f() E.f = E_f # Test D d = D() assert d.f() == 'DBCA' # Instance binding assert D.f(d) == 'DBCA' # Class binding # Test E e = E() assert e.f() == 'EBCA' # Instance binding assert E.f(e) == 'EBCA' # Class binding ------------------------------------------------------------ Regardless of the flaws in my code, I still like the idea of a decorator syntax to specify methods that want to receive a super object as a parameter. It could use the same 'magic' that allows the new python 3000 super() to determine the method's class from the stack frame, but doesn't depend on grabbing the first argument as the instance (i.e. breaking use with inner functions).
Ehhh! The PEP's "reference implementation" is useless and probably doesn't even work. The actual implementation is completely different. If you want to help, a rewrite of the PEP to match reality would be most welcome! On Tue, Mar 4, 2008 at 6:36 PM, Anthony Tolle <artomegus@gmail.com> wrote:
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Tue, Mar 4, 2008 at 10:24 PM, Guido van Rossum <guido@python.org> wrote:
Yep, I knew the actual implementation was completely different from the reference implementation. I was really just trying to offer a different take on 'fixing' super, even though I know it is too late to suggest this type of change for python 3000. That's one reason I refrained from posting in the python-3000 list. I was enamored with the idea of passing the super object as an actual parameter to the method that needs it. Using a decorator with descriptor behavior (like staticmethod or classmethod) seemed the best way to do this. The only downside is that my implementation depends on using a metaclass to fix up the decorator objects after the class definition is completed (or catching assignment to class attributes after the fact). It would be nice if the decorator class could be self-contained without depending on an associated metaclass. However, the __get__ method of the decorator would have to dynamically determine the class that the wrapped function belongs to. Since functions can be defined outside of a class and then arbitrarily assigned to a class attribute (or even multiple classes!), this seems to be difficult. In fact, the code in my previous post has a bug related to this. Which brings me to posting a new version of my code: -- Defined __setattr__ in the metaclass to make demo code more consistent (and less ugly). -- Modified __init__ function in the metaclass so it doesn't generate __get__ calls. -- Fix-ups now create new instance of autosuper_method object instead of modifying cls attribute of existing object. Reason: assigning a decorated function to multiple classes would modify the original object, breaking functionality for all classes but one. -- Known issue: cases such as E.f = D.f are not caught, because __get__ on D.f doesn't return an instance of autosuper_method. Can be resolved by having autosuper_method.__get__ return a callable sublass of autosuper_method. However, it makes me wonder if my idea isn't so hot after all. :/ Here's the new version: ------------------------------------------------------------ #!/usr/bin/env python # # autosuper.py class autosuper_method(object): def __init__(self, func, cls=None): self.func = func self.cls = cls def __get__(self, obj, type=None): # return self if self.cls is not set - prevents use # by methods of classes that don't subclass autosuper if self.cls is None: return self if obj is None: # class binding - assume first argument is instance, # and insert superclass before it def newfunc(*args, **kwargs): if not len(args): raise TypeError('instance argument missing') return self.func(super(self.cls, args[0]), *args, **kwargs) else: # instance binding - insert superclass as first # argument, and instance as second def newfunc(*args, **kwargs): return self.func(super(self.cls, obj), obj, *args, **kwargs) return newfunc class autosuper_meta(type): def __init__(cls, name, bases, clsdict): # fix up all autosuper_method instances in class for attr in clsdict: value = clsdict[attr] if isinstance(value, autosuper_method): setattr(cls, attr, autosuper_method(value.func, cls)) def __setattr__(cls, attr, value): # catch assignment after class definition if isinstance(value, autosuper_method): value = autosuper_method(value.func, cls) type.__setattr__(cls, attr, value) class autosuper(object): __metaclass__ = autosuper_meta if __name__ == '__main__': class A(autosuper): def f(self): return 'A' # Demo - standard use class B(A): @autosuper_method def f(super, self): return 'B' + super.f() # Demo - reference super in inner function class C(A): @autosuper_method def f(super, self): def inner(): return 'C' + super.f() return inner() # Demo - define function before class definition @autosuper_method def D_f(super, self): return 'D' + super.f() class D(B, C): f = D_f # Demo - define function after class definition class E(B, C): pass @autosuper_method def E_f(super, self): return 'E' + super.f() E.f = E_f # Test D d = D() assert d.f() == 'DBCA' # Instance binding assert D.f(d) == 'DBCA' # Class binding # Test E e = E() assert e.f() == 'EBCA' # Instance binding assert E.f(e) == 'EBCA' # Class binding ------------------------------------------------------------ Regardless of the flaws in my code, I still like the idea of a decorator syntax to specify methods that want to receive a super object as a parameter. It could use the same 'magic' that allows the new python 3000 super() to determine the method's class from the stack frame, but doesn't depend on grabbing the first argument as the instance (i.e. breaking use with inner functions).
participants (2)
-
Anthony Tolle -
Guido van Rossum