On 11/13/07, Oleg Broytmann wrote:

On Tue, Nov 13, 2007 at 12:23:40AM -0700, Neil Toronto wrote:

...

I've got a working prototype metaclass and class instance (require_super) and decorator (super_required).

Is this restricted to __init__ (and __new__?) or could it be used on any method? Is there (and should there be?) a way around it, by catching the TypeError? By creating a decoy object to call super on?

Chicken and egg problem, in my eyes. If the user is clever enough to use the class and the decorator isn't she clever enough to call inherited __init__?

It may not be the same user. A library or framework writer would create the base class and use the decorator to (somewhat) ensure that subclasses meet the full interface requirements. A subclass writer should call the super.__init__ because it is in the API, but Neil's metaclass makes it easier to debug if they forget. -jJ

Since I know you're all dying to see the code... ;) This works for instance methods, classmethods, staticmethods (if cls or self is the first parameter, as in __new__), and probably most decorated methods. Current Issues I Can Think Of: - @classmethod overrides that don't redecorate with @classmethod always raise TypeError (maybe not such a bad thing) - exceptions can cause entries in the __supercalls__ set to accumulate unboundedly (can be fixed) Anyway, here's a concrete implementation. Is the "missed super call" problem big or annoying enough to warrant having language, runtime, or library support for a similar solution? import threading import types class _supercall_set(object): def __init__(self, *args, **kwargs): # Removing threading.local() should make this faster # but not thread-safe self.loc = threading.local() self.loc.s = set(*args, **kwargs) def add(self, key): self.loc.s.add(key) def discard(self, key): self.loc.s.discard(key) def __contains__(self, key): return self.loc.s.__contains__(key) def __repr__(self): return self.loc.s.__repr__() def _unwrap_rewrap(func): '''For supported types (classmethod, staticmethod, function), returns the actual function and a function to re-wrap it, if necessary. Raises TypeError if func's type isn't supported.''' if isinstance(func, classmethod): return func.__get__(func).im_func, type(func) elif isinstance(func, staticmethod): return func.__get__(func), type(func) elif isinstance(func, types.FunctionType): return func, lambda func: func raise TypeError('unsupported type %s' % type(func)) def super_required(func): ''' Marks a method as requiring subclass overrides to call it, either directly or via a super() call. Works with all undecorated methods, classmethods, staticmethods (fragile: only if 'cls' or 'self' is the first parameter) including __new__, and probably most other decorated methods. Correct operation is guaranteed only when the method is in a subclass of require_super. If a super_required override has a superclass method that is also super_required, the override will not be required to call the superclass method, either directly or via a super() call. The superclass call requirement can be cancelled for a method and methods of the same name in all future subclasses using the super_not_required decorator. The implementation should be as thread-safe as the classes it's used in. Recursion should work as long as the last, innermost call calls the superclass method. (It's usually best to avoid it.) This is not robust to method injection, but then again, what is? Examples: class A(require_super): @super_required def __init__(self): pass class B(A): def __init__(self): pass b = B() # TypeError: B.__init__: no super call # B.__init__ needs a super(B, self).__init__() class C(require_super): @super_required @classmethod # order of decorators doesn't matter def clsmeth(cls): pass class D(C): @classmethod def clsmeth(cls): pass d = D() d.clsmeth() # TypeError: D.clsmeth: no super call # C.clsmeth needs a super(C, cls).clsmeth() ''' func, rewrap = _unwrap_rewrap(func) name = func.func_name def super_wrapper(self_or_cls, *args, **kwargs): retval = func(self_or_cls, *args, **kwargs) # Flag that the super call happened self_or_cls.__supercalls__.discard((id(self_or_cls), name)) return retval super_wrapper.func_name = func.func_name super_wrapper.func_doc = func.func_doc super_wrapper.__super_required__ = True # Pass it down return rewrap(super_wrapper) def super_not_required(func): '''Marks a method as no longer requiring subclass overrides to call it. This is only meaningful for methods in subclasses of require_super.''' func.__super_required__ = False return func def _get_sub_wrapper(func, class_name, method_name): '''Returns a wrapper function that: 1. Adds key to __supercalls__ 2. Calls the wrapped function 3. Checks for key in __supercalls__ - if there, raises TypeError''' def sub_wrapper(self_or_cls, *args, **kwargs): key = (id(self_or_cls), method_name) self_or_cls.__supercalls__.add(key) retval = func(self_or_cls, *args, **kwargs) if key not in self_or_cls.__supercalls__: return retval self_or_cls.__supercalls__.discard(key) raise TypeError("%s.%s: no super call" % (class_name, method_name)) sub_wrapper.func_name = func.func_name sub_wrapper.func_doc = func.func_doc sub_wrapper.__super_required__ = True # Pass it down return sub_wrapper class _require_super_meta(type): def __new__(typ, cls_name, bases, dct): # Search through all attributes for method_name, func in dct.items(): try: func, rewrap = _unwrap_rewrap(func) except TypeError: continue # unsupported type if hasattr(func, '__super_required__'): continue # decorated - don't wrap it again # See if a base class's method is __super_required__ for base in bases: try: if getattr(base, method_name).__super_required__: break except AttributeError: pass # not there or no __super_required__ else: continue # outer loop # Wrap up the function newfunc = _get_sub_wrapper(func, cls_name, method_name) dct[method_name] = rewrap(newfunc) return type.__new__(typ, cls_name, bases, dct) class require_super(object): '''Inheriting from require_super makes super_required and super_not_required decorators work.''' __metaclass__ = _require_super_meta # This will be visible to classes and instances __supercalls__ = _supercall_set()

Jim Jewett wrote:

I don't yet see what those simplifications should actually be, but maybe someone else will if you publish and wait long enough.

The first thing I noticed was that the naming scheme is confusing. Between required_super and super_required, neither of them indicate to me which is the function decorator and which is the base class. Furthermore, I don't see why required_super (the base class) needs a distinct name. Perhaps I am being a bit to clever, but couldn't we just overload the __new__ method of the base class. def _super_required(func): ... class super_required(object): ... def __new__(cls, *func): if len(func) > 0: return _super_required(*func) return object.__new__(cls) Leaving your example now being spelled as: class A(super_required): @super_required def __init__(self): pass I can't think of a case that the the base class would ever be passed arguments, so this seems ok and rids us of the naming oddities. -Scott -- Scott Dial scott@scottdial.com scodial@cs.indiana.edu