[Python-ideas] Eliminating special method lookup (was Re: Missing Core Feature: + - * / | & do not call __getattr__)

[Steven D'Aprano]

On Fri, Dec 04, 2015 at 12:02:46PM -0800, Andrew Barnert via Python-ideas wrote:

> So, maybe the way to get from here to there is to explicitly document 
> the methods CPython treats as magic methods, 

That's probably a good idea regardless of anything else.


> and allow only allow 
> other implementations to do the same for (a subset of) the same, and 
> people can gradually tackle and remove parts of that list as people 
> come up with ideas, and if the list eventually becomes empty (or gets 
> to the point where it's 2 rare things that aren't important enough to 
> keep extra complexity in the language), then the whole notion of 
> special method lookup can finally go away. 

Well, maybe. I haven't decided whether special method lookup is an 
annoying inconsistency or a feature. It seems to me that operators, at 
least, are somewhat different from method calls, and the special 
handling should be considered a deliberate design feature. My 
reasoning goes like this:

Take a method call:

    obj.spam(arg)

That's clearly called on the instance itself (obj), and syntactically we 
can see that the method call "belongs" to obj, and so semantically we 
should give obj a chance to use its own custom method before the method 
defined in the class.

(At least in languages like Python where this feature is supported. I 
believe that the "Design Patterns" world actually gives this idiom a 
name, and in Java you have to jump through flaming hoops to make it 
work, but I can never remember what it is called.)

So it makes sense that ordinary methods should be first looked up on the 
instance, then the class. But now take an operator call:

    obj + foo

(where, for simplicity, both obj and foo are instances of the same 
class). Syntactically, that's *not* clearly a call on a specific 
instance. There's no good reason to think that the + operator "belongs" 
to obj, or foo, or either of them.

We could nevertheless arbitrarily decide that the left hand instance obj 
gets first crack at this, and if it doesn't customise the call to 
__add__, the right hand instance foo is allowed to customise __radd__, 
and if that doesn't exist we go back to the class __add__ (and if that 
also fails to exist we try the class __radd__). That's okay, I guess, 
but the added complexity feels like it would be a bug magnet.

So *maybe* we should decide that the right design is to say that 
operators don't belong to either instance, they belong to the class, and 
neither obj nor foo get the chance to override __[r]add__ on a 
per-instance basis.

Similar reasoning applies to __getattr__ and other special methods. We 
can, I think, convince ourselves that they belong to the class, not the 
instance, in a way that ordinary methods are not.

We define ordinary methods in the class definition for convenience and 
efficency, but syntactically and semantically obj.spam looks up spam in 
the instance obj first, so obj has a chance to override any spam defined 
on the class. That's a good thing.

But syntactically, special dunder methods like __getattr__ and __add__ 
don't *clearly* belong to the instance, so maybe we should decide that 
it is a positive feature that they can't be overridden by the instance. 
By analogy, if we use an unbound method instead:

TheClass.spam(obj, arg)

then I think we would all agree that the per-instance obj.spam() method 
should *not* be called. It seems to me that operator methods "feel like" 
they are closer to unbound method calls than bound method calls. And 
likewise for __getattr__ etc.

As I said, I haven't decided myself which way I lean. If Python didn't 
already support per-instance method overriding, I would argue strongly 
that it should. But per-instance operator overloading? I'm undecided.


-- 
Steve