[Python-ideas] Eliminating special method lookup (was Re: Missing Core Feature: + - * / | & do not call __getattr__)

[Andrew Barnert]

On Dec 5, 2015, at 09:30, Guido van Rossum <guido at python.org> wrote:
> 
> I wish there was less armchair language design and post-rationalization on this topic, and more research into *why* we actually changed this.

I think the documentation already describes the reasoning just fine:

1. Performance. It "provides significant scope for speed optimizations within the interpreter, at the cost of some flexibility in the handling of special methods (the special method must be set on the class object itself in order to be consistently invoked by the interpreter)."

2. As a solution to the metaclass problem with the handful of methods implemented by both type and normal types--e.g., you don't want hash(int) to call int.__hash__, but type.__hash__. (This one obviously doesn't affect most operators; you can't generally add or xor types--but List[int] shows why that can't be ignored.)

Anyway, I had already looked at the relevant PEPs, what's new, descrintro, and the old PLAN.txt. But, at your suggestion, I decided to take a look back at the commit comments and Python-dev archives as well, to see if there is some other reason for the change beyond those mentioned in the docs.

In fact, there was almost no discussion at all. The descr branch was merged to trunk on 2 August 2001, nobody mentioned either problem, and then on 28 Aug you committed #19535 and #19538.

The performance issue was raised in descrintro and the PEP, and a brief early thread, but it seems like everyone was happy to wait until you declared it done before griping about performance problems that may or may not result. Everyone was happy with 2.2a4 (which was after the change), so nobody had a reason to complain.

As far as I can see, nobody noticed the hash(int) issue before you found and fixed it. It wasn't mentioned in PLAN.txt, any mailing list threads, etc. It apparently didn't even come up after the fact (except in a brief aside about PEP 266) until much later, when people got used to using the new features in practice and started asking why they can't do instance overrides for some special methods (at which point the answer was already in the docs).

Unless I'm missing something (which is certainly possible), or you can remember your reasoning 14 years ago, I think what the docs say is pretty much all there is to say.

In particular, do you actually have a reason that spam + eggs shouldn't look at spam.__dict__['__add__'] other than for performance and for consistency with the hash(int) solution?

> I recall we did not take the decision lightly. Surely correlating the source control logs and the mailing list archives can shed more insight on this topic than thought experiments about classes with five special values. :-)

> 
> Patches to the reference manual are also very welcome, BTW. We're not proud of the imprecision of much of the language there -- I would love for some language lawyers to help tighten the language (not just on this topic but all over the reference manual).


> Finally. Regardless of what the reference manual may (not) say, other implementations do *not* have the freedom to change the operator lookup to look in the instance dict first.

IIRC, Jython 2.2 looks at the instance first unless the type is a Java class, and nobody ever complained (in fact, I remember someone complaining about CPython breaking his code that worked in Jython...).

But it would certainly be easier to tighten up the language in 3.3.9 if it applies to all Python implementations.

> (However, giving the metaclass more control is not unreasonable. There also seems to be some interesting behavior here related to slots.)


I'm not sure what you're suggesting here. That implementations can let a metaclass __getattribute__ hook special method lookup, but some implementations (including CPython 3.6) won't do so?