[Python-ideas] Eliminating special method lookup (was Re: Missing Core Feature: + - * / | & do not call __getattr__)

[Andrew Barnert]

On Friday, December 4, 2015 11:05 AM, Brendan Barnwell <brenbarn at brenbarn.net> wrote:
> 
>    This is 
> (https://docs.python.org/3/reference/datamodel.html#special-method-lookup) 
> documented but it is often surprising to new users.  Personally I find 
> it an annoying inconsistency and hope that at some time in the future 
> Python will become fast enough overall that the extra overhead will be 
> acceptable.

Unless it's changed since I last looked, Python doesn't actually define _which_ methods are looked up this way; it just allows implementations to do it for any subset of (presumably) the methods defined in the Data Model chapter.

I don't think any of the major implementations besides CPython need this optimization at all (a JIT is generally going to do the lookup once and cache it, right?), but at least PyPy follows CPython exactly anyway, to avoid any possible compatibility problems.

Meanwhile, there might not be one solution once and for all. For example, __add__ has to switch on the other object's type, unbox the values, malloc and box up a result, etc.; __bool__ often just has to check a struct member != 0 and return the constant True or False, so reducing the overhead to 10% on __add__ may still mean 105% on __bool__. From another angle, the __add__ lookup is part of a larger thing that includes __radd__ lookup and comparing types and so on, and that process might end up with some way to optimize lookup of the methods that wouldn't apply to unary operators or non-operator methods.

So, maybe the way to get from here to there is to explicitly document the methods CPython treats as magic methods, and allow only allow other implementations to do the same for (a subset of) the same, and people can gradually tackle and remove parts of that list as people come up with ideas, and if the list eventually becomes empty (or gets to the point where it's 2 rare things that aren't important enough to keep extra complexity in the language), then the whole notion of special method lookup can finally go away.