[Python-ideas] Eliminating special method lookup (was Re: Missing Core Feature: + - * / | & do not call __getattr__)

[Guido van Rossum]

I distinctly remember that by the time I committed that code I had seen
numerous examples (perhaps in proprietary code at Zope) of attempts to use
per-instance overloading going wrong.

I believe what was happening was that users were trying to overload
operators using instance attributes named after operators, and were trying
to come up with clever tricks to ensure that the instance argument was
passed to the method (there use case was not special cases like Infinities,
they had code that needed access to both arguments). When I saw the
resulting ugly code I figured there would be little loss if we just not
allowed the practice.

That's really all I can recall. I don't think there's a need to make
another U-turn on this topic. Regarding the docs we may still have to add
language to make it clear that the behavior you describe for Jython is
wrong. (I also believe there was a request earlier in this thread to
clarify exactly which methods are not looked up on the instance.)

On Sat, Dec 5, 2015 at 5:56 PM, Andrew Barnert <abarnert at yahoo.com> wrote:

> On Dec 5, 2015, at 09:30, Guido van Rossum <guido at python.org> wrote:
> >
> > I wish there was less armchair language design and post-rationalization
> on this topic, and more research into *why* we actually changed this.
>
> I think the documentation already describes the reasoning just fine:
>
> 1. Performance. It "provides significant scope for speed optimizations
> within the interpreter, at the cost of some flexibility in the handling of
> special methods (the special method must be set on the class object itself
> in order to be consistently invoked by the interpreter)."
>
> 2. As a solution to the metaclass problem with the handful of methods
> implemented by both type and normal types--e.g., you don't want hash(int)
> to call int.__hash__, but type.__hash__. (This one obviously doesn't affect
> most operators; you can't generally add or xor types--but List[int] shows
> why that can't be ignored.)
>
> Anyway, I had already looked at the relevant PEPs, what's new, descrintro,
> and the old PLAN.txt. But, at your suggestion, I decided to take a look
> back at the commit comments and Python-dev archives as well, to see if
> there is some other reason for the change beyond those mentioned in the
> docs.
>
> In fact, there was almost no discussion at all. The descr branch was
> merged to trunk on 2 August 2001, nobody mentioned either problem, and then
> on 28 Aug you committed #19535 and #19538.
>
> The performance issue was raised in descrintro and the PEP, and a brief
> early thread, but it seems like everyone was happy to wait until you
> declared it done before griping about performance problems that may or may
> not result. Everyone was happy with 2.2a4 (which was after the change), so
> nobody had a reason to complain.
>
> As far as I can see, nobody noticed the hash(int) issue before you found
> and fixed it. It wasn't mentioned in PLAN.txt, any mailing list threads,
> etc. It apparently didn't even come up after the fact (except in a brief
> aside about PEP 266) until much later, when people got used to using the
> new features in practice and started asking why they can't do instance
> overrides for some special methods (at which point the answer was already
> in the docs).
>
> Unless I'm missing something (which is certainly possible), or you can
> remember your reasoning 14 years ago, I think what the docs say is pretty
> much all there is to say.
>
> In particular, do you actually have a reason that spam + eggs shouldn't
> look at spam.__dict__['__add__'] other than for performance and for
> consistency with the hash(int) solution?
>
> > I recall we did not take the decision lightly. Surely correlating the
> source control logs and the mailing list archives can shed more insight on
> this topic than thought experiments about classes with five special values.
> :-)
>
> >
> > Patches to the reference manual are also very welcome, BTW. We're not
> proud of the imprecision of much of the language there -- I would love for
> some language lawyers to help tighten the language (not just on this topic
> but all over the reference manual).
>
>
> > Finally. Regardless of what the reference manual may (not) say, other
> implementations do *not* have the freedom to change the operator lookup to
> look in the instance dict first.
>
> IIRC, Jython 2.2 looks at the instance first unless the type is a Java
> class, and nobody ever complained (in fact, I remember someone complaining
> about CPython breaking his code that worked in Jython...).
>
> But it would certainly be easier to tighten up the language in 3.3.9 if it
> applies to all Python implementations.
>
> > (However, giving the metaclass more control is not unreasonable. There
> also seems to be some interesting behavior here related to slots.)
>
>
> I'm not sure what you're suggesting here. That implementations can let a
> metaclass __getattribute__ hook special method lookup, but some
> implementations (including CPython 3.6) won't do so?




-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151207/ea796cce/attachment.html>