[Python-Dev] Intricacies of calling __eq__

[Maciej Fijalkowski]

On Tue, Mar 18, 2014 at 11:35 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 18 March 2014 17:52, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> Hi
>>
>> I have a question about calling __eq__ in some cases.
>>
>> We're thinking about doing an optimization where say:
>>
>> if x in d:
>>    return d[x]
>>
>> where d is a dict would result in only one dict lookup (the second one
>> being constant folded away). The question is whether it's ok to do it,
>> despite the fact that it changes the semantics on how many times
>> __eq__ is called on x.
>
> I'll assume the following hold:
>
> - we're only talking about true builtin dicts (the similarity between
> __contains__ and __getitem__ can't be assumed otherwise)

yes

> - guards will trigger if d is mutated (e.g. by another thread) between
> the containment check and the item retrieval

yes

>
> Semantically, what you propose is roughly equivalent to reinterpreting
> the look-before-you-leap version to the exception handling based
> fallback:
>
>     try:
>         return d[x]
>     except KeyError:
>         pass
>
> For a builtin dict and any *reasonable* x, those two operations will
> behave the same way. Differences arise only if x.__hash__ or x.__eq__
> is defined in a way that most people would consider unreasonable.
>
> For an optimisation that actually changes the language semantics like
> that, though, I would expect it to be buying a significant payoff in
> speed, especially given that most cases where the key lookup is known
> to be a bottleneck can already be optimised by hand.
>
> Cheers,
> Nick.

the payoff is significant. Note that __eq__ might not be called at all
(since dicts check identity first). It turns out not all people write
reasonable code and we can't expect them to micro-optimize by hand. It
also covers cases that are hard to optimize, like:

if d[x] > 3:
   d[x] += 1

etc.