On Sun, Aug 23, 2020 at 1:09 PM Brett Cannon <brett@python.org> wrote:

If you read the language reference for augmented arithmetic assignment, you will note that it essentially says, "call __i<op>__, and if that doesn't work call as if you were doing a <op> b". Unfortunately it appears **= does not follow the rule of falling back on the binary arithmetic expression semantics. I have a GitHub gist with demonstration code that shows this happening in both 3.8 and master (https://gist.github.com/brettcannon/fec4152857e0ed551b4da515dc63e580). This was reported in https://bugs.python.org/issue38302, although initially it didn't cover __pow__, only __rpow__ being skipped.

Wow, very subtle bug. (Note that the issue was initially raised on StackOverflow.)

 

This appears to happen because the opcode  for **= calls PyNumber_InPlacePower() (https://github.com/python/cpython/blob/802726acf6048338394a6a4750835c2cdd6a947b/Objects/abstract.c#L1159) which calls ternary_op for __ipow__ or __pow__ depending on which is defined, but will never try both (https://github.com/python/cpython/blob/802726acf6048338394a6a4750835c2cdd6a947b/Objects/abstract.c#L849). All of the other augmented arithmetic assignment operators have a special binary_iop() function to call which takes care of the fallback logic, so no other augmented arithmetic assignments appear to have this problem (I tested them all regardless).

Ugh, for some reason the public C API PyNumber_InPlacePower() takes three arguments, like pow(x, y, z) (which computes x**y %  z), even though there is no way to invoke it like that from Python. I'm sure this was set in stone when augmented assignments were first introduced -- long before we even had type slots. But because of this someone (me?) probably was being lazy and thought that implementing the full fallback strategy for **= was more effort than it was worth. (I don't think I have ever in my life used `**=`. :-)

 

I think there are two options to fixing this:

1. Add a note to the data model that **= is special and does not fall back (obviously the most backwards-compatible)

I think we ought to do this for 3.8 and 3.9 -- it's too late to change in 3.9.0.

 

2. Fix **= (which makes sense from a language consistency perspective)

We should do this in 3.10.

 

Personally, my vote is for #2 as I don't want to have to remember that **= is somehow special compared to all other augmented assignments. I also don't think the backwards-compatibility risk is at all large since the semantics of turning `a **= b` into `a = a ** b` shouldn't really be different.

But it is enough to give me pause about doing this in bugfix releases.

 

P.S. Why are some of the PyNumber_InPlace*() functions handwritten while others are defined using a macro which mirrors the handwritten ones? Just something I noticed while investigating this.

Is this about some using INPLACE_BINOP and others not using it? I can't tell the difference for InPlaceFloorDivide and -TrueDivide, possibly because these were added at a later time? git blame show that the INPLACE_BINOP macro was introduced by Neil Schemenauer in 2001 for PEP 208 (Reworking the Coercion Model, by Neil and MAL). We didn't have truediv and floordiv then, and I guess when they were added later the same year, for PEP 238 (Changing the Division Operator, by Moshe Zadka and myself) we did it differently. FWIW the in-place power glitch also originated in the PEP 208 commit.

--

--Guido van Rossum (python.org/~guido)

Pronouns: he/him (why is my pronoun here?)