If you read the language reference for augmented arithmetic assignment, you will note that it essentially says, "call __i<op>__, and if that doesn't work call as if you were doing a <op> b". Unfortunately it appears **= does not follow the rule of falling back on the binary arithmetic expression semantics. I have a GitHub gist with demonstration code that shows this happening in both 3.8 and master (https://gist.github.com/brettcannon/fec4152857e0ed551b4da515dc63e580
). This was reported in https://bugs.python.org/issue38302
, although initially it didn't cover __pow__, only __rpow__ being skipped.
Wow, very subtle bug. (Note that the issue was initially raised on StackOverflow.)
Yeah, I independently ran into it while writing a future blog post and it took me a bit to realize that it wasn't me. 😄
Ugh, for some reason the public C API PyNumber_InPlacePower() takes three arguments, like pow(x, y, z) (which computes x**y % z), even though there is no way to invoke it like that from Python. I'm sure this was set in stone when augmented assignments were first introduced -- long before we even had type slots. But because of this someone (me?) probably was being lazy and thought that implementing the full fallback strategy for **= was more effort than it was worth. (I don't think I have ever in my life used `**=`. :-)
I suspect very few have, hence how this has managed to exist as a bug for as long as it has.
I think there are two options to fixing this:
1. Add a note to the data model that **= is special and does not fall back (obviously the most backwards-compatible)
I think we ought to do this for 3.8 and 3.9 -- it's too late to change in 3.9.0.
Yes, I agree. Sorry for forgetting to mention I was assuming these options were targeting for 3.10.
2. Fix **= (which makes sense from a language consistency perspective)
We should do this in 3.10.
BTW I don't have the bandwidth to review the PR that is attached to the issue (although it looks like Serhiy has reviewed it; are you up for meringing it, Serhiy?).
Personally, my vote is for #2 as I don't want to have to remember that **= is somehow special compared to all other augmented assignments. I also don't think the backwards-compatibility risk is at all large since the semantics of turning `a **= b` into `a = a ** b` shouldn't really be different.
But it is enough to give me pause about doing this in bugfix releases.
Yeah, I agree that I don't think we can do this as a bugfix because it is such a shift in semantics.
P.S. Why are some of the PyNumber_InPlace*() functions handwritten while others are defined using a macro which mirrors the handwritten ones? Just something I noticed while investigating this.
Is this about some using INPLACE_BINOP and others not using it? I can't tell the difference for InPlaceFloorDivide and -TrueDivide, possibly because these were added at a later time? git blame show that the INPLACE_BINOP macro was introduced by Neil Schemenauer in 2001 for PEP 208 (Reworking the Coercion Model, by Neil and MAL). We didn't have truediv and floordiv then, and I guess when they were added later the same year, for PEP 238 (Changing the Division Operator, by Moshe Zadka and myself) we did it differently. FWIW the in-place power glitch also originated in the PEP 208 commit.