[Python-Dev] Python-versus-CPython question for __mul__ dispatch

[Nick Coghlan]

On 15 May 2015 at 10:45, Nathaniel Smith <njs at pobox.com> wrote:
> Hi all,
>
> While attempting to clean up some of the more squamous aspects of
> numpy's operator dispatch code [1][2], I've encountered a situation
> where the semantics we want and are using are possible according to
> CPython-the-interpreter, but AFAICT ought not to be possible according
> to Python-the-language, i.e., it's not clear to me whether it's
> possible even in principle to implement an object that works the way
> numpy.ndarray does in any other interpreter. Which makes me a bit
> nervous, so I wanted to check if there was any ruling on this.

It's a known CPython operand precedence bug due to the fact several of
the builtin types only implement sq_concat & sq_repeat without
implementing nb_add & nb_mul: http://bugs.python.org/issue11477

There's then a related problem where we *don't* process
"NotImplemented" results from sq_concat and sq_repeat properly, so all
the builtin sequence types throw TypeError directly, instead of
returning NotImplemented when they don't recognise the other type.

I wrote a preliminary patch attempting to fix it a few years back
after the issue was discovered by Mike Bayer and Alex Gaynor when
porting SQL Alchemy to PyPy, but never committed it because my own
verdict on the approach I used was that it rendered the abstract
object API implementation for __mul__ and __add__ utterly
unmaintainable.

The better fix would be to make defining sq_concat and sq_repeat more
like defining __add__ and __mul__ at the Python level: PyType_Ready
should implicitly fill in nb_add and nb_mul references to standard
implementations that delegate to sq_concat and sq_repeat, and we
should update the implementations of the latter for the standard
library sequence types implemented in C to return NotImplemented
rather than throwing TypeError directly.

However, my intermittent attempts to get anyone else interested in
fixing it haven't borne any fruit, and I've prioritised other projects
over coming up with a different patch myself.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia