[Python-Dev] Python-versus-CPython question for __mul__ dispatch

[Guido van Rossum]

I expect you can make something that behaves like list by defining __mul__
and __rmul__ and returning NotImplemented.

On Thursday, May 14, 2015, Stefan Richthofer <Stefan.Richthofer at gmx.de>
wrote:

> >>Should I be worried?
>
> You mean should *I* be worried ;)
>
> Stuff like this is highly relevant for JyNI, so thanks very much for
> clarifying this
> subtle behavior. It went onto my todo-list right now to ensure that JyNI
> will emulate
> this behavior as soon as I am done with gc-support. (Hopefully it will be
> feasible,
> but I can only tell in half a year or so since there are currently other
> priorities.)
> Still, this "essay" potentially will save me a lot of time.
>
> So, everybody please feel encouraged to post things like this as they come
> up. Maybe
> there could be kind of a pitfalls-page somewhere in the docs collecting
> these things.
>
> Best
>
> Stefan
>
>
> > Gesendet: Freitag, 15. Mai 2015 um 02:45 Uhr
> > Von: "Nathaniel Smith" <njs at pobox.com <javascript:;>>
> > An: "Python Dev" <python-dev at python.org <javascript:;>>
> > Betreff: [Python-Dev] Python-versus-CPython question for __mul__ dispatch
> >
> > Hi all,
> >
> > While attempting to clean up some of the more squamous aspects of
> > numpy's operator dispatch code [1][2], I've encountered a situation
> > where the semantics we want and are using are possible according to
> > CPython-the-interpreter, but AFAICT ought not to be possible according
> > to Python-the-language, i.e., it's not clear to me whether it's
> > possible even in principle to implement an object that works the way
> > numpy.ndarray does in any other interpreter. Which makes me a bit
> > nervous, so I wanted to check if there was any ruling on this.
> >
> > Specifically, the quirk we are relying on is this: in CPython, if you do
> >
> >   [1, 2] * my_object
> >
> > then my_object's __rmul__ gets called *before* list.__mul__,
> > *regardless* of the inheritance relationship between list and
> > type(my_object). This occurs as a side-effect of the weirdness
> > involved in having both tp_as_number->nb_multiply and
> > tp_as_sequence->sq_repeat in the C API -- when evaluating "a * b",
> > CPython tries a's nb_multiply, then b's nb_multiply, then a's
> > sq_repeat, then b's sq_repeat. Since list has an sq_repeat but not an
> > nb_multiply, this means that my_object's nb_multiply gets called
> > before any list method.
> >
> > Here's an example demonstrating how weird this is. list.__mul__ wants
> > an integer, and by "integer" it means "any object with an __index__
> > method". So here's a class that list is happy to be multiplied by --
> > according to the ordinary rules for operator dispatch, in the example
> > below Indexable.__mul__ and __rmul__ shouldn't even get a look-in:
> >
> > In [3]: class Indexable(object):
> >    ...:     def __index__(self):
> >    ...:         return 2
> >    ...:
> >
> > In [4]: [1, 2] * Indexable()
> > Out[4]: [1, 2, 1, 2]
> >
> > But, if I add an __rmul__ method, then this actually wins:
> >
> > In [6]: class IndexableWithMul(object):
> >    ...:     def __index__(self):
> >    ...:         return 2
> >   ...:     def __mul__(self, other):
> >    ...:         return "indexable forward mul"
> >    ...:     def __rmul__(self, other):
> >    ...:         return "indexable reverse mul"
> >
> > In [7]: [1, 2] * IndexableWithMul()
> > Out[7]: 'indexable reverse mul'
> >
> > In [8]: IndexableWithMul() * [1, 2]
> > Out[8]: 'indexable forward mul'
> >
> > NumPy arrays, of course, correctly define both __index__ method (which
> > raises an array on general arrays but coerces to int for arrays that
> > contain exactly 1 integer), and also defines an nb_multiply slot which
> > accepts lists and performs elementwise multiplication:
> >
> > In [9]: [1, 2] * np.array(2)
> > Out[9]: array([2, 4])
> >
> > And that's all great! Just what we want. But the only reason this is
> > possible, AFAICT, is that CPython 'list' is a weird type with
> > undocumented behaviour that you can't actually define using pure
> > Python code.
> >
> > Should I be worried?
> >
> > -n
> >
> > [1] https://github.com/numpy/numpy/pull/5864
> > [2] https://github.com/numpy/numpy/issues/5844
> >
> > --
> > Nathaniel J. Smith -- http://vorpus.org
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org <javascript:;>
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de
> >
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org <javascript:;>
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (on iPad)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150514/7a2f76bf/attachment.html>