Re: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power

18 Mar 2014

      On 18 Mar 2014 22:18, "M.-A. Lemburg"  wrote:
...
On 18.03.2014 12:27, Nick Coghlan wrote:
...
On 18 March 2014 20:47, Robert Kern  wrote:
...
On 2014-03-18 08:02, Nick Coghlan wrote:
...
...
operator.matmul and PyObject_MatrixMultiply are obvious enough, but
I'm afraid I'm not too clear on the tradeoffs about adding a C
level
...
...
...
...
...
type slot, or even entirely sure what the alternative is. (I guess
I
just assumed that all special methods used C level type slots and
there was nothing to think about.) Do you (or anyone) have any
thoughts?
I suspect you're going to want one, as without it, the implementation
method
ends up in the class dict instead (the context management protocol
works
that way).
I suspect the design we will want is a new struct for Py_Matrix slots
(akin to
those for numbers, etc). The alternative would be to just add more
"Number"
slots, but that isn't really accurate.
So, here's the change to PyHeapType object that makes the most sense
to me (assuming both "@" and "@@" are added - drop the new "power"
methods if "@@" is dropped from the PEP):
- add "PyMatrixMethods as_matrix;" as a new field in PyHeapTypeObject
- define PyMatrixMethods as:
typedef struct {
        binaryfunc mt_multiply;
        binaryfunc mt_power;
        binaryfunc mt_inplace_multiply;
        binaryfunc mt_inplace_power;
   } PyMatrixMethods;
This approach increases the size of all type objects by one pointer.
The other way to do it would be to just add four new slots to
PyNumberMethods:
binaryfunc nb_matrix_multiply;
        binaryfunc nb_matrix_power;
        binaryfunc nb_inplace_matrix_multiply;
        binaryfunc nb_inplace_matrix_power;
This approach increases the size of all type objects that define one
or more of the numeric functions by four pointers, and doesn't really
make sense at a conceptual level. The latter is the main reason I
prefer the separate PyMatrixMethods struct.
I don't think that it's a good idea to make all type objects
larger just to address matrix multiplications which none of the
builtin types will actually use, so +1 on adding to the number
methods, even if a matrix isn't a number (they still use numbers,
so it's not that far off :-)), but -1 on adding another slot
struct.
Aside: Mechanisms such as namedtuple add lots of new type
objects to the runtime environment, so it's no longer safe to
assume that the size of type objects doesn't really matter much
in real-life applications.
So, here's the problem: operand precedence for sequences implemented in C
is currently broken in certain relatively obscure cases (and has been since
forever - we only found out about due to an SQL Alchemy failure when
porting to PyPy revealed PyPy was right and CPython was wrong). This is why
returning NotImplemented from sq_concat and sq_repeat doesn't work right.
The problem doesn't arise for sequences implemented in Python, because we
always populate the nb_add and nb_multiply slots for those.

I've tried fixing that directly, and it turned abstract.c into an
unmaintainable mess, so I abandoned that approach. My current preferred
solution is now similar to the one we use from Python: drop the direct
calls to sq_concat and sq_repeat from abstract.c, and instead automatically
add nb_add and nb_multiply implementations during type creation that
delegate dynamically to the sequence slots.

So a *lot* of types already have their PyNumberMethods slot allocated
(including sequence types implemented in Python), and I'd like to expand
that to include all sequence types, even those implemented in C.

That makes the space trade-off here substantially less clear, particularly
if matrix power ends up being added in the future (it has been dropped from
the current PEP).

I accept that adding the new slots to PyNumberMethods is more conceptually
coherent than I thought, though. There's also the fact that we already mess
about populating different slots for pragmatic reasons that have nothing to
do with conceptual integrity :)

Cheers,
Nick.
...
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source  (#1, Mar 18 2014)
...
...
...
Python Projects, Consulting and Support ...   http://www.egenix.com/
mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/

2014-03-29: PythonCamp 2014, Cologne, Germany ...          11 days to go
2014-04-09: PyCon 2014, Montreal, Canada ...               22 days to go
::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/