[Matrix-SIG] Numeric Nits

[Rick White]

On Tue, 22 Jun 1999, David Ascher wrote:

>> The question is, is there a way for Numeric to determine that one of
>> operands is a temporary?  It looks to us as if the reference count for
>> a temporary result (which will be discarded after the operation) is
>> smaller than for a variable operand.  If so, it should be possible for
>> Numeric to reuse the temporary for the output in cases where the
>> temporary is the same type and shape as the output.  Maybe there is
>> something tricky here that we don't understand, but if not it seems like
>> a fairly simple change would make Numeric a lot less memory-hungry.
>
>This is an issue I've thought about a little, but alas I believe the
>solution requires experience in fields I'm got a provably bad record in,
>such as parsing, code tree manipulations, etc. so thinking is all I'll do
>on the topic.
>
>The problem is that there is no way to have NumPy do the detection you
>mention above because the assignment process is one that Python does not
>give any 'hooks' into (yet).

It appears to me it might be a lot simpler than this, though my
understanding of the Python/Numeric code is not very deep so I certainly
could be wrong.  For example, here is the Python C code to do a multiply
operation:

   case BINARY_MULTIPLY:
      w = POP();
      v = POP();
      x = PyNumber_Multiply(v, w);
      Py_DECREF(v);
      Py_DECREF(w);
      PUSH(x);
      if (x != NULL) continue;
      break;

Presumably if v and/or w are Numeric arrays, PyNumber_Multiply
eventually calls the Numeric multiply routine.  In the current Numeric
implementation, a new Numeric array is allocated, filled with the
product, and returned for assignment to x.  Every operation results in
the creation of another temporary array.

Suppose v is the intermediate result of another expression.  Then its
reference count will be 1, so that after returning from the multiply
routine its memory will be released by the DECREF.  Couldn't the
Numeric multiply routine check the reference count of v and, if it is 1
and if the datatype and size of v are appropriate, put the result into
v instead of into a new array?  Then the ref count for v would be
incremented and v would be returned as the function result.

This doesn't require any knowledge about the parser or the context
in which the expression is being evaluated.  All that is required
is the ability to modify one of the input arrays and return it
as the result.  I figure the only change to Numeric would be to
check the reference counts, types, and sizes of the input arguments
at the point when the result array is about to be allocated,
and to reuse one of the input arrays if possible.

As I said, maybe I'm being naive in my assumptions -- is there
some reason this would not work?