[Matrix-SIG] Numeric Nits

[Chase, Christopher J (Chris), ALNTK]

> -----Original Message-----
> From: rlw@stsci.edu [mailto:rlw@stsci.edu]
> Sent: Tuesday, June 22, 1999 1:58 PM
> To: da@ski.org
> Cc: barrett@stsci.edu; matrix-sig@python.org; perry@stsci.edu;
> rlw@stsci.edu
> Subject: Re: [Matrix-SIG] Numeric Nits
> 
> 
> On Tue, 22 Jun 1999, David Ascher wrote:
> 
> >> The question is, is there a way for Numeric to determine 
> that one of
> >> operands is a temporary?  It looks to us as if the 
> reference count for
> >> a temporary result (which will be discarded after the operation) is
> >> smaller than for a variable operand.  If so, it should be 
> possible for
> >> Numeric to reuse the temporary for the output in cases where the
> >> temporary is the same type and shape as the output.  Maybe there is
> >> something tricky here that we don't understand, but if not 
> it seems like
> >> a fairly simple change would make Numeric a lot less memory-hungry.
> >
> >This is an issue I've thought about a little, but alas I believe the
> >solution requires experience in fields I'm got a provably 
> bad record in,
> >such as parsing, code tree manipulations, etc. so thinking 
> is all I'll do
> >on the topic.
> >
> >The problem is that there is no way to have NumPy do the 
> detection you
> >mention above because the assignment process is one that 
> Python does not
> >give any 'hooks' into (yet).
> 
> It appears to me it might be a lot simpler than this, though my
> understanding of the Python/Numeric code is not very deep so 
> I certainly
> could be wrong.  For example, here is the Python C code to do 
> a multiply
> operation:
> 
>    case BINARY_MULTIPLY:
>       w = POP();
>       v = POP();
>       x = PyNumber_Multiply(v, w);
>       Py_DECREF(v);
>       Py_DECREF(w);
>       PUSH(x);
>       if (x != NULL) continue;
>       break;
> 
> Presumably if v and/or w are Numeric arrays, PyNumber_Multiply
> eventually calls the Numeric multiply routine.  In the current Numeric
> implementation, a new Numeric array is allocated, filled with the
> product, and returned for assignment to x.  Every operation results in
> the creation of another temporary array.
> 
> Suppose v is the intermediate result of another expression.  Then its
> reference count will be 1, so that after returning from the multiply
> routine its memory will be released by the DECREF.  Couldn't the
> Numeric multiply routine check the reference count of v and, 
> if it is 1
> and if the datatype and size of v are appropriate, put the result into
> v instead of into a new array?  Then the ref count for v would be
> incremented and v would be returned as the function result.
> 

I was thinking of the same approach.  

The problem is that the caller owns the reference.  The numeric multiply
routine has no idea who the caller is and what the caller will do with the
reference, e.g. whether the caller is going to do a DECREF on return.  Also
the caller is not guaranteed to be the interpreter.  Rather it could be a
program using the C API.

One possible solution is if python objects could support an TEMPORARY
attribute perhaps in the basic object_HEAD structure.  Then a caller that
will release an object after a call can mark an object as TEMPORARY, meaning
"I (the caller) will not access this object and delete my reference to the
object after the call)".  Another meaning for TEMPORARY might be "Pending
DECREF on return".

A called function receiving an object as TEMPORARY and a reference count of
one is allowed to overwrite or reuse the object.

This could work but is not a pretty nor elegant approach.  But I don't know
if there is a better approach given the caller owns reference approach of
Python.


Chris
 

> This doesn't require any knowledge about the parser or the context
> in which the expression is being evaluated.  All that is required
> is the ability to modify one of the input arrays and return it
> as the result.  I figure the only change to Numeric would be to
> check the reference counts, types, and sizes of the input arguments
> at the point when the result array is about to be allocated,
> and to reuse one of the input arrays if possible.
> 
> As I said, maybe I'm being naive in my assumptions -- is there
> some reason this would not work?
> 
> _______________________________________________
> Matrix-SIG maillist  -  Matrix-SIG@python.org
> http://www.python.org/mailman/listinfo/matrix-sig
>