[PYTHON MATRIX-SIG] Final matrix object renaming and packaging

[James Hugunin]

   From: "Jim Fulton, U.S. Geological Survey" <jfulton@usgs.gov>

   > 2) One C module that must be statically linked called "multiarraymodule.c"

   By statically linked, I assume you mean statically linked into the interpreter.
   Right?

   > Having this particular module statically linked will eliminate the
   > need for getting the CObject proposal working before release.

   As Guido said, the CObject proposal is working.  I'll send it to you in a
   separate note.

   I finished the (very small) CObject implementation very soon after the
   workshop because I feel it is important to use it for the Matrix (Numeric)
   module.

   I feel strongly that the Matrix module should export it's C interface
   using CObjects so that modules using matrices to not require that any
   of the matrix software be statically linked.  In my distribution of
   Python, I statically link as little as possible to keep the
   interpreter and the interpreter start-up time small.  I
   plan to modify FIDL to use the CObject-exported interface.

   I'd be happy to assist you with this if you wish.

I guess we run on faster networks at MIT, I never bother with dynamic
linking, and find that 4MB binaries launch as fast as I wish.  I'd be
more than happy to have somebody else implement the CObject interface,
but I just don't see the time it would take (including getting myself
up to speed on the vagaries of dynamic linking) is worthwhile.  If you
(Jim Fulton) want to try to add this interface, I'd be happy to help.

   >
   > Use "PyArray_" as the name of the Matrix Object.  This is a simple
   > renaming of the existing "PyMatrix_".
   >
   > Use "array(sequence, typecode='d')" as the default
   > constructor for this new C type.

   Is this a replacement for the existing array type?

I decided it was not worth the huge set of compromises that would have
been necessary to make the matrix/array object truly compatible with
the existing array object.  Still, the right name for this object
really is array.

There's no problem with using PyArray as the C name for the object
because the existing array object does not export an interface.  Also,
there's no problem with using the name "array" as a constructor
because avoiding these sorts of naming conflicts is why python has a
module system in the first place.  No existing code that imports
"arraymodule" will be broken, but hopefully people in the future will
start using the new multiarraymodule for the same tasks.

   > 4) Two python objects, "Array.py" and "Matrix.py"

   Are these imported by Numeric, or would the user be importing
these?

I plan to have everything in the basic distribution imported into a
flat name space under the Numeric module.  However, there's nothing to
stop people from importing these independently if they wish.

   Is the current built-in array module going away?  If not, then there
   is a name conflict on case-insensitive file systems.  

The array module is not going away.  I'm confused about the problem of
case-insensitive file systems, what do they do with tkintermodule and
TkInter?  If this is in fact an issue, then "Array.py" can be changed
to "UserArray.py" in the spirit of UserList, etc.  

	I like the idea
   of having the user import Numeric and then use the
   Numeric.Matrix_d(...) or Numeric.Array_f(...) rather than than
   importing Matrix and Array.  Is there any reason for the user to
   import Array directly?  I am strongly opposed to using "Array" unless
   the "array" module goes away.

I agree with all this (except the last line, which I'm willing to
conceed after you explain my tkinter question).

   > In order to support these python objects (and others like them), two
   > special data members will be added, "__array__", and "__object__".  If
   > an object has the member "__array__", then the C functions that handle
   > matrices will attempt to retrieve the matrix from this member when
   > passed in a python object.

   Are we taking about python members or C structure members?  Is the
   __array__ member supposed to be the C pointer to a block of memory?

The __array__ member is a member of a python object which is expected
to contain a python object of type array (the type created in C that
this whole thing is based on).

   > In addition, they will attempt to convert
   > their result to an object of class "__object__" upon return.

   Class __object__?  So __object__ is a pointer to a Python class
   object?

This is still a python member.  In python what it would do is call
m.__object__(new_array).  I assume that a similar thing can be done in C
(I haven't implemented this in C yet).

   > This
   > means that umath.sin(Array([0, pi/2, pi])) == Array([0.,1.,0.]).

   OK.  This makes sense

Remember that Array is a python object here, that's the trick I'm
trying to make work out.

   > Hopefully, this convention will allow these python objects to coexist
   > well with any numeric libraries.

   Could you provide some additional details?

Here's a bit of code for a unary function expecting a single PyArray
argument of type "double" of two dimensions:

	PyObject *op;	
	PyArrayObject *ap, *rp;

	TRY(PyArg_ParseTuple(args, "O", &op));
	TRY(ap = PyArray_ContiguousFromObject(op, PyArray_DOUBLE, 2, 2));

	// Do something with ap to get rp

	Py_DECREF(ap);

	return PyArray_Return(rp, op);

With the exception of the second argument to PyArray_Return, this is
the current way of writing such a chunk of code.

PyArray_ContiguousFromObject will convert any python sequence type to
an array of the appropriate type and dimensions if possible.  If the
argument is already an array of the appropriate type and dimensions,
then that array will be increfed and returned (unless its data points
to a discontiguous chunk of memory in which case it will be copied
into a new array with contiguous memory).  

The new feature that I want to add to this function is that if its
argument is a python object with the attribute "__array__", then this
function wil return the PyArrayObject contained in that attribute (if
this is indeed the case).

PyArray_Return is used because some operations wind up producing a
0-dimensional array.  These will be converted to the appropriate
python scalars on return.

The new feature that I want to add here is that if the second argument
has a "__class__" attribute, then the constructor for that class will
be used to return a new python object with the returned PyArrayObject
in its "__array__" attribute.

This is the simplest method I could come up with to get my
"sin(Array())" example to work.

   >
   > 6) Great documentation and tutorials (hopefully written by Paul
   > DuBois).

   Wat cool.  Will we also get doc strings?

doc strings are already done (probably could use some polishing, but...

-Jim

=================
MATRIX-SIG  - SIG on Matrix Math for Python

send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================