[PYTHON MATRIX-SIG] [munro@icf.llnl.gov: Re: Python Matrix extension tutorial]

P. Dubois dubois@kristen.llnl.gov
Fri, 10 Nov 1995 08:40:40 -0800

Dear Matrix-SIGers,
   Yesterday at LLNL Brian Yang taught a tutorial on the matrix extension
to about 20 people. I asked that people give me comments to forward to
the SIG. In general, we concluded that the extension does 
what we need it to do. Here are some of the comments for your consideration.
I will send them as separate messages so as to compartmentalize the discussion
a little. The first is from Dave Munro. Dave is a physicist who has developed
his own interpreted language for postprocessing called Yorick. The code this
is most often used to postprocess, Lasnex, is a Fortran program with lots of
arrays, so Yorick has facilities especially devoted to manipulating arrays
easily. Here is Dave's suggestion:
------- Start of forwarded message -------

In the handout at the meeting on page 7 is an example of the heavily
used Yorick feature I mentioned: You have two arrays

   b  with shape (2,3)
   c  with shape (2,2,3)

and you want to produce:

   d[i][j][k]= b[i][k] + c[i][j][k]

{The example would be much clearer if the ranks were given as (2,3)
and (2,4,3), so it only made sense one way.}  In Python, this common
operation is carried out by the expression:

   d= add[1,2](b,c)

Yorick has a special syntax for this same operation:

   d= b(,-,) + c

which I find much clearer; the "-" in an index list is a place holder
indicating that the result is to have an additional dimension in the
slot where the "-" was.  Thus b(,-,) is a rank-3 array which can be
directly compared with c.  With this notation, b(,,-)+c is the same as
b[j][k]+c[i][j][k] (the default), while b(,-,)+c produces
b[i][k]+c[i][j][k] and b(-,,)+c produces b[i][j]+c[i][j][k].  I call
the "-" a "pseudo-index" in Yorick.

In order for this to work, you need to broaden the definition of when
two arrays are conformable; instead of requiring an exact match for
the shape of the largest common cells of the two operands, you need to
treat 1-length dimensions as "wildcards" matching any length.  Thus,
if b has shape (2,3) then b(,-,) has shape (2,1,3) -- exactly the same
as reshape(b, (2,1,3)) -- which you can consider conformable to the
(2,4,3) array c by the rule that [i][0][k] matches [i][j][k].
Ordinary scalar broadcasting is just a specific case of this general
conformability rule.


I have the impression that the syntax would present very little
difficulty in Python; on the model of the All and Reverse index
objects, it would be easy to introduce a Pseudo object which did the

   d= b[(All,Pseudo,All)] + c

However, I imagine that the implementor will be loathe to change his
notion of array conformability at this late date.  Nevertheless, you
might as well ask to see what he says; probably the change wouldn't
actually break any existing correct code, but merely give meaning to
what would previously have produced a runtime error.  That "1-length
wildcard" rule has been a tremendous winner for Yorick; if you can
convince the Python guys to use it, I doubt they would ever regret it.

In Yorick, the combination of the relaxed conformability rule,
pseudo-indices and a general transpose operator that can produce any
permutation of dimensions allows for nearly all meaningful
combinations of arrays.


Yorick has one other indexing syntax which has proven useful, which I
call "rubber indices".  They address the problem of writing
interpreted code which extracts slices of arrays when you don't know
beforehand how many dimensions the array has.  An example is an
opacity array for which you know that the most slowly varying index
represents photon wavelength, but there might be zero, one, two, or
three faster varying dimensions representing position in space.  These
situations can probably be handled with Python's reshape operator, so
again this is a question of ease of use.

------- End of forwarded message -------

MATRIX-SIG  - SIG on Matrix Math for Python

send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org