From jfulton@usgs.gov Tue Sep 12 00:04:39 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Mon, 11 Sep 1995 19:04:39 -0400 Subject: [PYTHON MATRIX-SIG] Let's get going In-Reply-To: <412hu7$92c@GRAPEVINE.LCS.MIT.EDU> Message-ID: <199509112304.XAA13015@servrcolkr.cr.usgs.gov> Well, now that we're all here, let's get started. :-) As you know, the purpose of this SIG is to develop a proposal (and possible an implementation) for a new Python matrix type. I'd like to use Jim Hugunin's proposal as a starting point. I give my response to his proposal below. Before beginning this, however, I thought I'd summarize my involvement to date, so you know where I'm coming from. At the python workshop, I presented some work in progress on a Fortran (77) Interface Description Language (FIDL) for Python. This tool allows one to very quickly create Python modules that call Fortran libraries given brief high-level descriptions of the library routines to be called. The primary data structure in Fortran 77 is the fortran array, which is a multi-dimensional block of homogenous data elements. Of course, many interesting C routines, especially in mathematic and scientific domains, use this same data structure. To facilitate interfacing to these sorts of routines, I wanted an an efficient Python implementation of the same sort of data structure. In particular: o I wanted a python built-in data type containing a data structure that I could pass directly to Fortran or C routines. I did not want to have to do alot of data conversion or copying on each call. For example, if a Fortran routine returns a matrix, then it should be possible to pass this matrix to another Fortran routine without reallocating a data structure or converting the entire matrix. o I wanted multi-dimensional access as would be familiar to a python programmer. That is, if I have two-dimensional arrays, a and b, then: a[i][j] = b[k][l] should work as expected and should work efficiently. In general, given an n-dimensional matrix, m, if n > 1, then m[i] should return an n-1 dimensional matrix, and if n = 1, then m[i] should return a scalar. Note that for "m[i][j] = spam" to work correctly, assignment to the jth element of m[i] must be reflected in m. o I need to support *all* data types supported by Fortran 77, including strings. Not only do I have to interface to mathematical routines, but I need to interface to "legacy" systems (Some of which are still being written :). What I came up with was a fairly simple implementation of a matrix type that was a pure container type. My matrix implementation currently provides no numeric behavior, although a matrix type cries out for numeric behavior. My matrix implementation includes the following features: o Matrix objects have an internal data pointer that points to a homogenous block of memory. This pointer (or some offset from it, of course) can be passed directly to C or Fortran. o The internal data structure may be shared among multiple matrix objects. In particular, a __getitem__ from a n-dimensional matrix, where n>1, returns a matrix object that shares the same data (at an appropriate offset) as the original matrix. This shared data structure is reference counted, so if you have: a=Matrix([[1,2,3],[4,5,6],[7,8,9]]) b=a[1] # b == Matrix([4,5,6]) del a b is not invalidated by the third line. Note that if you have: a=Matrix([[1,2,3],[4,5,6],[7,8,9]]) b=Matrix([11,22,33]) a[1]=b a[1][1]=99 You end up with: a == Matrix([[1,2,3],[11,99,33],[7,8,9]]) b == Matrix([11,22,33]) That is, assigning to a matrix copies data. So assignment is by copy, even though access is by reference. (As you can see, matrices can be created from sequences of sequences. Matrices can also be assigned from arbitrary sequences of the right dimension or level of nesting.) o I chose to preserve the copy semantics of slices from lists. So in: a=Matrix([[1,2,3],[4,5,6],[7,8,9]]) b=a[1][:] # b == Matrix([4,5,6]) b[1]=99 # does not affect a The third line does not change a. So that's where I'm coming from. I think this type is generally useful, which is why I helped start this SIG. I'm pretty open on how and if matrices should have numeric behavior. With that, here are my comments on Jim's proposal. On 18 Aug 1995 17:16:55 GMT James Hugunin said: > I apologize for the length of this posting, but you were warned in the > subject header. > > There seems to be a fair amount of interest in the python community > concerning the addition of numeric operations to python. My own desire is > to have as large a library of matrix based functions available as possible > (linear algebra, eigenfunctions, signal processing, statistics, etc.). In > order to ensure that all of these libraries interoperate, there needs to > be agreement on a basic matrix object that can be used to represent arrays > of numbers. The following is a proposal for such an object. It can be > viewed as an extension of the current array object to higher dimensions > and numerical operations. > > The wheels have already been set in motion to create a Matrix SIG > mailing-list for further discussion of this proposal, as well as other > python numerics related issues. This should come on-line sometime next > week and serious discussion of this proposal should probably be deferred > until it can be done on that list. Summaries of that discussion will be > posted to the main list as consensus develops. > > Below I specify the behavior of a Matrix object within python. The C (or > FORTRAN) API to such an object is obviously of significant importance as > this object is primarily intended as an interface to existing numerical > libraries. I'm currently working on that proposal (at least the C part) > as well, but I think everyone will agree that this post is long enough as > it is. > > Note: Parts of this proposal are stolen from the python documentation for > the array object, and from discussions on the net. I'd like to > particularly thank Jim Fulton for letting me review (and steal ideas from) > his current Matrix object and for providing feedback on the first draft of > this proposal (though he certainly doesn't agree with everything in it, > even now). We aren't that far apart now, at least not on things that matter to me. :-) > > Matrix Object > > This defines a new object type which can efficiently represent a > multidimensional array of basic values (char, unsigned byte, signed byte, > int, short, long, float, double, complex float and complex double). > Matrices are defined as both a sequence type and a mapping type, and > usually (unless it's a matrix of char's) as a number type. It should be > noted that allowing a type to be both a sequence and a number and to > behave properly for addition and multiplication, will require some very > small patches to the ceval.c code. While I think the patches are OK, I don't think they're necessary, so if anyone objects to this proposal on the grounds that changes are required to ceval.c, you should not be concerned. > All of the examples use the following matrices: > Assume A = [1,2,3], B = [11,12,13], C = [[1,4,9],[16,25,36]], > M=[[ 1, 2, 3, 4, 5],[11,12,13,14,15],[21,22,23,24,25],[31,32,33,34,35]] > > > **Numeric operations** > > "+", "-", "*", "/", "^"(as power), abs, unary"-", and unary"+" are defined > as the corresponding element-wise arithmetic operations. Because these I'd rather see multiplication and division not be elementwise. But I don't feel strongly about this. > assume element-wise correspondence between the two matrices, the operandi > must be of the same dimensions (however, note the dimensions-coercion > section below). > > A+B = [12,14,16] > A*B = [11,24,39] > C*C = [[2,8,18],[32,50,72]] > -A = [-1,-2,-3] > > Possible suggestions for the other numeric operations: > "%" represents matrix multiplication (or vector dot product). If we go with elementwise interpretation for * and /, then I think that all operators that make sense for floating point numbers should have an elementwise interpretation, so m%spam should compute a matrix of mods. Perhaps one would carry this same argument to bitwise operators, since you could have matrices of integers. If we go with elementwise interpretation of numeric operations, then I'm inclined to use functional, rather than matrix notation for matrix multiplication, dot products, and so on. > "~" represents transposition. > Other suggestions? Perhaps, as a compromize, there could be a special method or operator that creates a reference copied version of a matrix that provides a different semantics. For example, perhaps the default matrix type could provide standard matrix interpretation for * and /, but there could be a method, elementwise that would allow: m.elementwise() + a to do elementwise arithmentic. IMPORTANT NOTE: Mixed-mode arithmetic between matrices and scalars should be defined. > > **Sequence operations** > > Concatenation and multiplication sequence operations are not defined, but > instead the arithmetic operations are invoked for the "+" and "*" > operators. Right. Note if we don't need elementwise bitwise operators, then we could concatinate with "|", which I find most natural in a matrix context: spam = foo | bar I suppose one could even repeat with >> or <<, but this is not so natural. > Indexing by both single indices and by slices is supported. > All returned matrices are returned by reference rather than by value. > len(M) is defined to return the size of the first dimension in in matrix. > > > examples: > len(M) -> 4 > > D = M[0] > D[0] = 66 > print M[0] > [66,2,3,4,5] Right, for my matrix type. > The semantics of returning a slice by reference is consistent with the > mapping operations given below, however, this is inconsistent with the > list objects semantics. This is open to debate. > > D = M[0:2] > D[0][0] = 66 > print M[0] > [66,2,3,4,5] I'm not sure what you were trying to say here, but this example holds only if M is a list of lists. If M is one of my matrices, then M is unchanged. Perhaps that is your point. Note that I tried to maintain the spirit of list semantics, in which the slice operator has copy semantics, but with a list, the things you are copying are, themselves, references. Anything is open to debate, but how else could this work, and satisfy other requirements at the same time? > > **Mapping operations** > > Recent newsgroup discussions have offered some convincing arguments for > treating a matrix as a mapping type where sequences of ints are used as > the keys. The sequence must be of length less than or equal to the numer > of dimensions in the matrix. Each element of the sequence is either an > integer or a sequence. If it is an integer, it returns the corresponding > element for that dimension, if it is a sequence then it returns all of the > elements given in the sequence. > > ie. A[0] -> 1, A[((1,2))] -> [2,3] > M[0] -> [1,2,3,4,5] > M[(0,3)] -> 4 > M[((0,2),0)] -> [1,21] > M[(range(1,3),range(2,4))] -> [[13,14],[23,24]] A way of visualizing this is as multi-dimensional slices that let you, for example, take a rectangular slice out of a 2-d matrix. > In order to simplify the syntax of these references, it would be valuable > to change the syntax of python so that M[0,3] is equivalent to M[(0,3)]. > This is a small change to the current indexing semantics which will give a > reasonable meaning to a currently illegal syntactic item. > > When used as a setvalue, the corresponding elements of the given array > will be set to the left hand side. All of these operations that return a > matrix will return it by-reference, meaning that the elements of the new > array will be references to the old one and changes to either array will > effect the other. Note that if one wants a reference, not a copy, one can always apply a copy operator, [:] as one often does not with lists. > > > **Instantiation** > > The following function will be used to instantiate a matrix: > Matrix(datasource(optional), d1, d2, ..., typecode='d') > > The typecodes are as follows: > c - char (non-numeric) > 1 - unsigned char > b - signed char > h - short > i - int > l - long > f - float > d - double > F - complex float > D - complex double > > The optional datasource can be a file object or a string object and will > fill the matrix with the raw data contained therein. In this case the > dimensions are required. The first dimension can be input as -1, in which > case the first dimension will be set to be as large as necessary to hold > the entire contents of the file or string. > > The optional datasource can also be a sequence, in which case the > dimensions of the array do not need to be passed in and are instead > determined by the hierarchical structure of the sequence. > > examples: > Matrix([range(3),(4,5,6)], 'i') -> [[0,1,2],[4,5,6]] > Matrix(2,3) -> [[0,0,0],[0,0,0]] > Matrix('abcdef', 2,3, 'u') -> [[97,98,99],[100,101,102]] > Matrix('abcdef', -1,2, 'c') -> ['ab', 'cd', 'ef'] Note that I have found it convenient to implement matrices so that 1-d matrices of chars behave alot like strings, as I often pass these to or get these back from Fortran routines in which they are just that. > > **Coercion** > > Dimension coercion:The following is a proposal to make the matrix object > behave as closely as possible to matrices in matlab (to make it easy to > convert matlab code to python), and to generalize this behavior to > arbitrarily high dimensional objects. > > If a matrix of fewer dimensions than that required is provided, then the > matrix can be converted to more dimensions by making copies of itself. > For purposes of dimension coercion, an int or a float is considered a 0-d > matrix. > > ie. A+1 -> [2,3,4], A+C -> [[2,6,12],[17,27,39]] > B^2 -> [121,144,169] vs. B^A -> [11, 144, 2197] I agree wrt scalars. > Simliar coercion should be performed for most functions acting on > matrices. Helper functions will be provided to make this easy, however > there is no way to enforce this. The following is a proposed standard to > allow functions to be applied to higher-dimensional objects in a uniform > way. > > If a function is defined to operate on scalars (say sqrt()), then when > applied to a matrix it should apply itself to every element in the matrix. > > ie. sqrt(C) -> [[1,2,3],[4,5,6]] > > This is generalized to higher dimensions so that if a function is provided > with a matrix of more dimensions that it requires, that function will > apply itself to each appropriately sized unit in the matrix. > > ie. sum(A) -> 6 whereas sum(C) -> [14,77] I am a bit concerned that this will lead to hard to find bugs. This also put's extra burden on the function implementor. I realize (since you explained it to me in a separate note :) your concern that this is needed for efficiency, otherwise the map operator, or perhaps a specialized matrix map operator could be used. > > Type coercion: No type coercion will be performed. However, it is > possible to have a function defined to work on multiple types (operator > overloading). So sqrt might be defined on matrices of both floats and > doubles in which case it will work properly when given either matrix as > input, but will return a type error if applied to a matrix of ints. > > > **Basic Data Items and Methods (non-numeric)** > > typecode - The typecode character used to create the matrix > > itemsize - The length in bytes of one matrix item in the internal > representation > > dimensions - A sequence containing the size of the matrix along each of ^^^^^^^^^^ tuple > its dimensions > > copy - Returns a copy of the matrix. This is a good way to take an array > returned by reference and turn it into one returned by value. This is not needed. Use [:]. > transpose - The transpose of the matrix (not needed if "~" is used for > transpose) I'm thinking that this (or ~) should return by reference. > byteswap - Useful for reading data from a file written on a machine with a > different byte order Hm. Does this operate in-place or return a value? You know, there really *should* be a matrix map operator. This should function like the standard map operator, but it will generate a matrix and it will probably want to be told at what level/dimension to apply the mapping function. In fact, this would give you arbitrary elementwise operations given scalar functions. > > typecast(t) - Returns a new matrix of the same dimensions where each item > is cast (using C typecasting rules) from the matrix's type to the new type > t. Our constructor already gives you this: new_matrix=Matrix(old_matrix,new_type) > tofile(f) - Write all items (as machine values) to the file object f. > > tolist() - Convert the matrix to an ordinary (possibly nested) list with > the same items. > > tostring() - Convert the array to an array of machine values and return > the > string representation (the same sequence of bytes that would > be written to a file by the tofile() method.) > > Note: the append, fromlist, etc. methods are not present because it is > assumed that a matrix object is of constant size (there are some strong > efficiency reasons for this assumption but this is likely to be another > source of arguments). > > **Basic functions taking matrix arguments (non-numeric)** > > Since it isn't possible to append to a matrix (or efficient if it's > decided it should be possible) I thought that some sort of function for > producing a new matrix from a list of matrices would be useful. So I'm > proposing the following function which is very similar in spirit to the > string.join operation. > > concat(l) - l must be a list of matrices, and each matrix in l must have > the same number of dimensions, as well as the same size for every > dimension except for the first. This will return a new matrix M whose > first dimension is the sum of the first dimensions of all the matrices in > the list, and whose data is filled from the data in all of these matrices. > > ie. concat([A,B]) -> [1,2,3,11,12,13] > > This is different from Matrix([A,B]) -> [[1,2,3],[11,12,13]] > > > **Basic functions taking matrix arguments (numeric)** > > It is hoped that there will be a large variety of special purpose modules > to do fancy numeric operations on matrices. There should also be a basic > set of operations provided in the Matrix module (the module with the > instantiation function). Because type coercion is not going to be > performed, it is important to decide both on the list of functions and on > the basic types they should support. The following is a very preliminary > list of the functions I consider most fundamental. > > rng(start, stop, step(optional), typecode='d') where all points in the > range must be strictly less than the stop point, according to the > semantics of range(). This function should possibly be added to the > instantiation options. > > ie. rng(0,0.4) = [0,0.1,0.2,0.3]; rng(0,0.1,0.41) = [0,0.1,0.2,0.3,0.4] > Hm. Why not: Matrix.range(start=1,stop,step=1, typecode='d') and Matrix.range([(start=1,stop,step=1), ...], typecode='d') That is: call it range, and give it same semantics, and include a multi-dimensional range operator, mrange, that takes a sequence of range specs. > > scalar operations (defined on floats, doubles, and all complex): > everything in mathmodule.c > round > > scalar operations (defined on all numeric types): > sign > > vector to scalar operations (defined on all numeric types): > max, min - Should these return the index of the max or min? > sum, prod > > vector to vector operations (defined on all numeric types): > cumsum, cumprod Comments anyone? -- Jim Fulton jfulton@usgs.gov (703) 648-5622 U.S. Geological Survey, Reston VA 22092 This message is being posted to obtain or provide technical information relating to my duties at the U.S. Geological Survey. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From graham@fishnet.net Tue Sep 12 08:46:50 1995 From: graham@fishnet.net (Graham Hughes) Date: Tue, 12 Sep 1995 00:46:50 -0700 Subject: [PYTHON MATRIX-SIG] Let's get going Message-ID: <199509120046.AAA31805@big.fishnet.net> At 07:04 PM 9/11/95 -0400, you wrote: > o I wanted a python built-in data type containing a data structure > that I could pass directly to Fortran or C routines. I did not > want to have to do alot of data conversion or copying on each > call. For example, if a Fortran routine returns a matrix, then > it should be possible to pass this matrix to another Fortran > routine without reallocating a data structure or converting the > entire matrix. Instant problem right there. To the best of my knowledge (and I've read this in several books), Fortran and C use completely different ways of storing multidimensional arrays. To wit: given the array m, like this: 3 4 5 6 7 8 9 0 3 -2 4 1 There are two ways of storing this array in a continuous memory location; by rows or by columns. By rows looks like this: '3 4 5 6 7 8 9 0 3 -2 4 1', and is used by both C and Pascal. By columns looks like this: '3 7 3 4 8 -2 5 9 4 6 0 1', and is used by Fortran. In other words, to pass to either Fortran or C functions you're going to need to know which type of function you have, and then convert the internal representation. This can be done, but it greatly complicates things. >> "+", "-", "*", "/", "^"(as power), abs, unary"-", and unary"+" are defined >> as the corresponding element-wise arithmetic operations. Because these > >I'd rather see multiplication and division not be elementwise. But I >don't feel strongly about this. Ah, but I'd prefer it be elementwise because it allows me to do wonderful APLish things with it :) A good reason for having multiplication/division (same thing on a mathematical level, although much stickier anywhere else) elementwise is that addition and subtraction already are, and it would be confusing to do it otherwise. However, a better reason is the inherent complications with matrix multiplications and divisions; specifically, multiplications require that the matricies have their rows and columns match in specific ways that are not immediately intuitive, and division is worse; not only does one need to be square, but the square matrix must be invertible, a condition not always fulfilled. Simple elementwise multiplication and division eliminates these non-intutive requirements in exchange for the same requirements that you have with addition and subtraction; both matricies must simply be of equal dimension. I'm not by any means suggesting that we not incorporate the matrixwise multiplication and division, but perhaps it would be better to have these as separate functions, both to avoid the above problems and to highlight their non-intuitive natures (I mean, really, there *still* isn't an easy method to invert a matrix, and the dimension requirements for matrix multiplication probably took us all a while to get used to)... >> Possible suggestions for the other numeric operations: >> "%" represents matrix multiplication (or vector dot product). > >If we go with elementwise interpretation for * and /, then I think >that all operators that make sense for floating point numbers should >have an elementwise interpretation, so m%spam should compute a matrix >of mods. Perhaps one would carry this same argument to bitwise >operators, since you could have matrices of integers. I agree. From experience with several kinds of matricies, I'm inclined to think that something similar to APL would be a really good idea... Given that, we *could* implement matrix multiplication, dot products, etc. as something similar to APL's outer and inner product functions... This would make it more general than simply matrix multiplication and therefore more useful, and the matrix multiplication could simply be a function that hardcodes a certain inner product operation... For those that haven't been exposed to APL, the inner and outer product functions are ways of implementing element-by-element functions to generalized tensors (although they're usually used on vectors and matricies only). An outer product returns a table of results of an operation foo on all pairs of one from matrix bar and one from matrix baz. That is, it does foo(bar[i],baz[j]) for each 0 <= i < len(bar) and 0 <= j < len(baz). Naturally, it works best in this form on vectors, but there is nothing preventing it being implemented on matricies; mail me for additional information on how to do this, as I need time to grovel over the manuals again so I can remember how to do it :) An inner product is similar; it takes two functions, foo and bar, and two matricies, baz and quux. It performs reduce(foo, (SUBSET1)) where SUBSET1 is bar(baz[i],quux[i]) for each 0 <= i < len(baz). The entirety can be formulated more like reduce(foo, map(bar, baz[0][0:], quux[0][0:])) but it is a little less clear IMHO. Plus, I'm not really sure I got those slices right; they're supposed to be the first row... One intriguing possiblity with using the APL stuff is it allows mixing and matching between matrix dimensions; there is nothing that says that you cannot perform an outer product on a matrix and a vector. This is even occasionally useful. (Some may ask, why the APL stuff? This is because I am convinced that APL is doing the Right Thing when it comes to matricies. J supports an even richer set of 'adverbs', but I have no intention of infecting Python with that kind of complexity ;) > >If we go with elementwise interpretation of numeric operations, then >I'm inclined to use functional, rather than matrix notation for matrix >multiplication, dot products, and so on. This matches with what I said *waaay* back before the inner and outer product stuff. >Perhaps, as a compromize, there could be a special method or operator >that creates a reference copied version of a matrix that provides a >different semantics. For example, perhaps the default matrix type >could provide standard matrix interpretation for * and /, but there >could be a method, elementwise that would allow: > > m.elementwise() + a > >to do elementwise arithmentic. I suppose if you use standard matrix interpretation for * and /, but if we don't and use elementwise stuff instead for the reasons already mentioned, than we can completely avoid this problem... >IMPORTANT NOTE: Mixed-mode arithmetic between matrices and scalars > should be defined. Here, we can stand on math; it is similar to elementwise arithmetic, i.e. m + s where m is a matrix and s is a scalar adds s to every element of m. This seems acceptable, and even intutive, to me. >> Concatenation and multiplication sequence operations are not defined, but >> instead the arithmetic operations are invoked for the "+" and "*" >> operators. > >Right. Note if we don't need elementwise bitwise operators, then we >could concatinate with "|", which I find most natural in a matrix >context: > > spam = foo | bar I can see your point... Dear me, looks like what we need are some APLlike functions... (kicks self) I happen to like having the "|" operator do bitwise stuff, simply because somebody's going to ask "well, what does & do?". They complement each other nicely. >I suppose one could even repeat with >> or <<, but this is not so natural. Yeah, plus << can give you a really *quick* multiplication by 2. >> Recent newsgroup discussions have offered some convincing arguments for >> treating a matrix as a mapping type where sequences of ints are used as >> the keys. The sequence must be of length less than or equal to the numer >> of dimensions in the matrix. Each element of the sequence is either an >> integer or a sequence. If it is an integer, it returns the corresponding >> element for that dimension, if it is a sequence then it returns all of the >> elements given in the sequence. >> >> ie. A[0] -> 1, A[((1,2))] -> [2,3] >> M[0] -> [1,2,3,4,5] >> M[(0,3)] -> 4 >> M[((0,2),0)] -> [1,21] >> M[(range(1,3),range(2,4))] -> [[13,14],[23,24]] > >A way of visualizing this is as multi-dimensional slices that let you, >for example, take a rectangular slice out of a 2-d matrix. Definitely useful; the most intutive (although not necessarily the fastest) way of calculating a determinant is to use multi-dimensional slices; I was stymied by the 'normal' Python list-of-lists implementation on this note when trying to implement a matrix inverter. >> **Instantiation** >> >> The following function will be used to instantiate a matrix: >> Matrix(datasource(optional), d1, d2, ..., typecode='d') >> >> The typecodes are as follows: >> c - char (non-numeric) >> 1 - unsigned char >> b - signed char >> h - short >> i - int >> l - long >> f - float >> d - double >> F - complex float >> D - complex double I'm not fond of this, as it breaks normal Python typing; let the matrix figure out what's there. Matter of fact, might be a good idea to make the matrix a general container like lists are now, so you can store, say, a tuple representing a complex number in there? As it stands, with these restrictions you can forget it. However, this makes it slightly more work for those wonderful Fortran library routines. >> **Coercion** >> >> Dimension coercion:The following is a proposal to make the matrix object >> behave as closely as possible to matrices in matlab (to make it easy to >> convert matlab code to python), and to generalize this behavior to >> arbitrarily high dimensional objects. >> >> If a matrix of fewer dimensions than that required is provided, then the >> matrix can be converted to more dimensions by making copies of itself. >> For purposes of dimension coercion, an int or a float is considered a 0-d >> matrix. >> >> ie. A+1 -> [2,3,4], A+C -> [[2,6,12],[17,27,39]] >> B^2 -> [121,144,169] vs. B^A -> [11, 144, 2197] > >I agree wrt scalars. > Interesting... I like it. So what happens if I convert a matrix of [1,2,3] into a 4x2 ([[x,x,x,x],[x,x,x,x]])? Do we get it wrapping around like [[1,2,3,1],[2,3,1,2]]? >> Simliar coercion should be performed for most functions acting on >> matrices. Helper functions will be provided to make this easy, however >> there is no way to enforce this. The following is a proposed standard to >> allow functions to be applied to higher-dimensional objects in a uniform >> way. >> >> If a function is defined to operate on scalars (say sqrt()), then when >> applied to a matrix it should apply itself to every element in the matrix. >> >> ie. sqrt(C) -> [[1,2,3],[4,5,6]] >> >> This is generalized to higher dimensions so that if a function is provided >> with a matrix of more dimensions that it requires, that function will >> apply itself to each appropriately sized unit in the matrix. >> >> ie. sum(A) -> 6 whereas sum(C) -> [14,77] Definitely. Back to the elementwise operations. >I am a bit concerned that this will lead to hard to find bugs. This >also put's extra burden on the function implementor. I realize (since >you explained it to me in a separate note :) your concern that this is >needed for efficiency, otherwise the map operator, or perhaps a >specialized matrix map operator could be used. I see your point. It might be better to use that matrix map operator... Best of both worlds, I guess. Oh, for APL's parallelism ;) That might not be a bad idea, but I think it would be a nasty bit of work for the interpreter and underlying source code. >> >> Type coercion: No type coercion will be performed. However, it is >> possible to have a function defined to work on multiple types (operator >> overloading). So sqrt might be defined on matrices of both floats and >> doubles in which case it will work properly when given either matrix as >> input, but will return a type error if applied to a matrix of ints. Well, if we simply let the caller figure out what it's got in it, like what I suggested above, this would be unnecessary. Let sqrt() (or indirectly, Python) figure out whether it's a double, int, or a complex. >> byteswap - Useful for reading data from a file written on a machine with a >> different byte order > >Hm. Does this operate in-place or return a value? > >You know, there really *should* be a matrix map operator. This should >function like the standard map operator, but it will generate a matrix >and it will probably want to be told at what level/dimension to apply >the mapping function. In fact, this would give you arbitrary >elementwise operations given scalar functions. Or, we can always use inner/outer products... :) >> typecast(t) - Returns a new matrix of the same dimensions where each item >> is cast (using C typecasting rules) from the matrix's type to the new type >> t. > >Our constructor already gives you this: > > new_matrix=Matrix(old_matrix,new_type) > Unnecessary if the matrix is a container. >> concat(l) - l must be a list of matrices, and each matrix in l must have >> the same number of dimensions, as well as the same size for every >> dimension except for the first. This will return a new matrix M whose >> first dimension is the sum of the first dimensions of all the matrices in >> the list, and whose data is filled from the data in all of these matrices. >> >> ie. concat([A,B]) -> [1,2,3,11,12,13] Hm. Seems to run into difficulty with >1D matricies. Ideally, concat should take 2 3 4 5 6 7 and 5 6 7 8 9 10 and return 2 3 4 5 6 7 5 6 7 8 9 10 But the given behaviour of concat depends greatly on whether [[x],[y]] is stored by rows or by columns, ie. is it like this: [ | , | ] V V [x] [y] or like this: [ -> [x] , -> [y] ] This is a conceptual difficulty. >> >> This is different from Matrix([A,B]) -> [[1,2,3],[11,12,13]] >> >> >> **Basic functions taking matrix arguments (numeric)** >> >> It is hoped that there will be a large variety of special purpose modules >> to do fancy numeric operations on matrices. There should also be a basic >> set of operations provided in the Matrix module (the module with the >> instantiation function). Because type coercion is not going to be >> performed, it is important to decide both on the list of functions and on >> the basic types they should support. The following is a very preliminary >> list of the functions I consider most fundamental. >> >> rng(start, stop, step(optional), typecode='d') where all points in the >> range must be strictly less than the stop point, according to the >> semantics of range(). This function should possibly be added to the >> instantiation options. >> >> ie. rng(0,0.4) = [0,0.1,0.2,0.3]; rng(0,0.1,0.41) = [0,0.1,0.2,0.3,0.4] >> > >Hm. Why not: > > Matrix.range(start=1,stop,step=1, typecode='d') > >and > > Matrix.range([(start=1,stop,step=1), ...], typecode='d') > >That is: > > call it range, and give it same semantics, and > > include a multi-dimensional range operator, mrange, that takes a > sequence of range specs. > Sounds good to me. >Comments anyone? Ditto. Anyone see something they'd like to pull apart (besides APL...)? Graham Graham Hughes Home page http://www.fishnet.net/~graham/ ``I think it would be a good idea.'' -- Mahatma Ghandi, when asked what he thought of Western civilization finger for PGP public key. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From ji@tarzan.math.jyu.fi Tue Sep 12 10:17:13 1995 From: ji@tarzan.math.jyu.fi (Jonne Itkonen) Date: Tue, 12 Sep 1995 12:17:13 +0300 (EETDST) Subject: [PYTHON MATRIX-SIG] Let's get going In-Reply-To: <199509120046.AAA31805@big.fishnet.net> Message-ID: On Tue, 12 Sep 1995, Graham Hughes wrote: > >I'd rather see multiplication and division not be elementwise. But I > >don't feel strongly about this. > > Ah, but I'd prefer it be elementwise because it allows me to do wonderful > APLish things with it :) I prefer the mathematical, not elementwise multiplication. We are handling matrixes here (not tables!). Besides, one doesn't want to define the * operator for complex numbers to do 'elementwise' multiplication, does she? It would be meaningless! In my opinion, the overloading of operators is to make things more clear than to mess them up. The elementwise multiplication by * operator would be clear to anyone with no knowledge of matrices, but not to someone who knows what matrices are. Perhaps a table object should be developed for elementwise calculation? > A good reason for having multiplication/division (same thing on a > mathematical level, although much stickier anywhere else) elementwise is > that addition and subtraction already are, and it would be confusing to do That is not a good reason. It's more confusing to use mathematical definition for some, and non-mathematical definitions for other operators, as in your '+ vs. *' example. -Jonne URL: http://www.math.jyu.fi/~ji/ ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From Michal.Spalinski@fuw.edu.pl Tue Sep 12 11:27:57 1995 From: Michal.Spalinski@fuw.edu.pl (Michal.Spalinski@fuw.edu.pl) Date: Tue, 12 Sep 95 12:27:57 +0200 Subject: [PYTHON MATRIX-SIG] Let's get going In-Reply-To: <199509120046.AAA31805@big.fishnet.net> (message from Graham Hughes on Tue, 12 Sep 1995 00:46:50 -0700) Message-ID: <9509121027.AA01195@albert4.fuw.edu.pl> If this is going to be useful to people like me who are very fond of Python and have concrete numerical work to do in the field of Physics or whatever, the `*`, `+` etc really should be reserved for matrix operations (not elementwise). This is because one would like to be able to type a complicated formula and have it parsed correctly, and at the same time one should be able to look at it a week later and see immediately whether a sign is wrong or a factor is missing. This is the problem with using lisp for this kind of thing - if the notation is different from that which one is used to (by writing on paper) it makes it laborious to check whether the formulae are really correct. Regarding the APL type of stuff, outer products and so on, I think it is worth having and potentially useful, but it should use a different syntax. Maybe new infix operators, or just function calls - but not `+`, `*` etc. -- Michal S. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From forrest@rose.rsoc.rockwell.com Tue Sep 12 14:34:38 1995 From: forrest@rose.rsoc.rockwell.com (Dave Forrest) Date: Tue, 12 Sep 1995 08:34:38 -0500 Subject: [PYTHON MATRIX-SIG] * and / operators (was Re: Let's get going) Message-ID: <9509121334.AA24449@feynman.rsoc.rockwell.com> [Heartwarming personal intro - skip if you're not interested] Hi, I'm Dave Forrest with Rockwell Space Ops. Co. A few of you have met my colleagues Robin Friedrich, Greg Boes, and Charlie Fly. We are working on ground-based simulation, monitoring, and analysis software for the Space Shuttle. So, we have mostly an engineering/physics/math bias in our use of Python - although we're watching the python GUI work with keen interest. We love this language! [Real comments - start reading here] First off I'd like to add another vote for * being a matrix multiply, rather than elementwise. As for /, however, that doesn't really have meaning for matrices. When we built a C++ Matrix class, we only defined / for element-wise division by a scalar. It works quite nicely. We end up with something like this: Matrix operator/(Matrix, scalar) Matrix operator*(Matrix, Matrix) Matrix operator*(Matrix, scalar) <- elementwise muliplication by scalar Note that this avoids the sticky situation with inversion of matrices - we found it better to leave that for derived classes so that people could implement different inversion routines that capitalize on any a_priori knowledge of what's in the Matrix. e.g. class SquareMatrix( Matrix ): def inv(): ... That being said, I would like to see elementwise multiplication, etc. in there somewhere - but we use them far less frequently than real matrix operations. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Tue Sep 12 19:08:18 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Tue, 12 Sep 1995 14:08:18 -0400 Subject: [PYTHON MATRIX-SIG] Python, matrices, and everything Message-ID: <199509121808.OAA03216@cyclone.ERE.UMontreal.CA> [Yet another introduction - skip the next paragraph if you want to read something about matrices...] My name is Konrad Hinsen (*not* Hinsen Konrad, as given in my address), and I am a computational physicist, currently working as a postdoc at the University of Montreal. My main field of work is computer simulations of various kinds, and I use Python for everything that it not time-critical. My opinion about Python is a mixture of admiration for its elegant and efficient structure and frustration about its lack of numerical features, which is why I joined the Matrix-SIG. [Here comes the real stuff.] Before jumping into technical details, it might be a good idea to define what we are aiming at. As it has already become clear from the dicussion, matrices are used in two different (but not entirely distinct) ways: 1) in linear algebra applications 2) as containers of values that are structured as tables In linear algebra, one needs mainly one- and two-dimensional matrices of real or complex numbers. Important operations are addition and multiplication, factorization, inversion, eigenvalues etc. There are efficient Fortran and C libraries for these operations, so what is needed is a suitable representation in Python that allows interfacing to these libraries and some convenient way of accessing them, i.e. operators (+ - *, maybe / for scalar division) and tons of operations that are best packaged into a matrix class. The second application is just as important (in my view) and much more complex. As APLers know, there are lots of useful operations on tables of values, which can be of any dimension and contain any data type, although the numerical are the most important ones (Python already provides good string handling). It makes sense to have all mathematical functions and operators acting by default elementwise, including of course multiplication, but also user-defined functions. Basically arrays would be extensions of numerical scalars, the latter becoming arrays of rank zero. The by far most elegant and powerful array concept I know of is the one implemented in J, a relatively new language that can be regarded as an improved version of APL. I certainly don't want to take over J's other features, but its concept of array and operator ranks is extremely useful. I'll give a short description below. But first let's see how the two applications can be reconciled in a single implementation. Many of the linear algebra operations make sense only for two-dimensional matrices, but the J approach of operator rank takes care of the nicely. And even without that approach, this would only mean some more error checking for such operations. The only point of conflict is the implementation of multiplication (there is no difference for addition and subtraction). Linear-algebra style matrix multiplication is regarded as an inner product with addition and multiplication operations from an APL/J point of view, so it is available, but arguably not in the most convenient form. On the other hand, once the * symbol means matrix multiplication, the total general structure of elementwise operations is ruined. I therefore propose to introduce some other symbol for matrix multiplication. One could choose some combination like ".*" or "*.", which looks similar enough to plain multiplication to make its meaning evident. And now I'll shortly describe J's rank system. Every object's rank is identical to the number of its dimensions - a scalar has rank 0, a vector rank 1, and so on. When used in operations however, a rank N array is viewed as a rank A array of rank B cells, where A+B=N. A rank 3 array, for example, can be used as a three-dimensional array of scalars, as a two-dimensional matrix of vectors, as a vector of two-dimensional matrices, or as a "scalar" containing a three-dimensional array. How it is used depends on the rank of the operation acting on it. Most scalar operations have infinite rank, which means that use their operators at the highest rank possible. Effectively that means elementwise operation in the classical sense, except that there are also rules that cover the combination of object of different rank, such as multiplication of an array by a scalar. But there are also other operations. Matrix inversion, for example, by default operates on rank 2 cells. So if you "invert" a three-dimensional array, this operation will be interpreted as inversion of each rank-2 cell in the array. The result is again a rank 3 array. The beauty of the system is that the rank of each operation can be changed at will, not only for the built-in operations, but also for user-defined functions. If, for example, you want to add a rank 2 matrix to each rank 2 cell of a rank 3 array, you just specify rank 2 addition and get what you want. Nothing could be simpler or more flexible. Unfortunately, I dislike many of J's other features as a programming language, which is why I am still a Python user... So that's my point of view in this discussion: let's concentrate on APLish (or Jish) arrays and then add the operations that are needed in linear algebra. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Tue Sep 12 20:21:49 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Tue, 12 Sep 1995 15:21:49 -0400 Subject: [PYTHON MATRIX-SIG] Let's get going In-Reply-To: <199509120046.AAA31805@big.fishnet.net> Message-ID: <199509121922.TAA12230@servrcolkr.cr.usgs.gov> On Tue, 12 Sep 1995 00:46:50 -0700 Graham Hughes said: > At 07:04 PM 9/11/95 -0400, you wrote: > > o I wanted a python built-in data type containing a data structure > > that I could pass directly to Fortran or C routines. I did not > > want to have to do alot of data conversion or copying on each > > call. For example, if a Fortran routine returns a matrix, then > > it should be possible to pass this matrix to another Fortran > > routine without reallocating a data structure or converting the > > entire matrix. > > Instant problem right there. To the best of my knowledge (and I've read this > in several books), Fortran and C use completely different ways of storing > multidimensional arrays. To wit: given the array m, like this: > > 3 4 5 6 > > 7 8 9 0 > > 3 -2 4 1 > > There are two ways of storing this array in a continuous memory location; by > rows or by columns. By rows looks like this: '3 4 5 6 7 8 9 0 3 -2 4 1', and > is used by both C and Pascal. By columns looks like this: '3 7 3 4 8 -2 5 9 > 4 6 0 1', and is used by Fortran. In other words, to pass to either Fortran > or C functions you're going to need to know which type of function you have, > and then convert the internal representation. > > This can be done, but it greatly complicates things. Not really. Fortran stores data in multidimensional arrays with the indexes changing most rapidly in the left. Some people give this the interpretation that Fortran stores data "by column". C multi-dimensional arrays are stored with the indexes changing most rapidly on the right. Some people give this the interpretation that C stores data "by row". The difference is mainly one of interpretation. Both C and Fortran store mult-dimensional data pretty much the same way, as a big homogenous "rectangulish" glob of data. They differ only in the way they order the indexes to this data, and this ordering is only important for a certain set of interpretations. I choose to interpret C and Python multidimensional indexes, especially in the context of matrices, in a way in which this difference in indexing does not present a problem. I think of n-dimensional matrices as sequences of n-1-dimensional matrices. That is, matrices are stored, "by sub-matrix". Now, in the case of 2-d matrices, I was taught to think of a matrix as a collection of column vectors. With this interpretation, C and python 2-d matrices are, indeed, stored by column. Consider m[i][j]. Since 2-d matrices are a collection of column vectors, m[i] is the ith column. That is, we give the column index *first*, and then the row index. With this interpretation, there is no difference between C and Fortran storage formats. (snip) > > >> **Instantiation** > >> > >> The following function will be used to instantiate a matrix: > >> Matrix(datasource(optional), d1, d2, ..., typecode='d') > >> > >> The typecodes are as follows: > >> c - char (non-numeric) > >> 1 - unsigned char > >> b - signed char > >> h - short > >> i - int > >> l - long > >> f - float > >> d - double > >> F - complex float > >> D - complex double > > I'm not fond of this, as it breaks normal Python typing; let the matrix > figure out what's there. Matter of fact, might be a good idea to make the > matrix a general container like lists are now, so you can store, say, a > tuple representing a complex number in there? As it stands, with these > restrictions you can forget it. However, this makes it slightly more work > for those wonderful Fortran library routines. But the data must be stored in a homogenous format that can be passed to C and Fotran libraries. Also, libraries often require specific types, so you want control over the types used. I suppose that the routines could be made smart enough to sniff at the first element and check for numeric, complex (objects with re and im attributes), or string data and pick the default type accordingly. Note that the routines are already smart enough to convert arbitrary compatible objects to the necessary format. (snip) > >> concat(l) - l must be a list of matrices, and each matrix in l must have > >> the same number of dimensions, as well as the same size for every > >> dimension except for the first. This will return a new matrix M whose > >> first dimension is the sum of the first dimensions of all the matrices in > >> the list, and whose data is filled from the data in all of these matrices. > >> > >> ie. concat([A,B]) -> [1,2,3,11,12,13] > > Hm. Seems to run into difficulty with >1D matricies. Ideally, concat should take > > 2 3 4 5 6 7 > and > 5 6 7 8 9 10 > > and return > > 2 3 4 5 6 7 > > 5 6 7 8 9 10 > > But the given behaviour of concat depends greatly on whether [[x],[y]] is > stored by rows or by columns, ie. is it like this: > > [ | , | ] > V V > [x] [y] > > or like this: > > [ > -> [x] > , > -> [y] > ] > > This is a conceptual difficulty. No, it is not a problem as long as you use the interpretation that n-dimensional matrices are sequences of n-1-dimensional matrices. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Tue Sep 12 20:56:30 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Tue, 12 Sep 1995 15:56:30 -0400 Subject: [PYTHON MATRIX-SIG] Let's get going In-Reply-To: <199509121922.TAA12230@servrcolkr.cr.usgs.gov> (jfulton@usgs.gov) Message-ID: <199509121956.PAA08479@cyclone.ERE.UMontreal.CA> Consider m[i][j]. Since 2-d matrices are a collection of column vectors, m[i] is the ith column. That is, we give the column index Of course you could also say that 2-d matrices are a collection of row vectors, which then give you the other interpretation. Anyway, I agree that this is just an interpretation problem and not serious, although users will have to be aware of how Python matrices correspond to Fortran/C matrices. I suppose that the routines could be made smart enough to sniff at the first element and check for numeric, complex (objects with re and im attributes), or string data and pick the default type accordingly. Speaking of complex numbers: has it ever been considered to make them built-in objects in Python? This would simplify the matrix implementation, and also make the math functions behave more reasonably (i.e. sqrt(-1.) would return i instead of causing an exception). It can't be much work and would be very useful for numerical work. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From chris.chase@jhuapl.edu Tue Sep 12 22:05:55 1995 From: chris.chase@jhuapl.edu (Chris Chase S1A) Date: Tue, 12 Sep 1995 17:05:55 -0400 Subject: [PYTHON MATRIX-SIG] Vector-style operations vs. matrix operations In-Reply-To: <9509121027.AA01195@albert4.fuw.edu.pl> References: <199509120046.AAA31805@big.fishnet.net> <9509121027.AA01195@albert4.fuw.edu.pl> Message-ID: <199509122104.RAA06307@python.org> Hello, This is a long post. Self-introduction: I am an electrical engineer PhD working at the Applied Physics Lab of Johns Hopkins University. I do a lot of remote sensor data analysis requiring alot of rapidly developed numerical and visualization software. I would like to express some of my opinions regarding the elementwise versus matrix operators discussion. Elementwise operations for the proposed matrix objects fall under the category of vector operations in many math libriaries. When the matrix object is allocated in a single contiguous block of memory as in C or Fortran (and has been proposed in this forum) vector operations view the matrix as a one dimensional vector. These types of libraries support as a minimum '+','-','*' (and maybe '/') in an elementwise fashion. Each is invaluable in code that wishes to avoid for loops. Many popular interactive array oriented languages support vector operations. Below I list how a few of these languages implement vector and matrix operations. In Matlab, vector operations (i.e. elementwise) are called array operations and use the operators "+", "-", ".*" "./" and ".^". "*", "/", and "^" are used for matrix multiplication, right inverse, and matrix powers, respectively. In IDL (which I use more frequently than Matlab), "+", "-", "*", and "/" are all vector operators. "#" is matrix multiplication (actually "#" is used for Fortran-style arrays and "##" is used for C style arrays which can be very confusing). There is no operator performing matrix division. In Mathematica, elementwise multiplication is a space or "*" and matrix multiplication is given by the "." operator. There is not a matrix division operator. In APL (http://www.acm.org/sigapl/) "+", "-", "*", and "/" are vector-style operations. They are even more general in that they can be applied to specific ranks (as posted by Hinsen Konrad). Special operators are used for matrix inversion. Matrix multiplication is expressed using a combination of operators with the inner product operator (specifically, "+.x"). As much as I like the generality and completeness of the functional-style operators in APL and its descendant J, I think that the operator explosion makes the code extremely unreadable for the uninitiated. The following are all the same in the use of vector and matrix operations to Matlab and may be of interest for their other features regarding arrays: scilab http://www.inria.fr/Logiciels/SCILAB-eng.html octave http://www.che.utexas.edu/octave.html (maintained by the FSF - the GNU people that produce gcc, g++, emacs, etc.) tela http://www.geo.fmi.fi/prog/tela.html rlab ftp://evans.ee.adfa.oz.au/pub/RLaB/ or ftp://csi.jpl.nasa.gov/pub/matlab/RLaB/ (also GNU). tela is tensor oriented (allowing >2 dimensional arrays), so it uses generalizations of matrix multiplication and inner products and is a nice improvenment on that point over Matlab. ----- My preferences: In short, I agree with the proposal by Hinsen Konrad , i.e., "+", "-", "*", and "/" as vector style operations but applied rank-wise. I prefer to have arbitrary dimensioned arrays [i.e., tensors or tables as hinsenk@ere.umontreal.ca (Hinsen Konrad) refered to them] rather than only one dimensional vectors or two dimensional matrices. I prefer that "+", "-", "*", "/" all perform vector style operations. This makes their behavior consistent among each other. (The posted comment about "*" not applying to complex valued arrays was incorrect.) In most of my scientific programming, I encounter these vector style operators much more often than matrix multiplication and division. They replace the many "for" loops that are so common in numerical algorithms. They are displayed compactly and execute much faster than the loops in an interpreted language. When using higher dimensional arrays, matrix multiplication is rather specialized even when generalized to tensors. Matrix multiplication is actually a specialization of a general inner product (e.g., the implementation in APL). As such, if matrix multiplication had to have an operator I would choose a new one. Or, we would be better served by a generalized inner product operator. But if we don't want to add new operators I would let "*" be the more common vector (elementwise) operator. The Matlab approach is good, but it requires additional operators like ".*" and "./". Matlab also suffers from limitations to two dimensions (this is the main motivation behind the development of Tela). "/" does not generalize easily to the higher dimensions and the generalization of "*" is useful but limited when applied (as in Tela) only along the inner dimensions of two tensors. At the higher dimensions the elementwise operators applied at specific ranks are much more common and usefule then a generalized matrix multiplication operator. If it is desirable to avoid the proliferation of operators as in APL and J then matrix multiplication and division would be best left as function calls. There are additional reasons to advise against a "/" operator for matrix division. First there is the issue of whether a left or right inverse is desired (this is handled in Matlab by using two different operators "/" and "\"). Second, many matrix inversion problems are best solved using decomposition algorithms, such as LU decomposition or SVD, that use multiple steps wherein the intermediate results of the decomposition may saved for solving the same problem (e.g. with many different parameter matrices.) Additionally, the user often would like to pick which type of inversion algorithm to use and may want to override default parameters for these algorithms. I would prefer that matrix inversion be left to a group of explicit function calls. I suppose what is implemented for an array extension to Python depends on the end goal: 1) a Matlab-like array extension for Python is oriented toward 2-dimensional matrix linear algebra. Although limited in scope, this is useful and in particular allows very compact readable notation for linear algebra equations. However, it also tends to be stuck on a row/column view of arrays. 2) an array extension to handle higher dimension arrays (call them tensors or tables) that can easily handle generalized rank arithmetic (APL or J style as briefly set forth in Hinsen Konrad's post) and with specialized functions for matrix multiplication and division. Generalized inner and outer products (taking user specified binary operators) would also be very helpful. This view is more compatible with Jim Fulton's "n-dimensional matrices are sequences of n-1-dimensional matrices." In either case, I think it would be a mistake to add too many new operators in the APL or J fashion. Such clutter would reduce the readability of Python programs. One also runs the risk of trying to make a subset of Python look like a functional programming language which could further confuse users. Sincerely, Chris Chase =============================== Bldg 24-E188 The Applied Physics Laboratory The Johns Hopkins University Laurel, MD 20723-6099 (301)953-6000 x8529 chris.chase@jhuapl.edu ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Tue Sep 12 22:28:16 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Tue, 12 Sep 1995 17:28:16 -0400 Subject: [PYTHON MATRIX-SIG] Vector-style operations vs. matrix operations In-Reply-To: <199509122104.RAA06307@python.org> (message from Chris Chase S1A on Tue, 12 Sep 1995 17:05:55 -0400) Message-ID: <199509122128.RAA12993@cyclone.ERE.UMontreal.CA> In either case, I think it would be a mistake to add too many new operators in the APL or J fashion. Such clutter would reduce the readability of Python programs. One also runs the risk of trying to make a subset of Python look like a functional programming language which could further confuse users. Let me add that I totally agree about that. There is no point in re-creating APL or J, especially since Python is already a better language in many respects (especially when it comes to legibility). All I would like to take over from J is its array and rank concept. In terms of array-specific operations, I suggest inner and outer product, transpose (in the general sense), and reduction (I hope I haven't forgotten anything). These would be implemented as methods, not as special operators as in APL or J. Together with the elementwise operations this would give an array system superior to most languages (except for APL/J of course) and extremely useful both for linear algebra and all kinds of data handling. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From guido@CNRI.Reston.VA.US Wed Sep 13 02:22:17 1995 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Tue, 12 Sep 1995 21:22:17 -0400 Subject: [PYTHON MATRIX-SIG] Let's get going In-Reply-To: Your message of "Tue, 12 Sep 1995 15:56:30 EDT." <199509121956.PAA08479@cyclone.ERE.UMontreal.CA> References: <199509121956.PAA08479@cyclone.ERE.UMontreal.CA> Message-ID: <199509130122.VAA23857@monty> > Speaking of complex numbers: has it ever been considered to make > them built-in objects in Python? This would simplify the matrix > implementation, and also make the math functions behave more > reasonably (i.e. sqrt(-1.) would return i instead of causing > an exception). It can't be much work and would be very useful > for numerical work. Hm. I think it *would* be much work, since the standard C math library doesn't have any functions working on complex numbers. I have a specific objection against making sqrt(-1) returning a complex number. In Python, as a rule, the *type* of a function's return value may depend on the *type* of its argument(s), but not in their *value*. Even though there is nothing (or at least very little) in the language's implementation that enforces this, it is a consistent convention throughout the language. It even has one important visible consequence: 1/3 is calculated as truncating, as in C (though it truncates towards -infinity rather than towards 0), rather than returning a floating point value if the result cannot be represented as an integer. I believe this is an important property -- it makes reasoning about the types of variables/arguments/functions in a Python program easier (and sooner or later, this reasoning is going to be done by programs as well as by people). --Guido van Rossum URL: (I have more to say about the matrix discussion but I need some more time to digest everything that's been said.) ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From graham@fishnet.net Wed Sep 13 05:21:25 1995 From: graham@fishnet.net (Graham Hughes) Date: Tue, 12 Sep 1995 21:21:25 -0700 Subject: [PYTHON MATRIX-SIG] Let's get going Message-ID: <199509122120.VAA22737@big.fishnet.net> At 12:17 PM 9/12/95 +0300, you wrote: >I prefer the mathematical, not elementwise multiplication. We are handling >matrixes here (not tables!). Besides, one doesn't want to define the * >operator for complex numbers to do 'elementwise' multiplication, does she? >It would be meaningless! Ah, but would it be? Part of the reason why I wanted to do elementwise multiplication and division is that matrix multiplication is nearly trivial with inner and outer products. E.g.; you want your `normal' complex multiplication? Try outer_product(lambda x,y: x+y,lambda x,y: x*y, v1, v2) which can easily be all a member function calls. Similarly, (BTW, this works with APL's data parallelism; depending on how we implement outer_product it may be a bit nastier), matrix multiplication is this: outer_product(lambda x,y: x+y, lambda x,y: x*y, m1, m2.transpose()) (Note the transposition.) Finally, we reach the question of efficiency. No matter how you implement it, matrix multiplication will be slow(er). Divison, besides which it won't always work, will take roughly forever; APL is quick on this note, but that's because to do matrix multiplication you use a special purpose function. This, and the odd dimension requirements for matrix multiplication, are why I'm not fond of having the `mathematical' methods be the default. Besides, if we do it that way, we can treat one-dimensional vectors consisting of only one element (i.e. [3] or some such) essentially as scalars, which is a parallelism I like; whereas matrix multiplication and division will take 10 times as long to get the same answer. > >In my opinion, the overloading of operators is to make things more clear >than to mess them up. The elementwise multiplication by * operator would >be clear to anyone with no knowledge of matrices, but not to someone who >knows what matrices are. Correct me if I'm wrong, but doesn't matrix multiplication use a special syntax all its own? All the matricies are either boxed or in boldface, some times they even use 'X' instead of a dot, all to make absolutely sure that the reader gets the point. Great efforts are taken to make sure that anyone reading knows that the multiplication does weird stuff here (because weird stuff happens; where else do you take two things of unequal size and come up with something else with a possibly completely different size?) I understand the engineering tendency to think in terms of matrix multiplication, but there is a precedent for divorcing it from the matrix addition; the two are radically different operations. > >Perhaps a table object should be developed for elementwise calculation? Probably not; too much common code to make it worthwhile. Besides, since Python is typeless, we *cannot* distinguish between multiplication by a scalar or by a matrix. There is no typeof() function to distinguish them. To avoid bizarrities concerning *scalar* multiplication, we must use elementwise multiplication. If this were C++, we could use two seperate operator *()s, but it's not. And we're pleased about that most of the time :) >That is not a good reason. It's more confusing to use mathematical >definition for some, and non-mathematical definitions for other operators, >as in your '+ vs. *' example. To you. To the person coming from APL, J or some other similar languages, it's the other way around. Plus it raises problems dealing with scalar multiplication. Graham Graham Hughes Home page http://www.fishnet.net/~graham/ ``I think it would be a good idea.'' -- Mahatma Ghandi, when asked what he thought of Western civilization finger for PGP public key. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Wed Sep 13 14:21:09 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Wed, 13 Sep 1995 09:21:09 -0400 Subject: [PYTHON MATRIX-SIG] Let's get going In-Reply-To: <199509122120.VAA22737@big.fishnet.net> Message-ID: <199509131321.NAA07565@servrcolkr.cr.usgs.gov> These are good discussions. I'm going to have to find some time to digest what's been said. Here are a few quick comments that don't require much thought. ;-) On Tue, 12 Sep 1995 21:21:25 -0700 Graham Hughes said: > At 12:17 PM 9/12/95 +0300, you wrote: (snip) > Finally, we reach the question of efficiency. No matter how you implement > it, matrix multiplication will be slow(er). Divison, besides which it won't > always work, A number of people (not to pick on you) have made the agument that "such and such shouldn't be done because it won't always work". I really don't think this is a valid argument. Most, if not all, of the operations and functions have some preconditions. You can't do elementwise arithmetic unless both operands have the same dimensions. You can't do elementwise division if *any* elements of the divisor are zero. Heck, scalar division fails if a divisor is zero, but Guido was still kind enough to include "/" in teh language. (Whew. :-) > Probably not; too much common code to make it worthwhile. Besides, since > Python is typeless, Python is not typeless, only it's variables are. > we *cannot* distinguish between multiplication by a > scalar or by a matrix. Not necessarily by reading the code. > There is no typeof() function to distinguish them. No, it's called type(). There's also hasattr, which can be used to check basic object signatures. > To > avoid bizarrities concerning *scalar* multiplication, we must use > elementwise multiplication. If this were C++, we could use two seperate > operator *()s, but it's not. > It is easy enough to handle this with a check in the single * operator. In fact, I think that whathever we do, numeric operations involving matrices should work with: o Matrices and matrices, o Matrices and scalars, and o Matrices and sequences of sequences (non-Matrix objects that support similar multi-dimensional access). Supporting this does not really depend on whether a element-wise interpretation is used for *. Note that applying the mapping operator to take matrix slices will *not* return matrix objects, since matrix objects are supposed to contain contiguous globs of memory, so that their data can be passed to C and Fortran libraries directly. These matrix slices will reference matrix objects so that modification of slice elements will be reflected in the matrices from which they are derived, but for mathematical operations, they will probably use different code. My suspicion is that there will be code that: o Operates on two matrices (and perhaps a matrix and a scalar), using knowledge of internal data structures to optimize performance, and separate code that o Operates on sequences of sequences. These will be written in C, for speed, but will use PySequence_GetItem calls to access operand data. > > >That is not a good reason. It's more confusing to use mathematical > >definition for some, and non-mathematical definitions for other operators, > >as in your '+ vs. *' example. Actually, the "mathematical" interpretation of + happens to be the same as the elementwise interpretation. Still, most of the people who call for a mathematical interpretation of * seem only to want that operator to be non-elementwise. This seems to me to stengthen the case of those who want an elementwise interpretation for all operations. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Sep 13 14:45:03 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Wed, 13 Sep 1995 09:45:03 -0400 Subject: [PYTHON MATRIX-SIG] Let's get going In-Reply-To: <199509130122.VAA23857@monty> (message from Guido van Rossum on Tue, 12 Sep 1995 21:22:17 -0400) Message-ID: <199509131345.JAA26202@cyclone.ERE.UMontreal.CA> Hm. I think it *would* be much work, since the standard C math library doesn't have any functions working on complex numbers. True, but complex libraries are easily available (e.g. GNU-C comes with one). I have a specific objection against making sqrt(-1) returning a complex number. In Python, as a rule, the *type* of a function's return value may depend on the *type* of its argument(s), but not in their *value*. Even though there is nothing (or at least very little) There is no need to make real and complex numbers different types. A real number would just be a complex number with an imaginary part of zero. Of course internally there should be a difference for efficiency reasons, but the user need not see it. Contrary to integers, reals numbers need not have a special type since any operation on complex numbers with zero imaginary part is exactly equivalent to an operation on real numbers (for integers there is a difference in division). consistent convention throughout the language. It even has one important visible consequence: 1/3 is calculated as truncating, as in C (though it truncates towards -infinity rather than towards 0), rather than returning a floating point value if the result cannot be represented as an integer. And I am sure this feature has already led to some surprises for people who thought they could use Python like a pocket calculator. For languages like Python, where numerical efficiency does not have top priority, I'd consider it more useful to have a number model that reflects mathematical categories instead of internal storage format categories. There should be only one type "number" with representations for integers, fractions, floating point numbers, and complex numbers, the latter having real and imaginary parts that can be of any of the basic representations. Then 1/3 would return a fraction. This is what anyone without programming experience would expect. Of course I realize that it is much too late to introduce such a change... I believe this is an important property -- it makes reasoning about the types of variables/arguments/functions in a Python program easier (and sooner or later, this reasoning is going to be done by programs as well as by people). I certainly agree on that. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@mama-bear.lcs.mit.edu Wed Sep 13 17:04:44 1995 From: jjh@mama-bear.lcs.mit.edu (James Hugunin) Date: Wed, 13 Sep 95 12:04:44 -0400 Subject: [PYTHON MATRIX-SIG] Vector-style operations vs. matrix operations References: <199509120046.AAA31805@big.fishnet.net> <9509121027.AA01195@albert4.fuw.edu.pl> <199509122104.RAA06307@python.org> Message-ID: <9509131604.AA04152@nineveh.LCS.MIT.EDU> Short personal intro, feel free to skip: Hi, my name's Jim Hugunin and I'm a PhD student working in MIT's Laboratory for Computer Science. My current work involves designing speech recognition systems. My Master's thesis involved the fabrication and computer modeling of superconducting transistors (using the Bogoliubov deGennes equations if anyone cares) so I'm reasonably familiar with numerical modelling in both the EE and the physics communities. I'm the one who wrote the initial proposal for the matrix object that Jim Fulton posted to start this discussion, so you already know my opinions on most of these issues. I've also implemented a matrix object (on top of Jim Fulton's) which provides most of the capabilities described in the proposal. Real comments begin here: First, some posts seem to be placing too much stock in the name "matrix" object. I propose that the name should be changed to either table or tensor (as mentioned by Chris Chase) if this is necessary to quell the arguments that a "matrix" should behave like a mathematical "matrix". (Note: I have no objections to arguments in favor of matrix-multiplication for "*", I just don't think they should be based on something as arbitrary as the object's name). Next, I'd like to add my vote to those in favor of element-wise operations. This just seems to me to be the most general purpose sort of functionality that could be provided by an object that stores contiguous blocks of homogeneous data. I think that all of the same element-wise operations should be provided as are provided for the int and float objects (note this is a small change to my proposal). This makes a table object simply a way of storing a multi-dimensional block of homogeneous data-elements, that can be operated on in exactly the same way as the equivalent scalars in python with greatly improved performance. I also completely agree with Konrad Hinsen's suggestion that J's rank system be used for table objects. This is in fact a very minor change to my original proposal. J's rank system is simply a clearer way of expressing my dimensionality coercion ideas. Chric Chase suggests: Generalized inner and outer products (taking user specified binary operators) would also be very helpful. I'm curious to better understand this proposal. If it means defining an outer product that can take python binary functions as it's arguments, I would argue that it would be incredibly inefficient (compared to native C), and therefore might as well be implemented in python as a set of "helper" functions. If it means specifying only primitive binary arithmetic operations (like "+" and "*") then I think that it could be implemented efficiently, but I would really like to see some examples of where this would be useful (other than matrix multiplication which will probably be done with special purpose code for efficiency reasons anyway). All of the above comments also relate to proposals for specialized "map" and "reduce" functions for tables/matrices. Unless there is some large efficiency gain possible, I'd rather see these implemented in python than C. -Jim Hugunin hugunin@mit.edu ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@mama-bear.lcs.mit.edu Wed Sep 13 17:39:13 1995 From: jjh@mama-bear.lcs.mit.edu (James Hugunin) Date: Wed, 13 Sep 95 12:39:13 -0400 Subject: [PYTHON MATRIX-SIG] Let's get going References: <199509112304.XAA13015@servrcolkr.cr.usgs.gov> Message-ID: <9509131639.AA04160@nineveh.LCS.MIT.EDU> In my previous post I comment on what appears to be the primary source of controversy for the matrix object, whether it should be treated as a table of numbers, or as a true mathematical matrix. I'm afraid that that issue is likely to be the source of substantial more discussion before anything is decided. I would also like to comment on some of the less controversial points raised by Jim Fulton in his first post regarding basic methods for matrix objects. >> transpose - The transpose of the matrix (not needed if "~" is used for >> transpose) > I'm thinking that this (or ~) should return by reference. I agree completely. In fact, when I went about implementing matrix slices (so that m[((1,3),)] can be properly returned by reference, I noticed that transposing a matrix was trivially implemented by converting it to a matrix slice. This of course raises the issue of how to properly deal with these matrix slices. For example, the way my code is written now, if I try to add a slice to a matrix, I can perform the operation quickly without needing to convert the slice to a matrix, as long as the slice meets a fairly obscure list of requirements. Basically these can be summarized as the fact that there must be a constant offset between the indices for each dimensions. ie. M[((1,3,5),)] would be easy to deal with (as fast as a matrix) whereas: M[((1,3,6),)] would not be What bothers me about this gets back to your earlier complaints regarding automatic coercion as being a possible source of errors (which I've come around to agreeing with in general). On the other hand, conceptually, it would be nice to be able to treat a slice exactly the same as a matrix whenever possible. I see three options, none of which seems perfect, but I am inclined towards the third. 1) Don't allow slices to be passed to functions which require a matrix argument, and don't allow numeric operations on slices 2) Treat a slice just like any other non-matrix sequence, and allow numeric operations, and allow it to be passed to matrix functions using the generic sequence to matrix functions (there is an interesting issue here having to do with copying data back from the matrix to the input sequence if it turns out that the function modifies it's input data. I'm curious as to how you deal with this, or whether you just don't allow this to happen). 3) Whenever a slice is passed to a function requiring a matrix argument, automatically copy the data from the slice to a contiguous block of memory. Allow slices to be used in numeric operations, doing this efficiently where possible, and copying the slice to a contiguous block of memeory where not possible. This would be the most efficient solution. >> byteswap - Useful for reading data from a file written on a machine with a >> different byte order > Hm. Does this operate in-place or return a value? In-place. In fact, I am tempted to try and make all of the methods on matrices operate "in-place" in order to be in line with list objects. >> typecast(t) - Returns a new matrix of the same dimensions where each item >> is cast (using C typecasting rules) from the matrix's type to the new type >> t. >Our constructor already gives you this: > new_matrix=Matrix(old_matrix,new_type) True. My problem is that I am frequently having to cast between types (I deal with samples of sound that usually have to come in and go out as shorts, but which I usually want to operate on as floats in the meantime). So, I agree that this should be done using the matrix constructor, and all that I'd add is that the matrix constructor should be written so that this typecasting is done efficiently. -Jim Hugunin hugunin@mit.edu ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@mama-bear.lcs.mit.edu Wed Sep 13 17:39:13 1995 From: jjh@mama-bear.lcs.mit.edu (James Hugunin) Date: Wed, 13 Sep 95 12:39:13 -0400 Subject: [PYTHON MATRIX-SIG] Let's get going References: <199509112304.XAA13015@servrcolkr.cr.usgs.gov> Message-ID: <9509131639.AA04160@nineveh.LCS.MIT.EDU> In my previous post I comment on what appears to be the primary source of controversy for the matrix object, whether it should be treated as a table of numbers, or as a true mathematical matrix. I'm afraid that that issue is likely to be the source of substantial more discussion before anything is decided. I would also like to comment on some of the less controversial points raised by Jim Fulton in his first post regarding basic methods for matrix objects. >> transpose - The transpose of the matrix (not needed if "~" is used for >> transpose) > I'm thinking that this (or ~) should return by reference. I agree completely. In fact, when I went about implementing matrix slices (so that m[((1,3),)] can be properly returned by reference, I noticed that transposing a matrix was trivially implemented by converting it to a matrix slice. This of course raises the issue of how to properly deal with these matrix slices. For example, the way my code is written now, if I try to add a slice to a matrix, I can perform the operation quickly without needing to convert the slice to a matrix, as long as the slice meets a fairly obscure list of requirements. Basically these can be summarized as the fact that there must be a constant offset between the indices for each dimensions. ie. M[((1,3,5),)] would be easy to deal with (as fast as a matrix) whereas: M[((1,3,6),)] would not be What bothers me about this gets back to your earlier complaints regarding automatic coercion as being a possible source of errors (which I've come around to agreeing with in general). On the other hand, conceptually, it would be nice to be able to treat a slice exactly the same as a matrix whenever possible. I see three options, none of which seems perfect, but I am inclined towards the third. 1) Don't allow slices to be passed to functions which require a matrix argument, and don't allow numeric operations on slices 2) Treat a slice just like any other non-matrix sequence, and allow numeric operations, and allow it to be passed to matrix functions using the generic sequence to matrix functions (there is an interesting issue here having to do with copying data back from the matrix to the input sequence if it turns out that the function modifies it's input data. I'm curious as to how you deal with this, or whether you just don't allow this to happen). 3) Whenever a slice is passed to a function requiring a matrix argument, automatically copy the data from the slice to a contiguous block of memory. Allow slices to be used in numeric operations, doing this efficiently where possible, and copying the slice to a contiguous block of memeory where not possible. This would be the most efficient solution. >> byteswap - Useful for reading data from a file written on a machine with a >> different byte order > Hm. Does this operate in-place or return a value? In-place. In fact, I am tempted to try and make all of the methods on matrices operate "in-place" in order to be in line with list objects. >> typecast(t) - Returns a new matrix of the same dimensions where each item >> is cast (using C typecasting rules) from the matrix's type to the new type >> t. >Our constructor already gives you this: > new_matrix=Matrix(old_matrix,new_type) True. My problem is that I am frequently having to cast between types (I deal with samples of sound that usually have to come in and go out as shorts, but which I usually want to operate on as floats in the meantime). So, I agree that this should be done using the matrix constructor, and all that I'd add is that the matrix constructor should be written so that this typecasting is done efficiently. -Jim Hugunin hugunin@mit.edu ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From forrest@rose.rsoc.rockwell.com Wed Sep 13 17:36:03 1995 From: forrest@rose.rsoc.rockwell.com (Dave Forrest) Date: Wed, 13 Sep 1995 11:36:03 -0500 Subject: [PYTHON MATRIX-SIG] Are we talking about one thing or two? Message-ID: <9509131636.AA00821@feynman.rsoc.rockwell.com> When we started talking about the use of operator* being elementwise or "mathematical", I thought that was the extent of it - a disagreement over one operator. But now I think we may actually be asking for two completely different animals. Mr. Hugunin suggested that "matrix" is, in effect, just a name. Well, maybe not. For what we do here we have definite semantics assigned to the word "matrix" and they are the "mathematical" semantics described earlier. The "elementwise" stuff is more like what we would call an "array" or "table" - although "tensor" sounds good to me. So, I'd like to suggest that we pursue not one, but two new entities. A "matrix" would be limited to two dimensions and support the "mathematical" notion that most of us engineers had beat into our brains in college. A "tensor" or "table" or even "array" would fit the more general interpretation and have element-wise operations. Why? For practical reasons. We here - and I would submit that many other organizations are like this - do more Matlab-like work. The way Matlab defines a matrix is not a problem but a huge advantage for us. Several of the rest of you apparently deal with more abstract notions either as data storage elements, set theoretic elements, or mathematical elements. You need the elementwise stuff just as badly as we need our "mathematical" stuff. Rather than trying to make a one-size-fits all that isn't exactly what anyone wants, why not make two interfaces (maybe one implementation?) that give each camp what they want to work with? ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Sep 13 18:43:37 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Wed, 13 Sep 1995 13:43:37 -0400 Subject: [PYTHON MATRIX-SIG] Are we talking about one thing or two? In-Reply-To: <9509131636.AA00821@feynman.rsoc.rockwell.com> (forrest@rose.rsoc.rockwell.com) Message-ID: <199509131743.NAA09394@cyclone.ERE.UMontreal.CA> When we started talking about the use of operator* being elementwise or "mathematical", I thought that was the extent of it - a disagreement over one operator. But now I think we may actually be asking for two completely different animals. Mr. Hugunin suggested that "matrix" is, in effect, just a name. Well, maybe not. For what we do here we have But the one operator is the only difference - apart from the interpretation of multiplication, "mathematical" matrices are just a special case of general arrays. "mathematical" semantics described earlier. The "elementwise" stuff is more like what we would call an "array" or "table" - although "tensor" sounds good to me. Not to me though. A tensor is a mathematical object representing a physical quantity in space that exists indepently of a choice of a coordinate system. A tensor is thus not just a table of numbers, but a set of numbers with a well-defined transformation property when changing from one coordinate system to another. I vote for "array" as the name for what we are trying to do. Why? For practical reasons. We here - and I would submit that many other organizations are like this - do more Matlab-like work. The way Matlab defines a matrix is not a problem but a huge advantage for us. But in what way would a more general concept be a disadvantage? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Sep 13 19:07:22 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Wed, 13 Sep 1995 14:07:22 -0400 Subject: [PYTHON MATRIX-SIG] Let's get going In-Reply-To: <9509131639.AA04160@nineveh.LCS.MIT.EDU> (message from James Hugunin on Wed, 13 Sep 95 12:39:13 -0400) Message-ID: <199509131807.OAA10603@cyclone.ERE.UMontreal.CA> 3) Whenever a slice is passed to a function requiring a matrix argument, automatically copy the data from the slice to a contiguous block of memory. Allow slices to be used in numeric operations, doing this efficiently where possible, and copying the slice to a contiguous block of memeory where not possible. This would be the most efficient solution. Why would it have to be copied whenever it is passed to a function? It would be sufficient to make a copy whenever the matrix is changed. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Sep 13 19:23:29 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Wed, 13 Sep 1995 14:23:29 -0400 Subject: [PYTHON MATRIX-SIG] Vector-style operations vs. matrix operations In-Reply-To: <9509131604.AA04152@nineveh.LCS.MIT.EDU> (message from James Hugunin on Wed, 13 Sep 95 12:04:44 -0400) Message-ID: <199509131823.OAA11504@cyclone.ERE.UMontreal.CA> Chric Chase suggests: Generalized inner and outer products (taking user specified binary operators) would also be very helpful. I'm curious to better understand this proposal. If it means defining an outer product that can take python binary functions as it's arguments, I would argue that it would be incredibly inefficient (compared to native C), and therefore might as well be implemented in python as a set of "helper" functions. If it means specifying only primitive binary arithmetic operations (like "+" and "*") then I think that it could be implemented efficiently, but I would really like to see some examples of where this would be useful (other than matrix multiplication which will probably be done with special purpose code for efficiency reasons anyway). Of course general inner and outer products, as well as reduction, should work with any binary operation, including user-defined functions. And I agree that this is best done in Python. However, the most important applications use only the built-in operations (arithmetic, comparison, logical), so it would make sense to provide a more efficient solution for them. There are lots of applications for these things, as any APLer will confirm. Unfortunately I can't include any APL idioms here due to character set restrictions :-( But let me give a simple example: suppose you have an array of data values x (one-dimensional). You want to make a cumulative histogram, i.e. an array of integers n in which each entry n[i] indicates the number of values in x that are larger than b[i], where b[i] is the value of each "bin" in the histogram. This is just an outer product using >, followed by a reduction with +: n = reduce(lambda a,b: a+b, 1, outer_product(lambda a,b: a>b, x, b)) in Pseudo-Python, where the second argument of "reduce" indicates the rank of the cells to be reduced. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Wed Sep 13 19:24:18 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Wed, 13 Sep 1995 14:24:18 -0400 Subject: [PYTHON MATRIX-SIG] Let's get going In-Reply-To: <199509131807.OAA10603@cyclone.ERE.UMontreal.CA> Message-ID: <199509131824.SAA17108@servrcolkr.cr.usgs.gov> On Wed, 13 Sep 1995 14:07:22 -0400 Hinsen Konrad said: > > 3) Whenever a slice is passed to a function requiring a matrix argument, > automatically copy the data from the slice to a contiguous block of memory. > Allow slices to be used in numeric operations, doing this efficiently where > possible, and copying the slice to a contiguous block of memeory where not > possible. This would be the most efficient solution. > > Why would it have to be copied whenever it is passed to a function? > It would be sufficient to make a copy whenever the matrix is > changed. This gets back to my earlier comment that slices, as proposed cannot be matrices because they store a reference to the data from which they were created and a bunch of offsets for accessing the orinal data properly. When passed to math/sci libraries, they'll have to be converted to real matrices so that their data are contiguous. JIm ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Sep 13 19:28:26 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Wed, 13 Sep 1995 14:28:26 -0400 Subject: [PYTHON MATRIX-SIG] Let's get going In-Reply-To: <199509131824.SAA17108@servrcolkr.cr.usgs.gov> (jfulton@wrdmail.er.usgs.gov) Message-ID: <199509131828.OAA11695@cyclone.ERE.UMontreal.CA> > Why would it have to be copied whenever it is passed to a function? > It would be sufficient to make a copy whenever the matrix is > changed. This gets back to my earlier comment that slices, as proposed cannot be matrices because they store a reference to the data from which they were created and a bunch of offsets for accessing the orinal data properly. When passed to math/sci libraries, they'll have to be converted to real matrices so that their data are contiguous. I see. "Function" means "function in an external library", not "Python function" as I had assumed. BTW, there is a good book called "Scientific and Engineering C++" that appeared recently, unfortunately I don't remember the names of the authors (but I can check at home). It contains a very good discussion of how a matrix class in C++ could be designed, and this covers many of the topics we are discussing here. Unfortunately they treat only one- and two-dimensional matrices. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Wed Sep 13 19:34:11 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Wed, 13 Sep 1995 14:34:11 -0400 Subject: [PYTHON MATRIX-SIG] Let's get going In-Reply-To: <199509131828.OAA11695@cyclone.ERE.UMontreal.CA> Message-ID: <199509131834.SAA17250@servrcolkr.cr.usgs.gov> On Wed, 13 Sep 1995 14:28:26 -0400 Hinsen Konrad said: > > > Why would it have to be copied whenever it is passed to a function? > > It would be sufficient to make a copy whenever the matrix is > > changed. > > This gets back to my earlier comment that slices, as proposed cannot > be matrices because they store a reference to the data from which they > were created and a bunch of offsets for accessing the orinal data > properly. When passed to math/sci libraries, they'll have to be > converted to real matrices so that their data are contiguous. > > I see. "Function" means "function in an external library", not > "Python function" as I had assumed. Python functions won't really care whether they get a "true" matrix, or a sequence of sequences, as they will use []s to access data in either case. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From forrest@rose.rsoc.rockwell.com Wed Sep 13 20:45:10 1995 From: forrest@rose.rsoc.rockwell.com (Dave Forrest) Date: Wed, 13 Sep 1995 14:45:10 -0500 Subject: [PYTHON MATRIX-SIG] Are we talking about one thing or two? Message-ID: <9509131945.AA01068@feynman.rsoc.rockwell.com> > From owner-matrix-sig@python.org Wed Sep 13 12:46 CDT 1995 [snip] > > Why? For practical reasons. We here - and I would submit that many > other organizations are like this - do more Matlab-like work. The way > Matlab defines a matrix is not a problem but a huge advantage for us. > > But in what way would a more general concept be a disadvantage? Easy - a more general concept might require me to worry about extra things that I'm not interested in (like the number of dimensions, or the "rank" that J uses). These things are not interesting and can only cause problems by being there to make mistakes on. We will end up writing a wrapper that gives us the interface we want and hides the things that we're not interested in - but since we're not the only ones who want this and it would be more efficiently done as an intrinsic part of the language I think that's the right thing to do. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Wed Sep 13 21:38:56 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Wed, 13 Sep 1995 16:38:56 -0400 Subject: [PYTHON MATRIX-SIG] Let's get going In-Reply-To: <9509131639.AA04160@nineveh.LCS.MIT.EDU> Message-ID: <199509132038.UAA22390@servrcolkr.cr.usgs.gov> On Wed, 13 Sep 95 12:39:13 -0400 James Hugunin said: > (snip) > > I would also like to comment on some of the less controversial points > raised by Jim Fulton in his first post regarding basic methods for matrix > objects. > > >> transpose - The transpose of the matrix (not needed if "~" is used for > >> transpose) > > I'm thinking that this (or ~) should return by reference. > > I agree completely. In fact, when I went about implementing matrix slices > (so that m[((1,3),)] can be properly returned by reference, I noticed that > transposing a matrix was trivially implemented by converting it to a matrix > slice. This of course raises the issue of how to properly deal with these > matrix slices. > > For example, the way my code is written now, if I try to add a slice to a > matrix, I can perform the operation quickly without needing to convert the > slice to a matrix, as long as the slice meets a fairly obscure list of > requirements. Basically these can be summarized as the fact that there must > be a constant offset between the indices for each dimensions. ie. > > M[((1,3,5),)] would be easy to deal with (as fast as a matrix) whereas: > M[((1,3,6),)] would not be I think this is too strange. > What bothers me about this gets back to your earlier complaints regarding > automatic coercion as being a possible source of errors (which I've come > around to agreeing with in general). On the other hand, conceptually, it > would be nice to be able to treat a slice exactly the same as a matrix > whenever possible. Right. This is polymorphism, not coercion. > I see three options, none of which seems perfect, but I am inclined towards > the third. > > 1) Don't allow slices to be passed to functions which require a matrix > argument, and don't allow numeric operations on slices Yuck. > 2) Treat a slice just like any other non-matrix sequence, and allow numeric > operations, and allow it to be passed to matrix functions using the generic > sequence to matrix functions (there is an interesting issue here having to > do with copying data back from the matrix to the input sequence if it turns > out that the function modifies it's input data. I'm curious as to how you > deal with this, or whether you just don't allow this to happen). > 3) Whenever a slice is passed to a function requiring a matrix argument, > automatically copy the data from the slice to a contiguous block of memory. > Allow slices to be used in numeric operations, doing this efficiently where > possible, and copying the slice to a contiguous block of memeory where not > possible. This would be the most efficient solution. I can't tell 2 and three apart. Currently, if FIDL calls a function that modifies one of it's arguments, the modified value is returned as an element of the return list. Non-scalar output/modify values are always matrices. If a modify argument *is* a matrix, then it gets modified, otherwise it gets copied. This is a bit gross but it handles most cases. > > >> byteswap - Useful for reading data from a file written on a machine with a > >> different byte order > > > Hm. Does this operate in-place or return a value? > > In-place. In fact, I am tempted to try and make all of the methods on > matrices operate "in-place" in order to be in line with list objects. I'm not sure what you mean by this. Surely, you aren't trying to make: m=[[1,2,3],[4,5,6],[7,8,9]] b=[11,22,33] m=[1]=b b[1]=99 cause m[1][1] to equal 99? Are you? Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From chris.chase@jhuapl.edu Wed Sep 13 21:47:19 1995 From: chris.chase@jhuapl.edu (Chris Chase S1A) Date: Wed, 13 Sep 1995 16:47:19 -0400 Subject: [PYTHON MATRIX-SIG] Are we talking about one thing or two? In-Reply-To: <9509131636.AA00821@feynman.rsoc.rockwell.com> References: <9509131636.AA00821@feynman.rsoc.rockwell.com> Message-ID: <199509132046.QAA11169@python.org> >>>>> "Dave" == Dave Forrest writes: Dave> Rather than trying to make a one-size-fits all that isn't exactly what Dave> anyone wants, why not make two interfaces (maybe one implementation?) Dave> that give each camp what they want to work with? By two interfaces do you mean having different meanings for an operator such as "*" depending on the arguments classes? I think it would be too confusing to have an operation "A*B" that performs matrix multiplication when A and B are matrices (two dimensional arrays) but performs elementwise multiplication for the a general class of arrays (whatever you want to call them - arrays, tensors, tables - I prefer arrays). I think that supporting the Matlab-like matrix linear algebra such as matrix multiplication and matrix division is a high priority for many people. One question is can the new operators be added to the language without difficulty, in a logical fashion, and without cluttering the language with operators that are only used for this "matrix" class? The Matlab approach uses "+", "-", ".*", "./" ".^" for elementwise operations (called array operations in Matlab. I call them vector operations.) "*", "/", "\", "^" for matrix operations. ("^" is a Matrix power operator). Python is much more general than a matrix language, so I think that using all of these would give Python operator bloat. Is adding new operators to the language even a possibility? New methods or functions would have to be defined anyway to implement the operators, e.g. matrixmultiply(a,b), vectormultiply(a,b), leftdivision(a,b), etc. Without operators that call these functions we would probably want shorter names. >> >> Why? For practical reasons. We here - and I would submit that many >> other organizations are like this - do more Matlab-like work. The way >> Matlab defines a matrix is not a problem but a huge advantage for us. >> >> But in what way would a more general concept be a disadvantage? Dave> Easy - a more general concept might require me to worry about extra Dave> things that I'm not interested in (like the number of dimensions, or Dave> the "rank" that J uses). These things are not interesting and can only Dave> cause problems by being there to make mistakes on. We will end up Dave> writing a wrapper that gives us the interface we want and hides the Dave> things that we're not interested in - but since we're not the only ones Dave> who want this and it would be more efficiently done as an intrinsic Dave> part of the language I think that's the right thing to do. Even if you limit yourself to only two dimensions you still have to worry about rank. In Matlab you can make a mistake using one-dimensional vectors when a matrix is required. Tela is a perfect example of a language that took the Matlab syntax and added higher dimensional arrays without breaking any of the basic Matlab matrix features. IDL and APL are additional examples with higher dimensional arrays that don't limit their capability to do matrix linear algebra. Matlab is two-dimensional more because of its limited scope and computer resources from historical beginnings. I even seem to recall being informed by Mathworks representatives at a tradeshow that higher dimensional arrays would be added in a future release to Matlab. The natural implemenation would be a multi-dimensional array class as Jim is proposing with a matrix linear algebra module to specialize this class [for implementing matrix multiplication, matrix inverses/adjoints (square and non-square "matrix division") , linear equation solvers, column rank, factorization, spectral/eigenvalue computations, special matrices (diagonal, Hankel, toeplitz, Hilbert, sparse, etc.), norms, matrix exponential, and on and on]. This would easily be a whole additional SIG - the Matrix Linear Algebra (Matlab clone) SIG. Such a SIG wouldn't have to implement another object type and external interface except for subclassing to special matrices (sparse in particular). Is the purpose of this SIG only for implementing a two-dimensional matrix linear algebra system within Python or for implementing a more general purpose multi-dimensional array class with homogeneous elements for easy interface to external scientific and mathematical libraries? Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@mama-bear.lcs.mit.edu Wed Sep 13 22:01:31 1995 From: jjh@mama-bear.lcs.mit.edu (James Hugunin) Date: Wed, 13 Sep 95 17:01:31 -0400 Subject: [PYTHON MATRIX-SIG] Let's get going References: <199509132038.UAA22390@servrcolkr.cr.usgs.gov> Message-ID: <9509132101.AA04255@nineveh.LCS.MIT.EDU> I've got to run to a meeting, but I wanted to reply to your points. I'm not at all sure that we disagree on the implementation of slices. Let me ask two simple questions. Let's say A is a slice taken from a matrix, and B is a contiguous matrix. 1) Should I be able to say C = A+B, and have the obvious thing happen (assuming that dimensions line up, etc.)? I'd say yes, and I have a few performance optimizations to make this fast in certain cases and nobody needs to worry about those. 2) Should I be able to say fft(A) where fft is an in-place FFT routine? Should it be expected to modify the memory referred to by A, or only to return a brand-new matrix which corresponds to the fft of A? I'm not sure what the right answer to this one is. > > > Hm. Does this operate in-place or return a value? > > > > In-place. In fact, I am tempted to try and make all of the methods on > > matrices operate "in-place" in order to be in line with list objects. > > I'm not sure what you mean by this. Surely, you aren't trying to > make: > > m=[[1,2,3],[4,5,6],[7,8,9]] > b=[11,22,33] > m=[1]=b > b[1]=99 > > cause m[1][1] to equal 99? Are you? Not at all! All I meant by this is that methods on list objects (like insert or append) actually change the list object that they are operating on, whereas operations like concatenation return a new object. In that vein, I feel that m.byteswap() should operate in-place on m and cause the memory associated with m to be byte-swapped. This is as opposed to having it return a new matrix in with the same dimensions as m, but with all values byte-swapped. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Wed Sep 13 21:55:28 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Wed, 13 Sep 1995 16:55:28 -0400 Subject: [PYTHON MATRIX-SIG] Are we talking about one thing or two? In-Reply-To: <199509132046.QAA11169@python.org> Message-ID: <199509132055.UAA23220@servrcolkr.cr.usgs.gov> On Wed, 13 Sep 1995 16:47:19 -0400 Chris Chase S1A said: > >>>>> "Dave" == Dave Forrest writes: > > Dave> Rather than trying to make a one-size-fits all that isn't exactly what > Dave> anyone wants, why not make two interfaces (maybe one implementation?) > Dave> that give each camp what they want to work with? > > By two interfaces do you mean having different meanings for an > operator such as "*" depending on the arguments classes? > > I think it would be too confusing to have an operation "A*B" that > performs matrix multiplication when A and B are matrices (two > dimensional arrays) but performs elementwise multiplication for the > a general class of arrays (whatever you want to call them - arrays, > tensors, tables - I prefer arrays). Hm. Well this is what happens now with numbers. Math on integers is different from math on numbers. > I think that supporting the Matlab-like matrix linear algebra such as > matrix multiplication and matrix division is a high priority for many > people. One question is can the new operators be added to the > language without difficulty, in a logical fashion, and without > cluttering the language with operators that are only used for this > "matrix" class? > > > The Matlab approach uses > > "+", "-", ".*", "./" ".^" for elementwise operations (called array > operations in Matlab. I call them vector operations.) > > "*", "/", "\", "^" for matrix operations. ("^" is a Matrix power operator). > > Python is much more general than a matrix language, so I think that > using all of these would give Python operator bloat. > > Is adding new operators to the language even a possibility? I'm 99% sure that the answer is no. I think that we should try to keep this proposal to things that can be implemented with a new module withion the bounds of the existing language. > New methods or functions would have to be defined anyway to implement > the operators, e.g. matrixmultiply(a,b), vectormultiply(a,b), > leftdivision(a,b), etc. Without operators that call these functions we > would probably want shorter names. > > >> > >> Why? For practical reasons. We here - and I would submit that many > >> other organizations are like this - do more Matlab-like work. The way > >> Matlab defines a matrix is not a problem but a huge advantage for us. > >> > >> But in what way would a more general concept be a disadvantage? > > Dave> Easy - a more general concept might require me to worry about extra > Dave> things that I'm not interested in (like the number of dimensions, or > Dave> the "rank" that J uses). These things are not interesting and can only > Dave> cause problems by being there to make mistakes on. We will end up > Dave> writing a wrapper that gives us the interface we want and hides the > Dave> things that we're not interested in - but since we're not the only ones > Dave> who want this and it would be more efficiently done as an intrinsic > Dave> part of the language I think that's the right thing to do. > > Even if you limit yourself to only two dimensions you still have to > worry about rank. In Matlab you can make a mistake using > one-dimensional vectors when a matrix is required. Tela is a perfect > example of a language that took the Matlab syntax and added higher > dimensional arrays without breaking any of the basic Matlab matrix > features. IDL and APL are additional examples with higher dimensional > arrays that don't limit their capability to do matrix linear algebra. > Matlab is two-dimensional more because of its limited scope and > computer resources from historical beginnings. I even seem to recall > being informed by Mathworks representatives at a tradeshow that higher > dimensional arrays would be added in a future release to Matlab. > > The natural implemenation would be a multi-dimensional array class as > Jim is proposing with a matrix linear algebra module to specialize > this class [for implementing matrix multiplication, matrix > inverses/adjoints (square and non-square "matrix division") , linear > equation solvers, column rank, factorization, spectral/eigenvalue > computations, special matrices (diagonal, Hankel, toeplitz, Hilbert, > sparse, etc.), norms, matrix exponential, and on and on]. This would > easily be a whole additional SIG - the Matrix Linear Algebra (Matlab > clone) SIG. Such a SIG wouldn't have to implement another object type > and external interface except for subclassing to special matrices > (sparse in particular). > > Is the purpose of this SIG only for implementing a two-dimensional > matrix linear algebra system within Python or for implementing a more > general purpose multi-dimensional array class with homogeneous > elements for easy interface to external scientific and mathematical > libraries? The latter. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Sep 13 22:17:54 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Wed, 13 Sep 1995 17:17:54 -0400 Subject: [PYTHON MATRIX-SIG] Are we talking about one thing or two? In-Reply-To: <199509132046.QAA11169@python.org> (message from Chris Chase S1A on Wed, 13 Sep 1995 16:47:19 -0400) Message-ID: <199509132117.RAA20989@cyclone.ERE.UMontreal.CA> The natural implemenation would be a multi-dimensional array class as Jim is proposing with a matrix linear algebra module to specialize this class [for implementing matrix multiplication, matrix inverses/adjoints (square and non-square "matrix division") , linear ... Why specialise? That only limits the flexibility of the whole package. Why shouldn't it be possible to have "inverse" invert each two-dimensional slice of a three-dimensional array? There are applications for it, it doesn't cost any extra effort to do it, so why not do it? Why invent artificial restrictions? Is the purpose of this SIG only for implementing a two-dimensional matrix linear algebra system within Python or for implementing a more general purpose multi-dimensional array class with homogeneous elements for easy interface to external scientific and mathematical libraries? I'd say both. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Sep 13 21:10:49 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Wed, 13 Sep 1995 16:10:49 -0400 Subject: [PYTHON MATRIX-SIG] Are we talking about one thing or two? In-Reply-To: <9509131945.AA01068@feynman.rsoc.rockwell.com> (forrest@rose.rsoc.rockwell.com) Message-ID: <199509132010.QAA17017@cyclone.ERE.UMontreal.CA> Easy - a more general concept might require me to worry about extra things that I'm not interested in (like the number of dimensions, or the "rank" that J uses). These things are not interesting and can only cause problems by being there to make mistakes on. We will end u writing a wrapper that gives us the interface we want and hides the But it doesn't, as several decades of APL experience have shown. I know several APL users who are doing linear-algebra type things and are not even aware of the existence of higher-dimensional arrays. Your "wrapper" would only map some operations on "matrices" on exactly the same operations on "arrays" and leave out some others. That's just an unnecessary complication. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Sep 13 22:30:21 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Wed, 13 Sep 1995 17:30:21 -0400 Subject: [PYTHON MATRIX-SIG] Let's get going In-Reply-To: <9509132101.AA04255@nineveh.LCS.MIT.EDU> (message from James Hugunin on Wed, 13 Sep 95 17:01:31 -0400) Message-ID: <199509132130.RAA21754@cyclone.ERE.UMontreal.CA> 2) Should I be able to say fft(A) where fft is an in-place FFT routine? Should it be expected to modify the memory referred to by A, or only to return a brand-new matrix which corresponds to the fft of A? I'm not sure what the right answer to this one is. Both operations are useful in different contexts, so why not have both? One possibility would be to have a function fft(A) that returns a new array, and a method a.fft() to do an in-place FFT. Implementationally the function would just copy A and call the method. The same applies to inversion, factorization etc. It would be nice to have some consistency here - always functions for copies, always methods for in-place modifications. > I'm not sure what you mean by this. Surely, you aren't trying to > make: > > m=[[1,2,3],[4,5,6],[7,8,9]] > b=[11,22,33] > m=[1]=b > b[1]=99 > > cause m[1][1] to equal 99? Are you? Actually this is not such a silly question, because Python's lists behave exactly in this way. So the question is whether we want arrays to have reference semantice (like Python lists) or value semantics (like Python integers, floats, and complex numbers). I strongly recommend the latter - reference semantics for arrays quickly produce confusion, as I had to find out while playing with such an implementation in Smalltalk. on, whereas operations like concatenation return a new object. In that vein, I feel that m.byteswap() should operate in-place on m and cause the memory associated with m to be byte-swapped. This is as opposed to having it return a new matrix in with the same dimensions as m, but with all values byte-swapped. This depends on what you would use byte swapping for. In fact, I am not at all convinced of its utility. What would happen when such a byte-swapped matrix is accessed? Would all operations on it still be allowed? That would create an enormous implementation effort. If not, then a byte-swapped matrix would no longer be a matrix, because none of the matrix operations could be used on it. The only reason I can see for a byte swapping operation is during I/O, so that's where it should go (as a flag to I/O functions). In general, I have the impression that we get too much lost in implementational details. Let's first define arrays as an abstract data type in terms of the operations allowed on it from a user's point of view and *then* worry about implementational details. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Wed Sep 13 22:40:42 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Wed, 13 Sep 1995 17:40:42 -0400 Subject: [PYTHON MATRIX-SIG] Let's get going In-Reply-To: <9509132101.AA04255@nineveh.LCS.MIT.EDU> Message-ID: <199509132140.VAA24360@servrcolkr.cr.usgs.gov> On Wed, 13 Sep 95 17:01:31 -0400 James Hugunin said: > > I've got to run to a meeting, but I wanted to reply to your points. > > I'm not at all sure that we disagree on the implementation of slices. Let > me ask two simple questions. Let's say A is a slice taken from a matrix, > and B is a contiguous matrix. > > 1) Should I be able to say C = A+B, and have the obvious thing happen > (assuming that dimensions line up, etc.)? Yes. > I'd say yes, and I have a few performance optimizations to make this fast > in certain cases and nobody needs to worry about those. Cool. > > 2) Should I be able to say fft(A) where fft is an in-place FFT routine? > Should it be expected to modify the memory referred to by A, or only to > return a brand-new matrix which corresponds to the fft of A? > > I'm not sure what the right answer to this one is. My opinion, in general is that functions should return new objects, however, reasonable performance arguments can be made for at least some cases of having functions modify large matrices rather than creating new ones. In the case above, even if I had fft modify it's argument, I would still have it return the argument as a return value. The problem is that you want to be able to call routines that expect contigous data. Slice data, even for the cases you mentioned aren't like that. Slices could be implemented with their own contiguous data area that they keep in sync with the original matrices data, but I think this would complicate the implementation of slices beyound their benefit. Of course, this is only a problem when calling Fortran or C. Python functions (or C functions written for Python, which are much faster than python functions, but have many of the benefits) should nor present a problem. I suppose when calling a library function that wants to modify it's data, one could (within the glue code) do something equivalent to: def spam_glue(m): if type(m) is MatrixType: return spam(m) else: t=Matrix(m) r=spam(t) m[:]=t return r That is the (automagically generated) glue code could create a temporrary matrix as copy of the original data, call the function with the copy, and then slice-assign the modified copy back to the argument. Note this should work work with any sequence, such as a slice, that allows assignment from any arbitrary sequence. (Grrrrrr. Unfortunately, this won't work with list arguments, as the following code fails: a=[1,2,3] a[:]=(7,8,9) # This fails, but shouldn't, IMHO This should work!) I think I could work this behavior into FIDL, without much trouble. Hm. So perhaps allowing slices to be used as modifyable arguments might not be too bad, as long as the implementor is willing to do a little extra work. (In my case, the implementor is a program, so I don't mind if it works hard. ;) You still pay the performance penalty of making a copy, but at least functions that provide performance wins when used with matrices can still function correctly with slices. > > > > > Hm. Does this operate in-place or return a value? > > > > > > In-place. In fact, I am tempted to try and make all of the methods on > > > matrices operate "in-place" in order to be in line with list objects. > > > > I'm not sure what you mean by this. Surely, you aren't trying to > > make: > > > > m=[[1,2,3],[4,5,6],[7,8,9]] > > b=[11,22,33] > > m=[1]=b > > b[1]=99 > > > > cause m[1][1] to equal 99? Are you? > > Not at all! All I meant by this is that methods on list objects (like > insert or append) actually change the list object that they are operating > on, whereas operations like concatenation return a new object. In that > vein, I feel that m.byteswap() should operate in-place on m and cause the > memory associated with m to be byte-swapped. This is as opposed to having > it return a new matrix in with the same dimensions as m, but with all values > byte-swapped. Cool. Actually, I would use a naming convention to make this clear, so I might have both "byteswap" which modifies in place, and "byteswapped" which returns a new objects. (BTW Guido, I wish lists has a "sorted" operation that returned a new sorted list.) Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Wed Sep 13 22:52:16 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Wed, 13 Sep 1995 17:52:16 -0400 Subject: [PYTHON MATRIX-SIG] Let's get going In-Reply-To: <199509132130.RAA21754@cyclone.ERE.UMontreal.CA> Message-ID: <199509132152.VAA25125@servrcolkr.cr.usgs.gov> On Wed, 13 Sep 1995 17:30:21 -0400 Hinsen Konrad said: > > 2) Should I be able to say fft(A) where fft is an in-place FFT routine? > Should it be expected to modify the memory referred to by A, or only to > return a brand-new matrix which corresponds to the fft of A? > > I'm not sure what the right answer to this one is. > > Both operations are useful in different contexts, so why not > have both? One possibility would be to have a function fft(A) that > returns a new array, and a method a.fft() to do an in-place > FFT. Implementationally the function would just copy A and call > the method. The same applies to inversion, factorization etc. > It would be nice to have some consistency here - always functions > for copies, always methods for in-place modifications. > > > I'm not sure what you mean by this. Surely, you aren't trying to > > make: > > > > m=[[1,2,3],[4,5,6],[7,8,9]] > > b=[11,22,33] > > m=[1]=b > > b[1]=99 > > > > cause m[1][1] to equal 99? Are you? > > Actually this is not such a silly question, because Python's lists > behave exactly in this way. I didn't say it was a silly question. I know that lists behave this way, but making matrices behave this way complicates things alot. > So the question is whether we want > arrays to have reference semantice (like Python lists) or value > semantics (like Python integers, floats, and complex numbers). > I strongly recommend the latter - reference semantics for > arrays quickly produce confusion, as I had to find out while > playing with such an implementation in Smalltalk. You have to have some reference semantics to make m[i][j] work. Actuall, the proposal has reference semantics for reference (ie __getitem__) and copy semantics for element/slice assignment (__setitem__). > on, whereas operations like concatenation return a new object. In that > vein, I feel that m.byteswap() should operate in-place on m and cause the > memory associated with m to be byte-swapped. This is as opposed to having > it return a new matrix in with the same dimensions as m, but with all values > byte-swapped. > > This depends on what you would use byte swapping for. In fact, I am > not at all convinced of its utility. What would happen when such > a byte-swapped matrix is accessed? Would all operations on it > still be allowed? That would create an enormous implementation effort. > If not, then a byte-swapped matrix would no longer be a matrix, > because none of the matrix operations could be used on it. > The only reason I can see for a byte swapping operation is during > I/O, so that's where it should go (as a flag to I/O functions). > > > In general, I have the impression that we get too much lost in > implementational details. Let's first define arrays as an abstract > data type in terms of the operations allowed on it from a > user's point of view and *then* worry about implementational > details. I agree that we should try to focus on requirements, however, a very basic requirement of this type is it's ability to support interfacing with existing numerical routines, which imposes some significant restrictions on it's implementation. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From chris.chase@jhuapl.edu Wed Sep 13 23:34:08 1995 From: chris.chase@jhuapl.edu (Chris Chase S1A) Date: Wed, 13 Sep 1995 18:34:08 -0400 Subject: [PYTHON MATRIX-SIG] Re: Are we talking about one thing or two? In-Reply-To: <199509132117.RAA20989@cyclone.ERE.UMontreal.CA> References: <199509132046.QAA11169@python.org> <199509132117.RAA20989@cyclone.ERE.UMontreal.CA> Message-ID: <199509132233.SAA11749@python.org> Hinsen> Why specialise? That only limits the flexibility of the whole Hinsen> package. Why shouldn't it be possible to have "inverse" invert Hinsen> each two-dimensional slice of a three-dimensional array? There Hinsen> are applications for it, it doesn't cost any extra effort to Hinsen> do it, so why not do it? Why invent artificial restrictions? I did not mean that this could not be done. Inverse() in this context is an operation on square matrices (R^NxN). It is not a concept that I have experience generalizing to higher dimensions. Your example maps the inverse() function onto the two-dimensional slices. I think of this as a mapping operator that is applied to an indexed set of projections followed by application of inverse(). In general, you could apply any matrix function in this manner. You could do this for spectral factorization, diagonalization, etc. There is no need to redefine your matrix linear algebra functions. Instead you use a projection mapping/iterator (rank selector or whatever you want to call it) to apply the matrix function to each two-dimensional slice. This would seem the natural way to implement it. Many matrix functions such as inverse() would be best implemented by a call to an already existing and efficient C or FORTRAN library function. Perhaps it would be more elegant (and like APL) if the rank selection was a part of the inverse() parameter list but it can just as easily be a part of a projection mapping function that does the rank selection and then applies inverse(), i.e., a variation of the builtin map(): map(function, array, rank selection parameters) This avoids having to recode the same mechanism for each matrix function. It also provides a method for applying already coded lower dimensional operators (1D, 2D, 3D, whatever) to higher dimensional arrays without having to recode. James Hugunin made a comment about this - many of the mapping and reduction operations are generalizations that can be implemented in Python with calls to the appropriate external matrix-based function (or whatever the lower dimensional function is). Chris Chase ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Thu Sep 14 02:15:57 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Wed, 13 Sep 1995 21:15:57 -0400 Subject: [PYTHON MATRIX-SIG] Let's get going In-Reply-To: <199509132152.VAA25125@servrcolkr.cr.usgs.gov> (jfulton@usgs.gov) Message-ID: <199509140115.VAA03573@cyclone.ERE.UMontreal.CA> You have to have some reference semantics to make m[i][j] work. Actuall, the proposal has reference semantics for reference (ie __getitem__) and copy semantics for element/slice assignment (__setitem__). Indexing is really just syntactic sugar for getitem() and setitem(), so I wouldn't call that reference semantics. I agree that we should try to focus on requirements, however, a very basic requirement of this type is it's ability to support interfacing with existing numerical routines, which imposes some significant restrictions on it's implementation. Sure. But that's just one point on the list of requirements ;-) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Thu Sep 14 02:46:15 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Wed, 13 Sep 1995 21:46:15 -0400 Subject: [PYTHON MATRIX-SIG] Re: Are we talking about one thing or two? In-Reply-To: <199509132233.SAA27339@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Wed, 13 Sep 1995 18:34:08 -0400) Message-ID: <199509140146.VAA04451@cyclone.ERE.UMontreal.CA> I did not mean that this could not be done. Inverse() in this context is an operation on square matrices (R^NxN). It is not a concept that I have experience generalizing to higher dimensions. Your example maps the inverse() function onto the two-dimensional slices. I think of this as a mapping operator that is applied to an indexed set of projections followed by application of inverse(). In the rank concept that I outlined before, matrix inversion (and similar operations) is an operation with an intrinsic rank of 2. Its meaning for rank 3 arrays is thus an automatic consequence of the general rank rules and *not* an additional rule. You could specify an explicit rank of 1, reducing matrix inversion to scalar inversion. Specifying a higher rank would lead to an error message. Nevertheless, the possibility of specifying explicit ranks is important, since they can also be variables to be determined at runtime. There are applications where this is useful. And I would like to stress again that this generalization comes at no disadvantage to "two-dimensional only" users. slice. This would seem the natural way to implement it. Many matrix functions such as inverse() would be best implemented by a call to an already existing and efficient C or FORTRAN library function. Implmentation is a different story. I agree that using existing libraries makes sense. I would nevertheless prefer C libraries to Fortran libraries, as otherwise a Fortran compiler would become necessary to install Python, and not every Unix system comes with a Fortran compiler. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From guido@CNRI.Reston.VA.US Thu Sep 14 13:27:04 1995 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Thu, 14 Sep 1995 08:27:04 -0400 Subject: [PYTHON MATRIX-SIG] Why I won't add complex numbers to the base language Message-ID: <199509141227.IAA01106@monty> [This is the first of a series of small essays that treat several issues that have been brought up in the matrix sig. I'd like to make an effort to at least separate the threads by subject, if not putting some issues to rest altogether.] It's been proposed to add complex numbers to the base language, and when I muttered I didn't like that the response was "there's no mathematical reason not to." Let me rebutt. 1. Mathematically, there is at least one difference: complex numbers can't be compared (e.g. z1 < z2 is undefined when either has a nonzero imaginary party). Python will have to define this operation anyhow, since deep inside it requires that each object type defines comparison -- but it leads me to believe that there should be plenty of algorithms out there that wouldn't work if some of their parameters were complex numbers. (At least if the designer of the code didn't think about the possibility of complex numbers, the correctness of the code can't be guaranteed in general.) As long as real and complex are distinct types in Python, there is no fear that complex numbers will be introduced into the computation accidentally (e.g. by taking the sqrt() of a negative number or by a rounding error) -- they are passed in or used explicitly. 2. The available of GNU or other pseudo-free packages for transcendental functions on complex numbers is still a big restriction for Python's portability. Everything that is part of the *core* language has to be portable to every new platform. Since some of these platforms are non Unix, portability of code written by a crowd who assume gcc is available "everywhere" is questionable. There's another problem with GNU code that's only relevant for a certain group but which I care about anyway: people want to be able to embed Python (at least it's core) in commercial applications, and if there's a GNU licence attached to any part of that core, this is a problem. (Don't get me wrong -- I don't mind having Python extensions that use GNU stuff, as long as they can be taken out cleanly and easily by those who can't use GNU stuff, for whatever reason.) 3. The implementation of complex numbers as the only floating-point type in Python would add considerable complexity (no pun intended :-). I take that this should mean that if x is a real number (e.g. 0.0) and z is a complex number (e.g. sqrt(-1.0)), type(x) == type(z). This means that the representation in memory of Python complex number objects either has to contain the imaginary part at all times, or it has to contain a flag telling whether there's an imaginary part or not. Since there's no space in the standard object header for such a flag (at least not without changes that would have repercussions at many other places), it would have to be an additional byte (or more) in the complex number object itself. Say an object header plus malloc overhead is 12 bytes and a double precision float is 8 bytes. If we choose to have an additional flag it gets rounded up to 4 bytes (these are typical and optimistic numbers -- on some platforms your mileage may vary). So we have two choices: either add a flag, so numbers without an imaginary part are 12 (header) + 8(real) + 4 (flag) == 24 bytes and numbers with one are 12 + 8 + 4 + 8(imaginary) == 32 bytes, or all numbers are 12 + 8 + 8 == 28 bytes. Since most Python programs won't be using numbers with imaginary parts (Python being a general programming language), we may want to choose the version with a flag -- but this add at least one if statement to each C function operating on complex numbers (at least two for binary operators). Timewise, the fastest solution would be to always have the imaginary part there -- this would also make the code considerably cleaner. Conclusion: there's a big price to be paid by every Python user for having complex numbers as the only floating point type. Consider the alternative: write an extension module ("complex") that defines arithmetic on complex numbers (there are already examples of how to do this in Python; a C version is totally plausible) and another extensions module ("cmath") that defines transcendental functions on complex number. Now the price is only paid by those who use the feature. I think this solution can still lead to very clean code. I would even sanction a coding style where one writes "from math import *" to use the standard math functions, so it would require only a one-line change to user the complex versions instead ("from cmath import *"). --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From guido@CNRI.Reston.VA.US Thu Sep 14 13:44:20 1995 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Thu, 14 Sep 1995 08:44:20 -0400 Subject: [PYTHON MATRIX-SIG] Forget adding new operators to Python Message-ID: <199509141244.IAA01172@monty> [The second in a series of short essays on subjects raised in the Matrix discussion.] Let me be brief on this one: adding new operators (like ".*", "./" or "\") to the language is no-no. I'm very fond of the fact that nearly all graphical elements of the Python language correspond pretty closely to their use in "everyday life" (with the C language considered to be part of everyday life :-). I should point out that even though the semantics of operators are (almost) entirely defined by their operands, their syntax (including priorities) is not -- the parser doesn't know the operand types. Another, more practical reason is that adding a new operator requires changes to many components of the language implementation -- e.g. if ".*" were to be added as a new numeric operator, I'd have to make changes to every module that implements numbers, if only to add a NULL pointer. The only thing I regret is not having added "**" as an exponentiation operator -- this may happen someday (contributions accepted!). Oh, and there's also agreement that operators like "+=", "*=" should eventually be added; though not "++" and "--". --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From guido@CNRI.Reston.VA.US Thu Sep 14 14:20:28 1995 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Thu, 14 Sep 1995 09:20:28 -0400 Subject: [PYTHON MATRIX-SIG] A problem with slicing Message-ID: <199509141320.JAA01256@monty> [The third in a series of short essays on subjects raised in the Matrix discussion.] Here's a problem where I have neither a strong opinion nor a perfect solution... Jim Fulton proposes an elegant indexing syntax for matrix objects which doesn't require any changes to the language: M[i][j] references the element at column i and row j (or was that column j and row i? Never mind...). This nicely generalizes to slicing, so you can write: M[i][j1:j2] meaning the column vector at column i with row indices j1...j2-1. Unfortunately, the analogous expression for a row vector won't work: M[i1:i2][j] The reason for this is that it works by interpreting M as a sequence of columns (and it's all evaluated one thing at a time -- M[i][j] means (M[i])[j], and so on). M[i] is column i, so M[i][j] is the element at row j thereof. But slice semantics imply that of M is a sequence of X'es, then M[i1:j1] is still a sequence of X'es -- just shorter. So M[p:q][r] is really the same as M[p+r] (assuming r URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From forrest@rose.rsoc.rockwell.com Thu Sep 14 14:39:37 1995 From: forrest@rose.rsoc.rockwell.com (Dave Forrest) Date: Thu, 14 Sep 1995 08:39:37 -0500 Subject: [PYTHON MATRIX-SIG] Forget adding new operators to Python Message-ID: <9509141339.AA04618@feynman.rsoc.rockwell.com> This is a strong reason to provide two different interfaces - matrix and array. You can implement them both with the same code, but just provide a simple interface to matrix that restricts itself to two dimensions and does matrix multiplication with operator *. Array then acts "element-wise" and generalizes to any dimension,etc. Result: People who want stuff like my folks want are happy, people who want the more general stuff are happy, it's only implemented once, and no new operators are added to the language. dF ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Thu Sep 14 15:15:42 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Thu, 14 Sep 1995 10:15:42 -0400 Subject: [PYTHON MATRIX-SIG] A problem with slicing In-Reply-To: <199509141320.JAA01256@monty> Message-ID: <199509141415.OAA16059@servrcolkr.cr.usgs.gov> On Thu, 14 Sep 1995 09:20:28 -0400 Guido van Rossum said: > [The third in a series of short essays on subjects raised in the > Matrix discussion.] > > Here's a problem where I have neither a strong opinion nor a perfect > solution... > > Jim Fulton proposes an elegant indexing syntax for matrix objects > which doesn't require any changes to the language: > > M[i][j] > > references the element at column i and row j (or was that column j and > row i? Never mind...). Actually, it's element j of sub-matrix i. If M is a 2-d matrix, then you may choose to call submatrices either "rows" or "columns". I prefer "columns". > This nicely generalizes to slicing, so you can write: > > M[i][j1:j2] > > meaning the column vector at column i with row indices j1...j2-1. > > Unfortunately, the analogous expression for a row vector won't work: > > M[i1:i2][j] > > The reason for this is that it works by interpreting M as a sequence > of columns (and it's all evaluated one thing at a time -- M[i][j] > means (M[i])[j], and so on). M[i] is column i, so M[i][j] is the > element at row j thereof. But slice semantics imply that of M is a > sequence of X'es, then M[i1:j1] is still a sequence of X'es -- just > shorter. So M[p:q][r] is really the same as M[p+r] (assuming r > > One way out of this is to adopt the syntax > > M[i, j] > > for simple indexing. This would require only a minor tweaking of the > grammar I believe. In fact, this could be as simple as saying that the comma operator generates tuples inside of []s. This is: M[i,j] is equivalent to M[(i,j)]. or even: M[i,] is equivalent to M[(i,)] > This could be extended to support > > M[i1:i2, j] > M[i1:i2, j1:j2] > M[i, j1:j2] > > (and of course higher-dimensional equivalents). > > This would require considerable changes of the run-time architecture > of slicing and indexing, and since currently everything is geared > towards one-dimensional indexing/slicing, but I suppose it would be > doable. I agree. > (Funny how I'm accepting this possibility of changing the language > here, while I'm violently opposed to it for operator definitions. I Yeah. Strange even! ;-) > guess with adding operators there is no end to the number of new > operators you could dream up, so there would be no end to the change; > while here there's a clear-cut one-time change.) Hm. I really don't think this is a good idea. I don't really think we need M[i1:i2, j1:j2]. M[range(i1,i2),range(j1,j2)] is fine for me. Plus, it also allows: M[(1,3,5),(2,4,6)], in other words, we can simply allow a sequence of indexes for a dimension and then let range generate the desired sequence when we want a range. > > Of course adopting such a change would completely ruin any possbility > of using things like > > M[3, 4, 7] = [1, 10, 100] > > as roughly equivalent to > > M[3] = 1 > M[4] = 10 > M[7] = 100 > > but then again I'm not too fond of that anyway (as a matter of fact, > I'd oppose it strongly). > > > Some other things that I haven't completely followed through, and that > may cause complications for the theoretical foundation of it all: > > - Allowing M[i, j] for (multidimensional) sequence types would also > meaning that D[i, j] would be equivalent to D[(i, j)] for > dictionaries. I see no reason to support M[i,j] for arbitrary sequence types. I'd say that if a type wants to support multiple arguments to [], then it should provide mapping behavior and have the mapping implementation sniff for either an integer or a tuple argument and do the right thing. I am *vary much* against a language change to support this. > - Should M[i][j] still be equivalent to M[i, j]? Yes. M[i,j] is really a compact form of M[((i),(j))]. > - Now we have multidimensional sequence types, should be have a > multidimensional equivalent of len()? Some ideas: I'm against multi-dimension sequence types. 8-> > - len(a, i) would return a's length in dimension i; len(a, i) == len(a) > > - dim(a) (or rank(a)?) would return the number of dimensions > > - shape(a) would return a tuple giving a's dimensions, e.g. for a > 3x4 matrix it would return (3, 4), and for a one-dimensional > sequence such as a string or list, it would return a singleton > tuple: (len(a),). Unnecessary. Matrices can provide special methods for this. > - How about multidimensional map(), filter(), reduce()? > > - map() seems easy (except there seems to be no easy way to specify > the rank of the operator): it returns a similarly shaped > multidimensional object whose elements are the function results for > the corresponding elements of the input matrix > > - filter() is problematic since the columns won't be of the same > length > > - reduce()??? -- someone who knows APL tell me what it should mean There have been a number of proposals for generic functions that operate over matrices in some fashion. I have not had time to digest them yet. Stay tuned. :-) (Geez, I really need to get back to my day job.) > - Multidimensional for loops? Or should for iterate over the first > dimension only? What is wrong with nested for loops. Ee-gads, what's gotten into you? :-] > One sees there are many potential consequences of a seemingly simple > change -- Simple? I agree that enabling the tuplefication opertor, ",", in []s is simple, but not adding mult-dimensional behavior to sequences. > that's why I insist that language changes be thought through > in extreme detail before being introduced... I really don't see any reason why the matrix type should require language changes (aside from the minor impact of the tuplefication operator). Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From guido@CNRI.Reston.VA.US Thu Sep 14 15:39:00 1995 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Thu, 14 Sep 1995 10:39:00 -0400 Subject: [PYTHON MATRIX-SIG] A problem with slicing In-Reply-To: Your message of "Thu, 14 Sep 1995 10:15:42 EDT." <199509141415.OAA16059@servrcolkr.cr.usgs.gov> References: <199509141415.OAA16059@servrcolkr.cr.usgs.gov> Message-ID: <199509141439.KAA01464@monty> > I really don't see any reason why the matrix type should require > language changes (aside from the minor impact of the tuplefication > operator). I guess one of my points was that, given the desire for a fairly consistent design, allowing tuples as indices is *not* a minor change... (More later.) --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Thu Sep 14 15:50:25 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Thu, 14 Sep 1995 10:50:25 -0400 Subject: [PYTHON MATRIX-SIG] A problem with slicing In-Reply-To: <199509141439.KAA01464@monty> Message-ID: <199509141450.OAA17703@servrcolkr.cr.usgs.gov> On Thu, 14 Sep 1995 10:39:00 -0400 Guido van Rossum said: > > I really don't see any reason why the matrix type should require > > language changes (aside from the minor impact of the tuplefication > > operator). > > I guess one of my points was that, given the desire for a fairly > consistent design, allowing tuples as indices is *not* a minor > change... I'm not suggesting that tuples should be allowed as indexes to sequence types. Tuples are *already* allowed as indexes to mapping types. I propose that matrices should provide *both* sequence and mapping behavior, where the mapping behavior is used to support matrix slicing. This requires *no* change to the language. The only *minor* change proposed, which I could live without is to allow the "," to generate tuples inside of []s, just as it does outside of []. In fact, I view the current non-recognition of tuples in []s as an inconsistency. For example: a=1,2,3 is equivalent to: a=(1,2,3) and: for spam in 1,2,3: ... is equivalent to: for spam in (1,2,3): ... so why isn't: a[1,2,3] equivalent to: a[(1,2,3) Note that this change is cosmetic (like making modules callable ;-), and is not *needed* for the matrix proposal. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From guido@CNRI.Reston.VA.US Thu Sep 14 15:55:32 1995 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Thu, 14 Sep 1995 10:55:32 -0400 Subject: [PYTHON MATRIX-SIG] A problem with slicing In-Reply-To: Your message of "Thu, 14 Sep 1995 10:50:25 EDT." <199509141450.OAA17703@servrcolkr.cr.usgs.gov> References: <199509141450.OAA17703@servrcolkr.cr.usgs.gov> Message-ID: <199509141455.KAA01555@monty> > The only *minor* change proposed, which I could live without is to > allow the "," to generate tuples inside of []s, just as it does > outside of []. In fact, I view the current non-recognition of tuples > in []s as an inconsistency. For example: [...] > so why isn't: > > a[1,2,3] > > equivalent to: > > a[(1,2,3)] Because there's also a[1:2] while there is no equivalent a(1:2) I could either tweak the priorities so that a[1,2:3,4] is parsed as a[(1,2) : (3,4)] or so that it is parsed as a[1, (2:3), 4] but neither appears very natural to me. I guess my problem is that ":" and "," have "fuzzy" priorities, and while everybody agrees that e.g. "*" binds tighter than "+", if you ask a few people in the street, or even computer programmers, you'd get confused answers. --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Thu Sep 14 16:13:03 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Thu, 14 Sep 1995 11:13:03 -0400 Subject: [PYTHON MATRIX-SIG] A problem with slicing In-Reply-To: <199509141455.KAA01555@monty> Message-ID: <199509141513.PAA18008@servrcolkr.cr.usgs.gov> On Thu, 14 Sep 1995 10:55:32 -0400 Guido van Rossum said: > > The only *minor* change proposed, which I could live without is to > > allow the "," to generate tuples inside of []s, just as it does > > outside of []. In fact, I view the current non-recognition of tuples > > in []s as an inconsistency. For example: > [...] > > so why isn't: > > > > a[1,2,3] > > > > equivalent to: > > > > a[(1,2,3)] > > Because there's also > > a[1:2] > > while there is no equivalent > > a(1:2) > > I could either tweak the priorities so that > > a[1,2:3,4] > > is parsed as > > a[(1,2) : (3,4)] Can't be, ":" wants integers. > or so that it is parsed as > > a[1, (2:3), 4] Can't be, (2:3) is not a valid expression, so it can't yield a valid element of the tuple. > but neither appears very natural to me. Good. :-) > I guess my problem is that ":" and "," have "fuzzy" priorities, and > while everybody agrees that e.g. "*" binds tighter than "+", if you > ask a few people in the street, or even computer programmers, you'd > get confused answers. But ":" only makes sense for sequences and "," only makes sense for mappings, so ":" and "," should never appear together in []s. I don't think this is a precedence issue. I see your point that ":" complicates things a bit. You not only have to recognize ",", but you have to make sure that it is not used in conjunction with ":". Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Thu Sep 14 17:42:21 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Thu, 14 Sep 1995 12:42:21 -0400 Subject: [PYTHON MATRIX-SIG] Why I won't add complex numbers to the base language In-Reply-To: <199509141227.IAA01106@monty> (message from Guido van Rossum on Thu, 14 Sep 1995 08:27:04 -0400) Message-ID: <199509141642.MAA06980@cyclone.ERE.UMontreal.CA> 1. Mathematically, there is at least one difference: complex numbers can't be compared (e.g. z1 < z2 is undefined when either has a nonzero imaginary party). Python will have to define this operation anyhow, So that would lead to an exception. Nothing exceptional ;-), since there are other operations that produce exceptions (like division by zero). -- but it leads me to believe that there should be plenty of algorithms out there that wouldn't work if some of their parameters were complex numbers. (At least if the designer of the code didn't think about the possibility of complex numbers, the correctness of the code can't be guaranteed in general.) As long as real and complex are True, but the same problem occurs with complex numbers defined in packages. Since Python's variables and function parameters are not types, nothing prevents me from passing complex numbers to an algorithm not designed for them. 2. The available of GNU or other pseudo-free packages for transcendental functions on complex numbers is still a big restriction for Python's portability. Everything that is part of the *core* language has to be portable to every new platform. Since some of That should not be a problem. All codes for complex numbers that I am aware of handle complex-number arithmetic in terms of (portable) real arithmetic. An implementation of complex numbers in portable C is just as possible as in portable Python. another problem with GNU code that's only relevant for a certain group but which I care about anyway: people want to be able to embed Python (at least it's core) in commercial applications, and if there's a GNU licence attached to any part of that core, this is a problem. (Don't There are non-GNU complex libraries, and even writing a new one is only a small task. 3. The implementation of complex numbers as the only floating-point type in Python would add considerable complexity (no pun intended :-). I take that this should mean that if x is a real number (e.g. 0.0) and z is a complex number (e.g. sqrt(-1.0)), type(x) == type(z). This Indeed. means that the representation in memory of Python complex number objects either has to contain the imaginary part at all times, or it has to contain a flag telling whether there's an imaginary part or Maybe. I don't know much about the internals of Python, I am just a simple user... But I know that most APL implementation use different internal types, but don't make this distinction visible to the user. In fact, they don't even distinguish between integers and reals. APL has only two types: character and numeric, although numeric has four internal representations (bits, integers, reals, and complex numbers). Consider the alternative: write an extension module ("complex") that defines arithmetic on complex numbers (there are already examples of how to do this in Python; a C version is totally plausible) and another extensions module ("cmath") that defines transcendental functions on complex number. Now the price is only paid by those who That would indeed be a good solution (I come to like Python's module system more and more every day). I'll explore that when I find some time... I don't see any real problem, everything necessary for a convenient implementation should be there (i.e. coercion from realfloat to complex, such that real constants can be entered conveniently). It would be nice to have a more convenient notation for complex constants, but that is not really essential. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Thu Sep 14 18:37:44 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Thu, 14 Sep 1995 13:37:44 -0400 Subject: [PYTHON MATRIX-SIG] Forget adding new operators to Python In-Reply-To: <199509141244.IAA01172@monty> (message from Guido van Rossum on Thu, 14 Sep 1995 08:44:20 -0400) Message-ID: <199509141737.NAA09446@cyclone.ERE.UMontreal.CA> The only thing I regret is not having added "**" as an exponentiation operator -- this may happen someday (contributions accepted!). Oh, and there's also agreement that operators like "+=", "*=" should eventually be added; though not "++" and "--". Great. I am waiting ;-) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Thu Sep 14 19:21:02 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Thu, 14 Sep 1995 14:21:02 -0400 Subject: [PYTHON MATRIX-SIG] A problem with slicing In-Reply-To: <199509141320.JAA01256@monty> (message from Guido van Rossum on Thu, 14 Sep 1995 09:20:28 -0400) Message-ID: <199509141821.OAA11773@cyclone.ERE.UMontreal.CA> One way out of this is to adopt the syntax M[i, j] for simple indexing. This would require only a minor tweaking of the grammar I believe. This could be extended to support M[i1:i2, j] M[i1:i2, j1:j2] M[i, j1:j2] (and of course higher-dimensional equivalents). How about allowing each of the index expressions to be an array (of integers) itself? That gives an enormous flexibility (consider that the indexing array could be a very complicated expression!). Of course adopting such a change would completely ruin any possbility of using things like M[3, 4, 7] = [1, 10, 100] as roughly equivalent to M[3] = 1 M[4] = 10 M[7] = 100 but then again I'm not too fond of that anyway (as a matter of fact, I'd oppose it strongly). According to my proposal, that could be done as M[[3,4,7]] = [1, 10, 100] - Should M[i][j] still be equivalent to M[i, j]? Basically the question is whether M[i] should be allowed (and with what meaning) if M has more than one dimension. Maybe it is better not to allow it at all, as it creates some confusion due to the imperfect analogy of arrays with (nested) sequences. - Now we have multidimensional sequence types, should be have a multidimensional equivalent of len()? Some ideas: - len(a, i) would return a's length in dimension i; len(a, i) == len(a) - dim(a) (or rank(a)?) would return the number of dimensions - shape(a) would return a tuple giving a's dimensions, e.g. for a 3x4 matrix it would return (3, 4), and for a one-dimensional sequence such as a string or list, it would return a singleton tuple: (len(a),). How about having len() return the total number of elements? shape() would be necessary anyway, and your proposed len(a,i) would just be equivalent to shape(a)[i]. Similarly, dim(a) would be len(shape(a)). APL actually has only shape, which returns a one-dimensional array of the lengths along each axis, i.e. a zero-length vector in case of a scalar. Then dim(a) is just shape(shape(a)). - How about multidimensional map(), filter(), reduce()? map() is no problem, as you pointed out. Actually, if one adopts a J-style rank concept, it wouldn't even be necessary. filter() and reduce() should operate along a specified axis. filter() would thereby return an array of equal dimension, but shorter along one axis, and reduce() would return an array with the dimension reduced by one. - Multidimensional for loops? Or should for iterate over the first dimension only? Again, iteration over an arbitrary axis would be useful. Iteration over all elements could be handled by map(). ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@mama-bear.lcs.mit.edu Thu Sep 14 19:41:39 1995 From: jjh@mama-bear.lcs.mit.edu (James Hugunin) Date: Thu, 14 Sep 95 14:41:39 -0400 Subject: [PYTHON MATRIX-SIG] A problem with slicing References: <199509141513.PAA18008@servrcolkr.cr.usgs.gov> Message-ID: <9509141841.AA04493@nineveh.LCS.MIT.EDU> I'm eager to see what Guido has to add in his "(More later.)", because at the moment I'm slightly confused by the trend in this discussion. If the point is that the proposed slicing semantics are a bad idea because they'd require basic changes to the language, then I'd answer that no changes are in fact necessary. If instead he is offering to open up this portion of the language to change in the hopes of creating a more "consistent design", then I think there are some interesting possibilities. In my opinion, the slicing semantics for matrices as currently proposed seem reasonable and can be implemented with zero changes to the core python language by using getitem and setitem for mapping objects (I know this because I've implemented them within the matrix module that way). However, there is a good argument to be made saying that - a[:3, :4] is a lot more consistent with the current implementation of lists than - a[(range(3), range(4))] In addition, something like - a[3:, 4:] is a lot clearer than - a[(range(3,shape(a[0]), range(4, shape(a)[1]))] However, one thing that I really like about the proposed indexing scheme is that I can say a[((1,3,5), (2,4,6))] and get back the desired non-contiguous chunk of the array. This is occasionally very useful from an efficiency point of view, and unfortunately, efficiency is something that needs to be kept in mind for large numeric arrays. If I had my wish, I would change the syntax of python so that start:step:stop, or start:stop was a shorthand for creating a range object (with some reasonable way of specifying start or stop as defaults). This would not need to change the semantics of basic sequence indexing, as these could be handled as a special case. I think that ":" should obviously be of higher precedence than ",". There are no cases that I can think of where it would be a reasonable thing to have a tuple as one element in a range. With this change, and the I feel completely reasonable proposal that "," be allowed for tuple creation within an index, then multidimensional arrays could be made to behave in a manner completely consistent with lists. And this would require minimal changes to the run-time architecture of slicing and indexing. On a slightly different track: I've been playing with another technique for indexing a matrix, borrowed from matlab. I've implemented indexing matrices with a matrix of booleans (integers) that is the same size as the matrix being indexed (this only makes sense for a setvalue). This is trivially done using the mapping semantics, and combined with some matrix comparision operators I've found this quite useful. ie. a.gt(x) is a matrix less than operator. (I'd prefer to use "a < x", but the point of this is to explore what can be done without changing the core python language that all of us love so much). a = matrix([1,2,3,4,5,4,3,2,1]) a[a.gt(3)] = 3 print a --> [1,2,3,3,3,3,3,2,1] I don't think that this is a substitute for any of the indexing methods currently being discussed, but I want to make sure that all candidate indexing methods are brought up as early on in the discussion as possible in order to ultimately create a coherent (rather than haphazard) indexing system. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Thu Sep 14 23:46:58 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Thu, 14 Sep 1995 18:46:58 -0400 Subject: [PYTHON MATRIX-SIG] A problem with slicing In-Reply-To: <9509141841.AA04493@nineveh.LCS.MIT.EDU> (message from James Hugunin on Thu, 14 Sep 95 14:41:39 -0400) Message-ID: <199509142246.SAA26506@cyclone.ERE.UMontreal.CA> I've been playing with another technique for indexing a matrix, borrowed from matlab. I've implemented indexing matrices with a matrix of booleans (integers) that is the same size as the matrix being indexed (this only makes sense for a setvalue). This is trivially done using the mapping semantics, and combined with some matrix comparision operators I've found this quite useful. Indeed. But how does it work for higher-dimensional arrays? APL provides a similar functionality using an operator called "compress". Its boolean argument is always one-dimensional and it a works along a specified axis. If used on the right-hand side of an expression, it is simply filter(). In APL2 it may also be used on the left hand side of an assignment, as btw can all expressions that evaluate to a subset of elements of an array. This is an extremely powerful feature, but so complicated to implement that to my knowledge only IBM's mainframe version does it without restrictions. In summary, I like your proposal, adding that it ought to work along one (arbitrary) axis for higher-dimensional arrays. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@mama-bear.lcs.mit.edu Fri Sep 15 16:29:09 1995 From: jjh@mama-bear.lcs.mit.edu (James Hugunin) Date: Fri, 15 Sep 95 11:29:09 -0400 Subject: [PYTHON MATRIX-SIG] A problem with slicing References: <199509142246.SAA26506@cyclone.ERE.UMontreal.CA> Message-ID: <9509151529.AA01177@nineveh.LCS.MIT.EDU> > I've been playing with another technique for indexing a matrix, borrowed > from matlab. I've implemented indexing matrices with a matrix of booleans > (integers) that is the same size as the matrix being indexed (this only > makes sense for a setvalue). This is trivially done using the mapping > semantics, and combined with some matrix comparision operators I've found > this quite useful. > > Indeed. But how does it work for higher-dimensional arrays? > > APL provides a similar functionality using an operator called > "compress". Its boolean argument is always one-dimensional and it a > works along a specified axis. If used on the right-hand side of an > expression, it is simply filter(). In APL2 it may also be used on > the left hand side of an assignment, as btw can all expressions > that evaluate to a subset of elements of an array. This is an > extremely powerful feature, but so complicated to implement that > to my knowledge only IBM's mainframe version does it without > restrictions. My initial idea for higher-dimensional arrays was to use a matrix of booleans that was the size of the entire higher-d array being indexed, this fits naturally with the outputs from matrix comparision operators. ie. a = [[1,2,3], [4,5,6]] --> a.gt(2) == [[0,0,1], [1,1,1]] a[a.gt(2)] = 99 --> a == [[1,2,99], [99,99,99]] However, I agree that it should be possible to rationally apply this to a subset of the matrix. Something like: a = [[1,2,3], [4,5,6]] --> sum(a).gt(7) == [0,1] a[sum(a).gt(7), :] = 99 --> a == [[1,2,3], [99,99,99]] This example suggests that it should be possible to mix these boolean indices with range indices, and with single integer indices. While this all sounds reasonable (and extremely powerful) to me, I have to confess that a few of my feeping creaturism warning lights go off when I sit back and look at it. I'd love to hear more impressions on this, pro and con. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Fri Sep 15 17:28:42 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Fri, 15 Sep 1995 12:28:42 -0400 Subject: [PYTHON MATRIX-SIG] A problem with slicing In-Reply-To: <9509151529.AA01177@nineveh.LCS.MIT.EDU> (message from James Hugunin on Fri, 15 Sep 95 11:29:09 -0400) Message-ID: <199509151628.MAA13937@cyclone.ERE.UMontreal.CA> My initial idea for higher-dimensional arrays was to use a matrix of booleans that was the size of the entire higher-d array being indexed, this fits naturally with the outputs from matrix comparision operators. ie. a = [[1,2,3], [4,5,6]] --> a.gt(2) == [[0,0,1], [1,1,1]] a[a.gt(2)] = 99 --> a == [[1,2,99], [99,99,99]] That's nice on the left hand side of an assignment, but what is the value of a[a.gt(2)] in an expression? It can't be an array! If all you want is some form of selective assignment, that can be done with mapping, although you have to type a bit more. You could also achieve the above result (admittedly less efficiently) with a = (not a > 2)*a + (a > 2)*99. Therefore I am not so sure whether your proposed feature is that important, except if it were very common. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From chris.chase@jhuapl.edu Fri Sep 15 18:26:17 1995 From: chris.chase@jhuapl.edu (Chris Chase S1A) Date: Fri, 15 Sep 1995 13:26:17 -0400 Subject: [PYTHON MATRIX-SIG] A problem with slicing In-Reply-To: <199509151628.MAA13937@cyclone.ERE.UMontreal.CA> References: <9509151529.AA01177@nineveh.LCS.MIT.EDU> <199509151628.MAA13937@cyclone.ERE.UMontreal.CA> Message-ID: <199509151725.NAA20418@python.org> Hinsen> a = [[1,2,3], [4,5,6]] --> a.gt(2) == [[0,0,1], [1,1,1]] Hinsen> a[a.gt(2)] = 99 --> a == [[1,2,99], [99,99,99]] Hinsen> That's nice on the left hand side of an assignment, but what Hinsen> is the value of a[a.gt(2)] in an expression? It can't be an Hinsen> array! It be an array if you define a selection type indexing, e.g., a.select(a.gt(2)) where select is a method that whose argument has the same dimensions as "a" and returns elements of "a" that correspond to "true" elements of a.gt(2). The language Octave allows for this type of selection indexing. There are several types of indexing schemes that I have seen for multi-demensional arrays. Without additions to the syntax, only one could be used with "[...]" and the others would have to be explicit method calls. I have a number of possible suggestions about different types of array indexing which I will post later. Chris Chase ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Fri Sep 15 19:25:10 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Fri, 15 Sep 1995 14:25:10 -0400 Subject: [PYTHON MATRIX-SIG] A problem with slicing In-Reply-To: <199509151725.NAA20418@python.org> (message from Chris Chase S1A on Fri, 15 Sep 1995 13:26:17 -0400) Message-ID: <199509151825.OAA21099@cyclone.ERE.UMontreal.CA> Hinsen> That's nice on the left hand side of an assignment, but what Hinsen> is the value of a[a.gt(2)] in an expression? It can't be an Hinsen> array! It be an array if you define a selection type indexing, e.g., a.select(a.gt(2)) where select is a method that whose argument has the same dimensions as "a" and returns elements of "a" that correspond to "true" elements of a.gt(2). But *how* does it return the elements that correspong to "true"? Specifically, what is the shape of the array returned? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From graham@fishnet.net Wed Sep 20 01:31:31 1995 From: graham@fishnet.net (Graham Hughes) Date: Tue, 19 Sep 1995 17:31:31 -0700 Subject: [PYTHON MATRIX-SIG] Assorted remarks Message-ID: <199509191730.RAA19637@big.fishnet.net> Another compendium of responses to various messages. --- Guido van Rossum --- >- Allowing M[i, j] for (multidimensional) sequence types would also >meaning that D[i, j] would be equivalent to D[(i, j)] for >dictionaries. Sounds good to me. Alternatively, we could interpret D[i,j] as D[i][j]... After all, basically that's what M[i,j] is. Might warrant consideration. >- Should M[i][j] still be equivalent to M[i, j]? Probably. While slicing would be difficult as your earlier remarks point out, it seems that allowing this will allow the use of sequences of sequences. Actually, I'm not entirely convinced that M[i:j][k] is *not* impossible; give me a week or so to decide on this, and I may come up with something that doesn't require any changing of the core language... >- Now we have multidimensional sequence types, should be have a >multidimensional equivalent of len()? Some ideas: Actually, I feel that for a multidimensional sequence, len() should behave the same way it always had. Allowing for greater dimensions will give greater flexibility, yes, but it may also break `older' functions that assume that what's being passed to them is a 2D array. This is mainly because len() is defined to return an integer, sadly, and not one of our magic sequences; if it did return the magic sequence, we would be able to totally ignore how many dimensions we have. > - len(a, i) would return a's length in dimension i; len(a, i) == len(a) As to extending len(), I don't think it's even necessary. Just do len(a[1]). Short, simple, requires nothing out of the ordinary, *and* will even return a 1 or something like that *if* [] returns an array of the proper rank (a 3D will return a rank 2 array, 2 will return 1, etc.). This will not in fact cause problems, because all operations on arrays should be defined to be 'elementwise' by default. This means we can pass a 3D array to something expecting a matrix, and it won't make a difference. APL/J ranking does this automatically, and given a little time I can probably work up an example matrix that will do that too. > - dim(a) (or rank(a)?) would return the number of dimensions If we have shape(), that's not terribly useful either. Just do len(shape(a)). > - shape(a) would return a tuple giving a's dimensions, e.g. for a > 3x4 matrix it would return (3, 4), and for a one-dimensional > sequence such as a string or list, it would return a singleton > tuple: (len(a),). Why a tuple, and not an array of rank 1? The array can be manipulated more effectively than the tuple... >- How about multidimensional map(), filter(), reduce()? Not needed, really; see following remarks. > - map() seems easy (except there seems to be no easy way to specify > the rank of the operator): it returns a similarly shaped > multidimensional object whose elements are the function results for > the corresponding elements of the input matrix If we use APL/J ranks, this is mostly taken care of automagically. The function will request what it wants from the matrix, and the matrix can do whatever it needs to with that. Ah, but let's suppose you want the dimensions in a different order; let's suppose you've got [ [ [1,2], [2,3] ] , [ [3,4], [4,5] ] ] (2x2x2 array). Normally, map would invert over [ [1,2], [2,3] ] and [ [3,4], [4,5] ]. But suppose you want it to behave differently; you want to 'rotate' the matrix. I argue that this is an important enough operation that it should be divorced from map, reduce, and filter, and brought into its own (notably unlike APL, BTW; APL had a special modifier on *every* sequence operation to do this. That is inefficient :). > - filter() is problematic since the columns won't be of the same > length filter() should work just like map(); i.e. just like it already does. The function that filter calls should know exactly what it wants out of the array passed; there's no reason for filter() to have this information. > - reduce()??? -- someone who knows APL tell me what it should mean Oh, ok; this follows right along the lines of the above stuff; it should work like it always did. Reducing a 2d array will simply have a grouping of 1d arrays being passed to the function. Since the function knows what it wants (sense a common theme here?), the overall effect will be transparent. I really *need* to get that sample matrix written up to illustrate this... >- Multidimensional for loops? Or should for iterate over the first >dimension only? Same as above, i.e. first dimension only. If you want a different dimension, rotate() the matrix. BTW, notice that if (very strong if) I can get slicing to work properly, *none* of this would require altering the language. Even if I can't, they shouldn't require any alterations beyond the original [i,j] change. Secret bonus; we can use [i,j] to signify non-continguous slices. While you said you're not fond of this earlier (not quoted), and I agree that the example given was a bit strange, this can give a great deal of additional power to the matrix class. More on this later. --- forrest@rose.rsoc.rockwell.com (Dave Forrest) --- >This is a strong reason to provide two different interfaces - matrix >and array. You can implement them both with the same code, but just >provide a simple interface to matrix that restricts itself to two >dimensions and does matrix multiplication with operator *. Array then >acts "element-wise" and generalizes to any dimension,etc. Not really. I'll reiterate what I've said before, that you don't need it because all your functions will work fine with whatever dimension thingy we pass it, with a bumper; *if* we do two different things, and you discover that your original code needs to be modified somehow, or someone gets confused and passes mixed arguments (matrix to array, or vice versa), all of a sudden everything gets thoroughly confused. This is not a problem in standard Python, since you can't add different types, but it might crop up here, particularly if we write most of the interface in Python (which is entirely possible, and possibly desirable). Basically, my main argument against splitting it up is that it adds unneeded complexity for a relatively small bonus (just write a matrix_multiply() function that expects rank 2 arrays, and everything works peachy), and that it would segregate the two entirely too much. For example, I would have no way of taking my array and doing parallel matrix multiplications on it (i.e. multiplying three sets of matrices at the same time), because while I can do the parallelism in 'array' class and the matrix multiplication in the 'matrix' class, I would have no way to fold them together. As a result, I would likely write a matrix_multiply() for the general matrix, and _entirely_ sidestep the submatrix class. If you ever need to multiply several matricies together (and you probably do) this parallelism is much more efficient (because you spend more time in C code multiplying than in Python figuring out which one to do next) than swapping matrices around. Examine your code; you probably do something like (dropping into C for a while) matrix m[4]; int i; for (i = 0; i < 4; i++) load_matrix(m[i]); matrix_multiply(m[0],m[1]); matrix_multiply(m[2],m[3]); Notice that you're doing two multiplications. In C, there is no reason to do parallelism (for efficiency sake) because you're not kicking back up into an interpreter every time you multiply something. The parallel matrix would work exactly the same way (would look a little different, I think, but not a real problem), but would only kick in the interpreter *once*. This, for 5, 10, 200 matrices is a *real* timesaver. This point is why APL is so fast, even though it is interpreted; it spends most of its time in C, not an interpreter. --- Graham Graham Hughes Home page http://www.fishnet.net/~graham/ ``I think it would be a good idea.'' -- Mahatma Ghandi, when asked what he thought of Western civilization finger for PGP public key. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From graham@fishnet.net Wed Sep 20 06:18:43 1995 From: graham@fishnet.net (Graham Hughes) Date: Tue, 19 Sep 1995 22:18:43 -0700 Subject: [PYTHON MATRIX-SIG] Slicing Message-ID: <199509192217.WAA29411@big.fishnet.net> I think I figured out how to get the slicing without any modification of the existing Python base to work. Earlier it was said that m[i:j][k] wouldn't work because of the precedence rules. I think one way of avoiding these problems is to look at the slicing this way: Assume for the moment that sequences of sequences are stored by row, i.e. like C. To get the slicing to work properly, we have to slice by *columns*. As an example, suppose we have [[1,2,3],[2,3,4]], or 1 2 3 2 3 4 if we accept the original premise. The matrix class will store it this way internally. However, every interaction with the user *must* make the matrix look like this: 1 2 2 3 3 4 Given this, slicing is relatively easy, and [:] will return a transpose of the internal storage. So m[0:1] will return in original form [[1],[2],[3]] or 1 2 3 This works great for slices. However, assigning to individual elements is a tad tricker... Note that a special case for single dimension arrays will simply do standard slicing, as the effect is the same. Graham Graham Hughes Home page http://www.fishnet.net/~graham/ ``I think it would be a good idea.'' -- Mahatma Ghandi, when asked what he thought of Western civilization finger for PGP public key. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Wed Sep 20 14:18:08 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Wed, 20 Sep 1995 09:18:08 -0400 Subject: [PYTHON MATRIX-SIG] Slicing In-Reply-To: <199509192217.WAA29411@big.fishnet.net> Message-ID: <199509201318.NAA29865@servrcolkr.cr.usgs.gov> On Tue, 19 Sep 1995 22:18:43 -0700 Graham Hughes said: > I think I figured out how to get the slicing without any modification of the > existing Python base to work. The original proposal *already* says how to get slicing without any modifications to the core language. > Earlier it was said that m[i:j][k] wouldn't work because of the precedence > rules. I think one way of avoiding these problems is to look at the slicing > this way: > > Assume for the moment that sequences of sequences are stored by row, i.e. > like C. See my earlier post. Matrices are stored by sub-matrix. Storage by "rows" or by "columns" is a question of interpretation. Under my system of interpretation, matrices are stored by column (in Fortran, C, and the proposed Python extension). > To get the slicing to work properly, we have to slice by *columns*. > As an example, suppose we have [[1,2,3],[2,3,4]], or > > 1 2 3 > > 2 3 4 What does this mean? > if we accept the original premise. The matrix class will store it this way > internally. However, every interaction with the user *must* make the matrix > look like this: > > 1 2 > > 2 3 > > 3 4 Do you mean like: [[1,2],[2,3],[3,4]]? > Given this, slicing is relatively easy, How? > and [:] will return a transpose of > the internal storage. Why? This would be inconsistent with other sequence types. > So m[0:1] will return in original form [[1],[2],[3]] or > > 1 > > 2 > > 3 What has this gained? (Perhaps an example with greater than two rows and columns would be better?) Let's be clear about the goal. Goven a matrix that looks (by rows or by columns, take your pick) like this: 0 10 20 30 1 11 21 31 2 12 22 32 3 13 23 33 one wants to be able to access a matrix that looks like this: 11 21 12 22 Some even want to be able to access a matrix that looks like this: 1 21 3 23 or even this: 0 30 3 33 and so on. And, of course, this needs to generalize to higher dimensions. Also, modification to this access should be reflected in the matrix being accessed. I don't see how switching indexes solves this problem. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Sep 27 02:38:49 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Tue, 26 Sep 1995 21:38:49 -0400 Subject: [PYTHON MATRIX-SIG] J-style arrays in Python Message-ID: <199509270138.VAA15833@cyclone.ERE.UMontreal.CA> To bring back some life into this discussion group, I'll distribute a Python implementation of J-like arrays, to give those unfamiliar with J a chance to become familiar with its array and rank system. And of course it is also usable, as long as the arrays don't become too big. This is a first test version; many important functions are still missing, others (like output formatting) need substantial improvement, and many could probably be made faster with some thought. To make (my) life simpler, I tried to stick to J's conventions as much as possible (but hopefully without violating Python traditions). I am not claiming that this is semantically the best way to implement arrays, but it is a start. Some general remarks: - Since my implementation uses nested lists to represent arrays, the elements can be arbitrary objects. - Like arrays in J, my arrays are immutable, i.e. there is no provision for changing individual elements. The reason for making arrays immutable in J was that J is half-way to being a functional language (it is not a pure functional language, but many substantial problems can easily be solved in a functional way). I have never missed element assignment, but probably there are some good applications... - All functions are implemented as external functions, not as methods. The main reason is that at first I could not think of a way to implement methods with variable rank, although later I figured out how to do this (in the same way as I implemented reduction). I'll send two Python files; the first is the Array module itself, and the second a kind of simple tutorial. Let me know if anything is unclear. And let me know what you think of the whole implementation! ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Sep 27 02:39:11 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Tue, 26 Sep 1995 21:39:11 -0400 Subject: [PYTHON MATRIX-SIG] Array.py Message-ID: <199509270139.VAA15839@cyclone.ERE.UMontreal.CA> # J-style array class # Arrays are represented by a scalar or a list, possibly containing # other lists in case of higher-rank arrays. Array rank is limited # only by the user's patience. # Send comments to Konrad Hinsen import math, string ###################################################################### # Error type ArrayError = 'ArrayError' ###################################################################### # Various functions that do the real work. Classes follow. # Construct string representation of array def _toString(data, dimension): s = '' if dimension == 0: s = s + `data` elif dimension == 1: for e in data: s = s + `e` + ' ' else: separator = (dimension-1)*'\n' for e in data: s = s + _toString(e,dimension-1) + separator return string.strip(s) # Find the shape of an array and check for consistency def _shape(data): if type(data) == type([]): if data and type(data[0]) == type([]): shapes = map(lambda x:_shape(x),data) for i in range(1,len(shapes)): if shapes[i] != shapes[0]: raise ArrayError, 'Inconsistent shapes' shape = [len(data)] shape = shape + shapes[0] return shape else: return [len(data)] else: return [] # Construct a one-dimensional list of all array elements def __ravel(data): if type(data) == type([]): if type(data[0]) == type([]): return reduce(lambda a,b: a+b, map(lambda x: __ravel(x), data)) else: return data else: return [data] def _ravel(array): return Array(__ravel(array._data), reduce(lambda a,b: a*b, array._shape, 1)) # Reshape an array def _reshape(array, shape): array = ravel(array) if len(shape._data) == 0: return take(array,0) else: array = array._data shape = shape._data n = reduce(lambda a,b: a*b, shape) if n > len(array): nr = (n+len(array)-1)/len(array) array = (nr*array)[:n] elif n < len(array): array = array[:n] for i in range(len(shape)-1, 0, -1): d = shape[i] n = n/d for j in range(n): array[j:j+d] = [array[j:j+d]] return Array(array,shape) # Map a function on the first dimensions of an array def _extract(a, index, dimension): if len(a[1]) < dimension: return a else: return (a[0][index], a[1][1:], a[2]) def _map(function, arglist, max_frame, scalar_flag): result = [] if len(max_frame) == 0: if scalar_flag: result = apply(function,tuple(map(lambda a: a[0], arglist))) else: result = apply(function,tuple(map(lambda a: Array(a[0],a[2]), arglist)))._data else: for index in range(max_frame[0]): result.append(_map(function, map(lambda a,i=index,d=len(max_frame): _extract(a,i,d), arglist), max_frame[1:], scalar_flag)) return result # Reduce an array with a given binary function def _reduce(function, array): function = function[0] array = array[0] if len(array._shape) == 0: return array elif array._shape[0] == 0: return reshape(function._neutral, array._shape[1:]) else: result = Array(array._data[0], array._shape[1:]) for i in range(1,array._shape[0]): result = function(result, Array(array._data[i], array._shape[1:])) return result # Find the higher of two ranks def _maxrank(a,b): if a == None or b == None: return None else: return max(a,b) ###################################################################### # Array class definition class Array: def __init__(self, scalar_or_list, shape = None): self._data = scalar_or_list if shape == None: self._shape = _shape(self._data) else: self._shape = shape def __str__(self): return _toString(self._data,len(self._shape)) __repr__ = __str__ def __len__(self): if type(self._data) == type([]): return len(self._data) else: return 1 def __getitem__(self, index): return take(self, index) def __getslice__(self, i, j): return take(self, range(i,j)) def __add__(self, other): return sum(self, other) __radd__ = __add__ def __sub__(self, other): return difference(self, other) def __rsub__(self, other): return difference(other, self) def __mul__(self, other): return product(self, other) __rmul__ = __mul__ def __div__(self, other): return quotient(self, other) def __rdiv__(self, other): return quotient(other, self) def __pow__(self,other): return power(self, other) def __rpow__(self,other): return power(other, self) def __neg__(self): return 0-self # Check for arrayness def isArray(x): return hasattr(x,'_shape') ###################################################################### # Array function class class ArrayFunction: def __init__(self, function, ranks, intrinsic_ranks=None): self._function = function if isArray(ranks): self._ranks = ranks._data elif type(ranks) == type([]): self._ranks = ranks else: self._ranks = [ranks] if intrinsic_ranks == None: self._intrinsic_ranks = self._ranks else: self._intrinsic_ranks = intrinsic_ranks if len(self._ranks) == 1: self._ranks = len(self._intrinsic_ranks)*self._ranks def __call__(self, *args): if len(self._ranks) != len(args): raise ArrayError, 'Wrong number of arguments for an array function' arglist = [] framelist = [] shapelist = [] for i in range(len(args)): if isArray(args[i]): arglist.append(args[i]) else: arglist.append(Array(args[i])) shape = arglist[i]._shape rank = self._ranks[i] intrinsic_rank = self._intrinsic_ranks[i] if rank == None: cell = 0 elif rank < 0: cell = min(-rank,len(shape)) else: cell = max(0,len(shape)-rank) if intrinsic_rank != None: cell = max(cell,len(shape)-intrinsic_rank) framelist.append(shape[:cell]) shapelist.append(shape[cell:]) max_frame = [] for frame in framelist: if len(frame) > len(max_frame): max_frame = frame for i in range(len(framelist)): if framelist[i] != max_frame[len(max_frame)-len(framelist[i]):]: raise ArrayError, 'Incompatible arguments' scalar_function = reduce(lambda a,b:_maxrank(a,b), self._intrinsic_ranks) == 0 return Array(_map(self._function, map(lambda a,b,c: (a._data,b,c), arglist, framelist, shapelist), max_frame, scalar_function)) def __getitem__(self, ranks): return ArrayFunction(self._function,ranks,self._intrinsic_ranks) class BinaryArrayFunction(ArrayFunction): def __init__(self, function, neutral_element, ranks, intrinsic_ranks=None): ArrayFunction.__init__(self, function, ranks, intrinsic_ranks) self._neutral = neutral_element self.over = ArrayFunction(ArrayOperator(_reduce, [self]), [None]) def __getitem__(self, ranks): return BinaryArrayFunction(self._function, self._neutral, ranks, self._intrinsic_ranks) class ArrayOperator: def __init__(self, operator, function_list): self._operator = operator self._functions = function_list def __call__(self, *args): return apply(self._operator, (self._functions, args)) ###################################################################### # Array functions # Structural functions shape = ArrayFunction(lambda a: Array(a._shape), [None]) reshape = ArrayFunction(_reshape, [None, 1]) ravel = ArrayFunction(_ravel, [None]) take = ArrayFunction(lambda a,i: Array(a._data[i._data], a._shape[1:]), [None, 0]) # Elementwise binary functions _sum = ArrayFunction(lambda a,b: a+b, [0, 0]) _difference = ArrayFunction(lambda a,b: a-b, [0, 0]) _product = ArrayFunction(lambda a,b: a*b, [0, 0]) _quotient = ArrayFunction(lambda a,b: a/b, [0, 0]) _power = ArrayFunction(pow, [0, 0]) sum = BinaryArrayFunction(_sum, 0, [None, None]) difference = BinaryArrayFunction(_difference, 0, [None, None]) product = BinaryArrayFunction(_product, 1, [None, None]) quotient = BinaryArrayFunction(_quotient, 1, [None, None]) power = BinaryArrayFunction(_power, 1, [None, None]) # Scalar functions of one variable sqrt = ArrayFunction(math.sqrt, [0]) exp = ArrayFunction(math.exp, [0]) log = ArrayFunction(math.log, [0]) log10 = ArrayFunction(math.log10, [0]) sin = ArrayFunction(math.sin, [0]) cos = ArrayFunction(math.cos, [0]) tan = ArrayFunction(math.tan, [0]) asin = ArrayFunction(math.asin, [0]) acos = ArrayFunction(math.acos, [0]) atan = ArrayFunction(math.atan, [0]) sinh = ArrayFunction(math.sinh, [0]) cosh = ArrayFunction(math.cosh, [0]) tanh = ArrayFunction(math.tanh, [0]) floor = ArrayFunction(math.floor, [0]) ceil = ArrayFunction(math.ceil, [0]) ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Sep 27 02:39:35 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Tue, 26 Sep 1995 21:39:35 -0400 Subject: [PYTHON MATRIX-SIG] ArrayTutorial.py Message-ID: <199509270139.VAA15853@cyclone.ERE.UMontreal.CA> # This file illustrates the use of the the Array class. # # Send comments to Konrad Hinsen # from Array import * ###################################################################### # Arrays are constructed from (nested) lists: a = Array(range(10)) b = Array([ [2,3,7], [9,8,2] ]) c = Array([ [ [4,5,6], [0,4,5] ], [ [1,6,5], [8,5,2] ] ]) # Scalars make arrays of rank 0: s = Array(42) # Array elements can be anything: text_array = Array(['Hello', 'world']) # Arrays can be printed: print 'a:\n', a print 'b:\n', b print 'c:\n', c print 's:\n', s # shape() returns an array containing the dimensions of another array: print 'shape(a):\n', shape(a) print 'shape(b):\n', shape(b) print 'shape(c):\n', shape(c) print 'shape(s):\n', shape(s) # Scalar functions act on each element of an array: print 'sqrt(b):\n',sqrt(b) # Binary operators likewise work elementwise: print 'c+c:\n',c+c # But you can also add a scalar: print 'c+s:\n',c+s # To understand the general rule for combining arrays of different # shapes in a function, we need some more technical terms: # The length of the shape vector of an array is called its rank. # The elements of an array along the first axis are called items. # The items of an array of rank N have rank N-1. More generally, # the shape vector is divided into frames and cells. Frames and # cells are not properties of an array, but describe ways of looking # at an array. For example, a rank-3 array can be regarded as # as just that - a single rank-3 array. It can also be regarded # as a rank-1 frame of rank-2 cells, or as a rank-2 frame of # rank-1 cells. Or even as a rank-3 array of rank-0 cells, i.e. # scalar cells. # # When two arrays are to be added (or multiplied, or...), their # shapes need not equal, but the lower-rank array must match # an equal-rank cell of the higher-rank array. The lower-rank # array will then be combined with each corresponding cell, and # the result will have the shape of the higher-rank array. print 'b+c:\n',b+c # The addition of a scalar is just a special case: it has rank 0 # and therefore matches the rank-0 cells (scalar elements) of any array! print 'b+s:\n',b+s # All operators are also available as normal binary function, # e.g. addition can be written as print 'sum(a,s):\n',sum(a,s) # You'll need this form to perform reductions, e.g. a sum # of all items of an array: print 'sum.over(a):\n',sum.over(a) # Let's do it with a higher-rank array: print 'product.over(b):\n',product.over(b) # But how do you get the product along the second axis # of b? Easy: print 'product.over[1](b):\n',product.over[1](b) # The [1] after the function name product.over modifies # the functions rank. Function ranks are related to array # ranks, in that a function of rank N operates on the # N-cells of its argument. If the argument has a higher # rank, the function is applied to each N-cell and the # result is combined with the frame of the argument. # In the last example, product.over will be # called for each of the 1-cells of b, returning a # rank-0 result for each, and the results will be # collected in the 1-frame of b, producing as a net # result an array of rank 1. # # All functions have ranks; if no rank is explicitly # given, the default rank will be used. The default # rank of all reductions is 'unbounded', which means # that the function will operate on the highest-level # cells possible. Many functions have unbounded rank, # for example shape(): print 'shape(c):\n',shape(c) # But of course you can modify the rank of shape(): print 'shape[1](c):\n',shape[1](c) print 'shape[2](c):\n',shape[2](c) # Functions with more than one argument can have a different # rank for each. The rank is applied to each argument, # defining its frames and cells for this purpose. The frames # must match in the same way as indicated above for # addition of two arrays. The function is then applied # to the cells, and if appropriate, the same matching # procedure is applied once again. This may seem confusing # at first, but it is really just the application of a # single principle everywhere! # # For example, let's take a (our rank-1 array) and add # b (our rank-2 array) to each of a's 0-cells: print 'sum[[0,None]](a,b):\n',sum[[0,None]](a,b) # 'None' stands for 'unbounded'. Since the rank of sum is # 0 for its first argument, a is divided into 1-frames # and 0-cells. For b the rank is unbounded, so it is # treated as a 0-frame with 2-cells. b's 0-frame matches # a's 1-frame (a 0-frame matches everything!), and # the result gets the 1-frame. The cells of the result # are sums of a rank-0 array (element of a) and a rank-2 # array (b), i.e. rank-2 arrays by the matching rules # given above. So the net total is a rank-3 array, # as you have seen. # From now on we will specify the default rank of each function. # It should be noted that specifying a higher rank than the # default rank has no effect on the function's behaviour. Only # lower ranks make a difference. # # All the scalar mathematical functions (sqrt, sin, ...) have # rank 0. The binary arithmetic functions (sum, product, ...) # have unbounded rank for both argument. For structural functions # (i.e. those that modify an array's shape rather than its # elements), the rank varies. As we have seen, shape() is # unbounded. The other structural functions are yet to be # introduced: # ravel() produces a rank-1 array containing all elements # of its argument. It has unbounded rank: print 'ravel(c):\n',ravel(c) # reshape() allows you to change the shape of an array. # It first obtains a rank-1 list of elements of its first # argument (like ravel()) and then reassembles the # elements into an array with the shape given by the # second argument: print 'reshape(a,[2,2]):\n',reshape(a,[2,2]) print 'reshape(b,[10]):\n',reshape(b,[10]) # As you have seen in the second case, reshape() reuses # the elements of its arguments all over if they get # exhausted. # You may have noticed that in some examples, a nested list # has been used instead of an array as a function argument. # In general, all array functions will convert its arguments # to arrays if necessary. # Now we need a way to select parts of an array. You can # use standard indexing and slicing to obtain items # (i.e. N-1 cells for a rank-N array): print 'c[1]:\n',c[1] print 'a[3:7]:\n',a[3:7] # You can also specify an array as the index and obtain an # array of the corresponding items: print 'a[[2,6,1]]:\n',a[[2,6,1]] # The function take() does exactly the same: print 'take(c,1):\n',take(c,1) # You will have to use take() if you want to modify its # ranks, which are [None,0] by default. There is little # point in changing the second rank (it wouldn't make # any difference), but changing the first rank lets you # select along other dimensions than the first: print 'take[1](c,0):\n',take[1](c,0) print 'take[2](c,1):\n',take[2](c,1) # Isn't there something wrong here? take() takes # two arguments and therefore needs two ranks. But for # convenience, only one rank must be given if all ranks # are to be the same. # And that's the end of this introduction. Stay tuned for # updates of the array package that will provide some # important still missing in this version. In the meantime, # play round with what there is to get a feeling for # how things work. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Wed Sep 27 10:37:12 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Wed, 27 Sep 1995 05:37:12 -0400 Subject: [PYTHON MATRIX-SIG] J-style arrays in Python In-Reply-To: <199509270138.VAA15833@cyclone.ERE.UMontreal.CA> Message-ID: <199509270931.JAA29752@qvarsx.er.usgs.GOV> On Tue, 26 Sep 1995 21:38:49 -0400 Hinsen Konrad said: > To bring back some life into this discussion group, I'll distribute a > Python implementation of J-like arrays, to give those unfamiliar with > J a chance to become familiar with its array and rank system. Cool. This will help me become more familiar with some of the generic processing operators that have been discussed. I for one have not had much to say lately because I'm waiting for a chance to study some of the suggestions for map-ish operators. > And of > course it is also usable, as long as the arrays don't become too big. This is perfectly appropriate for a prototype. A real implementation will need to handle large arrays well. > This is a first test version; many important functions are still > missing, others (like output formatting) need substantial improvement, > and many could probably be made faster with some thought. > > To make (my) life simpler, I tried to stick to J's conventions as much > as possible (but hopefully without violating Python traditions). I am > not claiming that this is semantically the best way to implement arrays, > but it is a start. Please keep in mind that: - If there is a standard matrix class, it will be implemented in C, for performance reasons, - There is already a base implementation, currently being worked on by James Hugunin. - Much of the base implementation was dictated by the *stated* goal that the implementation should hold the data in an homogenous and contingous block of data suitable for passing directly to existing Fortran and C libraries. > Some general remarks: > > - Since my implementation uses nested lists to represent arrays, > the elements can be arbitrary objects. Which violates one of the basic goals of this effort. I realize that you may not agree with the goal, but this was clearly stated in the announcement for *this* SIG. > - Like arrays in J, my arrays are immutable, i.e. there is no > provision for changing individual elements. The reason for > making arrays immutable in J was that J is half-way to being > a functional language (it is not a pure functional language, > but many substantial problems can easily be solved in a > functional way). I have never missed element assignment, but > probably there are some good applications... Gee. I would miss element assignment. > - All functions are implemented as external functions, not as > methods. The main reason is that at first I could not think of > a way to implement methods with variable rank, although later > I figured out how to do this (in the same way as I implemented > reduction). This brings up a good point. I think that whatever we come up with should adhere to the KISS (Keep It Simple Stupid) rule as much as possible. I'm in favor of a fairly lean matrix module with auxilary modules to provide support for specific application areas. > I'll send two Python files; the first is the Array module itself, > and the second a kind of simple tutorial. Let me know if > anything is unclear. And let me know what you think of the > whole implementation! I look forward to studying what you sent. (When I have a chance. :) Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Sep 27 14:35:32 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Wed, 27 Sep 1995 09:35:32 -0400 Subject: [PYTHON MATRIX-SIG] J-style arrays in Python In-Reply-To: <199509270931.JAA29752@qvarsx.er.usgs.GOV> (jfulton@usgs.gov) Message-ID: <199509271335.JAA06038@cyclone.ERE.UMontreal.CA> Please keep in mind that: - If there is a standard matrix class, it will be implemented in C, for performance reasons, I hope so! - Much of the base implementation was dictated by the *stated* goal that the implementation should hold the data in an homogenous and contingous block of data suitable for passing directly to existing Fortran and C libraries. Actually I can't think of any reason not to keep the data in a continous block... > Some general remarks: > > - Since my implementation uses nested lists to represent arrays, > the elements can be arbitrary objects. Which violates one of the basic goals of this effort. I realize that you may not agree with the goal, but this was clearly stated in the announcement for *this* SIG. I do not disagree with that goal at all; in fact I seriously considered adding type checking (or rather consistency checking) to my Python implementation. But it would have slowed down everything without producing much of an advantage (I assume no one will produce mixed arrays by accident), so I left it out. Again, I do not claim in the least that any future array module should resemble my implementation in any respect. On the contrary, I expect that both could be used in parallel for different applications. I started writing this because I had a need for flexible (but small) arrays, and then polished it up a bit to make it usable as a demonstration for people in this SIG. Gee. I would miss element assignment. Really? I realize that element assignment is necessary to implement many of the standard linear algebra algorithms, but these would be implemented in C/Fortran/whatever anyway. I have never missed element assignment in J; in fact, I only noticed its absence while working on my Python implementation! This brings up a good point. I think that whatever we come up with should adhere to the KISS (Keep It Simple Stupid) rule as much as possible. I'm in favor of a fairly lean matrix module with auxilary modules to provide support for specific application areas. So am I. But we must make sure that the auxiliary modules can be used together conveniently. For example, the function names should be distinct, to make it possible to import them all into a single namespace (important for calculator-style use). ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jfulton@usgs.gov Wed Sep 27 15:13:50 1995 From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey) Date: Wed, 27 Sep 1995 10:13:50 -0400 Subject: [PYTHON MATRIX-SIG] J-style arrays in Python In-Reply-To: <199509271335.JAA06038@cyclone.ERE.UMontreal.CA> Message-ID: <199509271408.OAA04669@qvarsx.er.usgs.GOV> On Wed, 27 Sep 1995 09:35:32 -0400 Hinsen Konrad said: > > Please keep in mind that: > > - If there is a standard matrix class, it will be implemented in C, > for performance reasons, > > I hope so! > > - Much of the base implementation was dictated by the *stated* goal > that the implementation should hold the data in an homogenous and > contingous block of data suitable for passing directly to existing > Fortran and C libraries. > > Actually I can't think of any reason not to keep the data in a > continous block... > > > Some general remarks: > > > > - Since my implementation uses nested lists to represent arrays, > > the elements can be arbitrary objects. > > Which violates one of the basic goals of this effort. I realize that > you may not agree with the goal, but this was clearly stated in the > announcement for *this* SIG. > > I do not disagree with that goal at all; in fact I seriously > considered adding type checking (or rather consistency checking) to my > Python implementation. But it would have slowed down everything > without producing much of an advantage (I assume no one will produce > mixed arrays by accident), so I left it out. > > Again, I do not claim in the least that any future array module > should resemble my implementation in any respect. On the contrary, > I expect that both could be used in parallel for different > applications. I started writing this because I had a need for > flexible (but small) arrays, and then polished it up a bit to > make it usable as a demonstration for people in this SIG. Sounds like we're on the same wave-length then. > Gee. I would miss element assignment. > > Really? I realize that element assignment is necessary to implement > many of the standard linear algebra algorithms, but these would be > implemented in C/Fortran/whatever anyway. I have never missed element > assignment in J; in fact, I only noticed its absence while working on > my Python implementation! > > This brings up a good point. I think that whatever we come up with > should adhere to the KISS (Keep It Simple Stupid) rule as much as > possible. I'm in favor of a fairly lean matrix module with auxilary > modules to provide support for specific application areas. > > So am I. But we must make sure that the auxiliary modules can > be used together conveniently. For example, the function names > should be distinct, to make it possible to import them all into > a single namespace I don't agree with this. In general, I'm not a fan of "from spam import *". > (important for calculator-style use). Why? Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Wed Sep 27 16:01:45 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Wed, 27 Sep 1995 11:01:45 -0400 Subject: [PYTHON MATRIX-SIG] J-style arrays in Python In-Reply-To: <199509271408.OAA04669@qvarsx.er.usgs.GOV> (jfulton@usgs.gov) Message-ID: <199509271501.LAA09849@cyclone.ERE.UMontreal.CA> I don't agree with this. In general, I'm not a fan of "from spam import *". Neither am I, in general. But... > (important for calculator-style use). Why? To avoid having to type the module name all over again, of course. I am using my array module mostly to work (interactively) on simulation results read in from files (I didn't include the I/O in the version I sent around, because it still works only for a few special cases). So I am typing things like sqrt(Array("file1"))*Array("file2) which I prefer a lot to Array.sqrt(Array.Array("file1"))*Array.Array("file2) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From guido@CNRI.Reston.VA.US Fri Sep 29 13:21:01 1995 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Fri, 29 Sep 1995 08:21:01 -0400 Subject: [PYTHON MATRIX-SIG] Time for recap? Message-ID: <199509291221.IAA19851@monty> I'll try to be short. I like the general idea of going with APL/J style multidimensional objects. I liked Konrad's sample implementation even though I agree an actual implementation would have to have a contiguous representation. I'm not sure about Konrad's particular notation for changing the rank of an operator (though it is very cool that that is possible in Python!). More thought needs to go in details like this. I think the best way to proceed is to use the built-in operators for APL/J style elementwise operations and to have the specific 2D linear algebra operations as methods. Likewise, instead of my earlier wild ideas about multidimensional slices, I propose to use a slice() method that can produce arbitrary slices; e.g. a[k][i:j] would be equivalent to a.slice(k, (i,j)). I'm not against dropping the parentheses in a[(i,j)] but I'm not sure if I can give it the proper thought and testing before October 13, the scheduled release of Python 1.3 (at 13:13:13 hours :-). We can work on a standard patch shortly after that date though. - - - I have one idea I would like to float by this group. How about separating out the representation and the structure? I believe I've seen C/Fortran matrix packages that made quite good use of this. The representation would be a simple 1-dimensional sequence. You'd normally not see or use this, but it would be there if you needed access to it (e.g. for passing to C/Fortran code). There's a simple way to map an index in an N-dim array into an index in the 1-dim representation array (Fortran compilers use it all the time :-). To make efficient use of this, I propose that slicing and indexing, as long as they return an object of rank >= 1, return an object that points into the same representation sequence. I propose one additional feature (which I believe I saw in Algol-68; some matrix packages may also use it): add a "stride" to each dimension (including the last). This makes it possible to have slices reference discontiguous parts of the underlying representation, and even to represent a transposed version! If we do things just right, it may be possible to pass in the sequence to be used as the representation -- it could be a Python list, tuple or array (from the array module). Again, all this can be prototyped in Python! I would give an example, but I have to run. --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From hinsenk@ere.umontreal.ca Fri Sep 29 14:45:05 1995 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Fri, 29 Sep 1995 09:45:05 -0400 Subject: [PYTHON MATRIX-SIG] Time for recap? In-Reply-To: <199509291221.IAA19851@monty> (message from Guido van Rossum on Fri, 29 Sep 1995 08:21:01 -0400) Message-ID: <199509291345.JAA27953@cyclone.ERE.UMontreal.CA> I liked Konrad's sample implementation even though I agree an actual implementation would have to have a contiguous representation. The two main reasons for choosing the nested list implementation (remember, that was before I even knew about the Matrix SIG) were: 1) the possibility of using nested lists as a notation for array constants; this means I rarely have to type Array(). 2) many of the array operations can be written concisely as recursive functions. The second reason is less important for our purposes, but we do have to think about a convenient notation for array constants. We might actually use nested lists and provide a coercion function that transforms them into a contiguous representation. I'm not sure about Konrad's particular notation for changing the rank of an operator (though it is very cool that that is possible in Python!). More thought needs to go in details like this. Definitely. I am not too pleased with it myself, mostly because the specification of ranks for functions with more than one argument is cumbersome, requiring two sets of square brackets. But I couldn't find another way to specify rank in Python! I think the best way to proceed is to use the built-in operators for APL/J style elementwise operations and to have the specific 2D linear algebra operations as methods. Why such a division? There is no clear borderline between "elementwise" and "linear algebra". If we should distinguish at all between methods and functions, I'd propose to use methods for procedures that change an array in place and functions for those that return a new array. I have one idea I would like to float by this group. How about separating out the representation and the structure? I believe I've seen C/Fortran matrix packages that made quite good use of this. The representation would be a simple 1-dimensional sequence. You'd normally not see or use this, but it would be there if you needed access to it (e.g. for passing to C/Fortran code). I have also seen C++ packages that make the representation available in this way, and it makes sense. To make efficient use of this, I propose that slicing and indexing, as long as they return an object of rank >= 1, return an object that points into the same representation sequence. Again, there are C++ classes that do this, and of course this makes slicing very efficient. On the other hand, it makes element assignment more complicated, since the array may have to be copied first. I propose one additional feature (which I believe I saw in Algol-68; some matrix packages may also use it): add a "stride" to each dimension (including the last). This makes it possible to have slices reference discontiguous parts of the underlying representation, and even to represent a transposed version! Again, this is used in C++ classes... There's a book that I have recommended before (Scientific end Engineering C++, by John J. Barton and Lee R. Nackman) that describes a matrix class with all the implementation features you just proposed. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@mama-bear.lcs.mit.edu Fri Sep 29 20:18:02 1995 From: jjh@mama-bear.lcs.mit.edu (James Hugunin) Date: Fri, 29 Sep 95 15:18:02 -0400 Subject: [PYTHON MATRIX-SIG] Time for recap? References: <199509291345.JAA27953@cyclone.ERE.UMontreal.CA> Message-ID: <9509291918.AA07305@nineveh.LCS.MIT.EDU> > I like the general idea of going with APL/J style multidimensional > objects. > > I liked Konrad's sample implementation even though I agree an actual > implementation would have to have a contiguous representation. > > I'm not sure about Konrad's particular notation for changing the rank > of an operator (though it is very cool that that is possible in > Python!). More thought needs to go in details like this. I found Konrad's implementation enlightening regarding the ways of APL. I have a different notation for many of the things he does. Some of this notation should be discussed ASAP, but I'll save that for a later message. > I think the best way to proceed is to use the built-in operators for > APL/J style elementwise operations and to have the specific 2D linear > algebra operations as methods. I agree completely! > Likewise, instead of my earlier wild ideas about multidimensional > slices, I propose to use a slice() method that can produce arbitrary > slices; e.g. a[k][i:j] would be equivalent to a.slice(k, (i,j)). I disagree slightly here. I still prefer a[(k, range(i,j))] for Guido's example. This makes a[(k, [x,y,z])] possible. > I have one idea I would like to float by this group. How about > separating out the representation and the structure? > > I believe I've seen C/Fortran matrix packages that made quite good use > of this. The representation would be a simple 1-dimensional sequence. > You'd normally not see or use this, but it would be there if you > needed access to it (e.g. for passing to C/Fortran code). > > There's a simple way to map an index in an N-dim array into an index > in the 1-dim representation array (Fortran compilers use it all the > time :-). > > To make efficient use of this, I propose that slicing and indexing, as > long as they return an object of rank >= 1, return an object that > points into the same representation sequence. > > I propose one additional feature (which I believe I saw in Algol-68; > some matrix packages may also use it): add a "stride" to each > dimension (including the last). This makes it possible to have slices > reference discontiguous parts of the underlying representation, and > even to represent a transposed version! I agree almost completely with this specification. In fact, in many ways this is exactly what I have been implementing on top of Jim Fulton's original matrix object. (btw - the real or the imaginary part of a complex matrix are also particularly easy to represent using this style). The one thing that I've been doing differently is that slices (ie. a[1:5]) are returned by value rather than by reference. This was Jim Fulton's original implementation and I kept it because it was similar to the notion of slicing a list. Conceptually, I have no problems with treating slices the same as any other discontinuous index (ie. a[(range(1,5), range(4,6))]) and returning them by reference. Actually, I like the simplicity of being able to think of every index into a matrix returning by reference. I would be interested in hearing other opinions on this issue. > If we do things just right, it may be possible to pass in the sequence > to be used as the representation -- it could be a Python list, tuple > or array (from the array module). This is an interesting notion, but I can't see what it would gain over using a 1d C-array of basic types as the representation. It wouldn't be too hard to expand the definition of basic type to include (PyObject *)'s if you'd like to have the possibility of a matrix of "real" python objects. I should have time to finalize some of these things this weekend so that I can make an alpha version of a matrix object written in C available to interested members of this group on Monday. It is surprisingly similar to Konrad's sample implementation in python (though a lot bigger, uglier and yet faster owing to it's C-implementation). -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From guido@CNRI.Reston.VA.US Sat Sep 30 19:07:39 1995 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Sat, 30 Sep 1995 14:07:39 -0400 Subject: [PYTHON MATRIX-SIG] Time for recap? In-Reply-To: Your message of "Fri, 29 Sep 1995 15:18:02 EDT." <9509291918.AA07305@nineveh.LCS.MIT.EDU> References: <199509291345.JAA27953@cyclone.ERE.UMontreal.CA> <9509291918.AA07305@nineveh.LCS.MIT.EDU> Message-ID: <199509301807.OAA22282@monty> > > Likewise, instead of my earlier wild ideas about multidimensional > > slices, I propose to use a slice() method that can produce arbitrary > > slices; e.g. a[k][i:j] would be equivalent to a.slice(k, (i,j)). > > I disagree slightly here. I still prefer a[(k, range(i,j))] for Guido's > example. This makes a[(k, [x,y,z])] possible. Ah. I was writing from memory, and forgot this feature. I don't like it. How many times do you have to select a more or less arbitrary group of rows/columns from a matrix? It makes the slice representation bulkier -- contiguous slices can be stored (in C) as 4 ints per dimension, while random selections will require a variable-length list of indices per dimension. (It is also wasteful to have to generate the full range of numbers when what you mean is a contiguous slice.) Perhaps a more important reason why I don't like this is that I see problems with various notations. In the current version of the language, you can't write a[2,3] -- you have to write a[(2,3)] and it will be used as a mapping index (not as a sequence index!). The proposal is to make this mean a[2][3]. That's fine with me. There are certain ambiguities with the use of parentheses in Python that I can't completely get rid of without breaking the type structure: (1) means the same as 1 -- a scalar, while (1,2) is a tuple of length 2. To write a tuple of length one, you can't write (1) -- you have to write (1,). (A tuple of length 0 is () -- easy.) For consistency, I think I'll have to make a[(1,)] mean the same as a[1], if a[(1,2)] is the same as a[1,2]. Finally, notice that to the object being indexed, there is no way to tell a[(1,2)] from x=(1,2); a[x]. A tuple is just one example of a sequence in Python. Lists are another example. In many situations, any sequence is acceptable and the results are the same (e.g. for loops). (And in situations where only lists or only tuples are accepted by the current version of the language, Steve Majewski has often made the point that there is no inherent reason why only one type should be accepted and that this should be fixed. I agree in most cases.) If we extend this rule to N-dimensional indexing, a[sequence], for some sequence of integers whose elements are i, j, k, ..., should be equivalent to a[i][j][k]..., and we can't make a[(1,2,3)] mean a[1][2][3] while at the same time interpreting a[[1,2,3]] as a[1:4]. (Sooner or later, you'll be passing a vector to the index. Then the question will arise, should this have the same meaning as a tuple or as a list. It's better if they all three mean the same.) Instead of supporting a[[2,3,5]] to select elements 2, 3 and 5 from a, I would propose to use filter() or a multi-dimensional extension thereof if you want to access selected subarrays. Or perhaps just a method a.select() where each argument is either an index (meaning a reduction of dimensionality in this dimension by picking just the sub-array with this index) or a sequence of indices (meaning selecting the set of sub-arrays with the indices in the sequence). How about it? Is this acceptable? > The one thing that I've been doing differently is that slices (ie. a[1:5]) > are returned by value rather than by reference. This was Jim Fulton's > original implementation and I kept it because it was similar to the notion > of slicing a list. Conceptually, I have no problems with treating slices > the same as any other discontinuous index (ie. a[(range(1,5), range(4,6))]) > and returning them by reference. Actually, I like the simplicity of being > able to think of every index into a matrix returning by reference. It's somewhat odd that slices are returned by reference (since they return a new value for lists), but not against the rules or silent assumptions of the language, and I think it is necessary to make the most of the flexibility that you get by using the notion of separating the indexing and the representation object. > I would be interested in hearing other opinions on this issue. Me too. > > If we do things just right, it may be possible to pass in the sequence > > to be used as the representation -- it could be a Python list, tuple > > or array (from the array module). > > This is an interesting notion, but I can't see what it would gain > over using a 1d C-array of basic types as the representation. I like this idea because the list or array containing the representation may already be available in memory -- so why copy it? Also by using an immutable underlying sequence (e.g. a tuple) it is easy to create immutable N-dimensional arrays without the need for a read-only flag. Finally it makes it possible to use a representation where the actual values are stored in disk and only fetched into memory when needed, using a cache -- this way you can implement your own virtual memory system, persistent matrices, etc. There could still be a "default" underlying representation that is highly optimized and that the indexing object knows about, for speedier access. > It wouldn't be too hard to expand the definition of basic type to > include (PyObject *)'s if you'd like to have the possibility of a > matrix of "real" python objects. Yes, the latter is definitely something that should be possible even if my idea doesn't find the acceptance I hope it will get. > I should have time to finalize some of these things this weekend so that I > can make an alpha version of a matrix object written in C available to > interested members of this group on Monday. It is surprisingly similar to > Konrad's sample implementation in python (though a lot bigger, uglier and > yet faster owing to it's C-implementation). Cool! --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From guido@CNRI.Reston.VA.US Sat Sep 30 19:30:24 1995 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Sat, 30 Sep 1995 14:30:24 -0400 Subject: [PYTHON MATRIX-SIG] Time for recap? In-Reply-To: Your message of "Fri, 29 Sep 1995 09:45:05 EDT." <199509291345.JAA27953@cyclone.ERE.UMontreal.CA> References: <199509291345.JAA27953@cyclone.ERE.UMontreal.CA> Message-ID: <199509301830.OAA22332@monty> (Me:) > I think the best way to proceed is to use the built-in operators for > APL/J style elementwise operations and to have the specific 2D linear > algebra operations as methods. (Konrad:) > Why such a division? There is no clear borderline between "elementwise" > and "linear algebra". If we should distinguish at all between methods > and functions, I'd propose to use methods for procedures that change > an array in place and functions for those that return a new array. I just meant to end the debate about whether a*b should mean matrix multiplication in the LA sense or elementwise multiplication like APL/J. This only an issue for * and /. For / most people agree that "matrix division" is numerically ill-defined and you should have specified the particular decomposition you need instead. This leaves *, and, in spite of the elegance of linear algebra matrix multiplication, I vote for the consistency of elementwise multiplication. This means that we need to define a method that implements LA matrix multiply, e.g. a.mul(b). Functions are out of the question (since they would have to live in the namespace reserved for built-in functions.) (Me:) > To make efficient use of this, I propose that slicing and indexing, as > long as they return an object of rank >= 1, return an object that > points into the same representation sequence. (Konrad:) > Again, there are C++ classes that do this, and of course this makes > slicing very efficient. On the other hand, it makes element assignment > more complicated, since the array may have to be copied first. I don't see much of a problem with that. Functions/methods that take an array and return a like-shaped array should always copy their argument before modifying it. Methods that are supposed to modify an array in place should not also return a reference to the array. Functions/methods that wish to modify the array only as a means of calculating some property of the array should come in to flavors: a "low-level" version whose name ends in "_inplace", which works on the array in place, and a "high-level" version which copies the array and then invokes the low-level version. The user can then decide to use the low-level version if performance so dictates, in which case the consequences of course must be considerd first. (I'd hate to see optional "in-place" flag arguments -- in general, I don't like to see flag arguments where the calls will always contain a constant argument value.) --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From jjh@mama-bear.lcs.mit.edu Sat Sep 30 19:53:50 1995 From: jjh@mama-bear.lcs.mit.edu (James Hugunin) Date: Sat, 30 Sep 95 14:53:50 -0400 Subject: [PYTHON MATRIX-SIG] Time for recap? References: <199509291345.JAA27953@cyclone.ERE.UMontreal.CA> <9509291918.AA07305@nineveh.LCS.MIT.EDU> <199509301807.OAA22282@monty> Message-ID: <9509301853.AA00377@nineveh.LCS.MIT.EDU> > > > Likewise, instead of my earlier wild ideas about multidimensional > > > slices, I propose to use a slice() method that can produce arbitrary > > > slices; e.g. a[k][i:j] would be equivalent to a.slice(k, (i,j)). > > > > I disagree slightly here. I still prefer a[(k, range(i,j))] for Guido's > > example. This makes a[(k, [x,y,z])] possible. > > Ah. I was writing from memory, and forgot this feature. I don't like > it. How many times do you have to select a more or less arbitrary > group of rows/columns from a matrix? It makes the slice representation > bulkier -- contiguous slices can be stored (in C) as 4 ints per > dimension, while random selections will require a variable-length > list of indices per dimension. (It is also wasteful to have to > generate the full range of numbers when what you mean is a contiguous > slice.) As far as implementation efficiency is concerned, I have implemented things so that each dimension requires 2 ints and an int pointer. If the dimension corresponds to a quasi-contiguous slice then the pointer is set to NULL, and the ints give the number of indices and a stride. If the dimension corresponds to a random slice, then the pointer points to an array of indices. So, very little efficiency is lost for quasi-contiguous slices, and yet it is still possible to represent random slices. My current favorite use for arbitrary indexing is for image zooming. ie: img = [1,2,3,4] a = img[((1,1,2,2,2,3,3,3,4,4),)] a = [1,1,2,2,2,3,3,3,4,4] This is a reasonably efficient approach for coarse-zooming in to even large 2d images, and it will work for on any arbitrary matrix (unlike C-code which I must write a special case for each matrix). My other reason for really liking this feature is that matlab supports it, and I would really like to see this object support as large a subset of matlab code as possible. > To write a tuple of length one, you can't write (1) -- > you have to write (1,). (A tuple of length 0 is () -- easy.) For > consistency, I think I'll have to make a[(1,)] mean the same as a[1], > if a[(1,2)] is the same as a[1,2]. Finally, notice that to the object > being indexed, there is no way to tell a[(1,2)] from x=(1,2); a[x]. I disagree completely with this. The proposal to allow tuplizing inside of []'s I feel should be completely compatible with "tupleizing" outside of brackets. Just as you can right in python "a = 1," and a will be a 1d tuple, you should be able to write a[1,] and have the index be a 1d tuple. Of course, this will ultimately be your decision. > If we extend this rule to N-dimensional indexing, a[sequence], for > some sequence of integers whose elements are i, j, k, ..., should be > equivalent to a[i][j][k]..., and we can't make a[(1,2,3)] mean > a[1][2][3] while at the same time interpreting a[[1,2,3]] as a[1:4]. > (Sooner or later, you'll be passing a vector to the index. Then the > question will arise, should this have the same meaning as a tuple or > as a list. It's better if they all three mean the same.) I actually agree with you completely on this. I have used Jim Fulton's generic sequence indexing functions in order to be sure that the index can happily be either a tuple or a list or a matrix (which actually should be special cased and optimized at some point). Either I missed a point somewhere, or you have a different interpretation of my indexing scheme than I do. As some examples: a[1,2,3] == a[(1,2,3)] == a[[1,2,3]] == a[1][2][3] ^ if this becomes allowed a[(1,2,3),] == a[((1,2,3),)] == [a[1], a[2], a[3]] However, if there is sufficient disapproval of this method of indexing, a.select() is a reasonable alternative. Just my two cents worth. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org =================