From owner-matrix-sig Mon Aug 28 17:08:11 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA32423 for matrix-sig-people; Mon, 28 Aug 1995 17:08:11 -0400 Received: from qvarsx.er.usgs.GOV (qvarsx.er.usgs.gov [130.11.51.82]) by python.org (8.6.12/8.6.12) with ESMTP id RAA32408; Mon, 28 Aug 1995 17:07:58 -0400 Received: from localhost (dsjfqvarsa [130.11.51.73]) by qvarsx.er.usgs.GOV (EMAIL 1.2.1) with ESMTP id VAA04470; Mon, 28 Aug 1995 21:04:06 GMT Message-Id: <199508282104.VAA04470@qvarsx.er.usgs.GOV> To: Guido van Rossum cc: "Barry A. Warsaw" cc: "James L Fulton, Hydrologist, Reston, VA ", meta-sig@python.org cc: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Re: [PYTHON META-SIG] Re: SIGs In-reply-to: <199508281956.PAA06875@monty> Date: Mon, 28 Aug 1995 17:08:45 -0400 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk On Mon, 28 Aug 1995 15:56:13 -0400 Guido van Rossum said: > Now that Barry's created the matrix and gui sigs, I've added their > email addresses to the sigs web page on python.org. I guess Jim > should prepare a little introduction to be placed in the .info > files and also post that to the net so people are aware of the new > sigs. (I could then also change the description of the sigs in the > sigs web page.) How about: matrix-sig: Special Interest Group for Built-in Matrix Types There is growing interest in using Python to interface with mathematical and scientific computation libraries. A commonly needed data structure is a matrix of homogenous low-level objects, such as numbers. The purpose of this special interest group is to develop a proposal for and possibly coordinate the implementation of a new Python built-in Matrix type. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Aug 30 20:36:59 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id UAA13287 for matrix-sig-people; Wed, 30 Aug 1995 20:36:59 -0400 Received: from qvarsx.er.usgs.GOV (qvarsx.er.usgs.gov [130.11.51.82]) by python.org (8.6.12/8.6.12) with ESMTP id UAA13283 for ; Wed, 30 Aug 1995 20:36:55 -0400 Received: from localhost (dsjfqvarsa [130.11.51.73]) by qvarsx.er.usgs.GOV (EMAIL 1.2.1) with ESMTP id AAA27988 for ; Thu, 31 Aug 1995 00:32:51 GMT Message-Id: <199508310032.AAA27988@qvarsx.er.usgs.GOV> From: "James L Fulton, Hydrologist, Reston, VA " Subject: [PYTHON MATRIX-SIG] Matrix Special Interest Group Formed to: matrix-sig@python.org Date: Wed, 30 Aug 1995 20:37:35 -0400 Sender: owner-matrix-sig@python.org Precedence: bulk A Python Software Activity (PSA) Matrix Special Interest Group (SIG) has been formed: There is growing interest in using Python to interface with mathematical and scientific computation libraries. A commonly needed data structure is a matrix of homogenous low-level objects, such as numbers. The purpose of this special interest group is to develop a proposal for and possibly coordinate the implementation of a new Python built-in Matrix type. The SIG will conduct its discussions using the mailing list matrix-sig@python.org. You can join the matrix sig (and the mailing list) by sending a message to matrix-sig-request@python.org with a message body containing the word "subscribe". To obtain a copy of the matrix-sig archive, you can send a message to matrix-sig-request@python.org with a message body containing: get matrix-sig.archive To find out more about the PSA, see: http://www.python.org/psa/. To find out more about PSA SIGS, see: http://www.python.org/sigs/. To find out about additional commands that you can send to matrix-sig-request@python.org, see http://www.python.org/sigs/md_cmds.html/. -- -- Jim Fulton jfulton@usgs.gov (703) 648-5622 U.S. Geological Survey, Reston VA 22092 This message is being posted to obtain or provide technical information relating to my duties at the U.S. Geological Survey. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Aug 30 20:39:14 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id UAA13304 for matrix-sig-people; Wed, 30 Aug 1995 20:39:14 -0400 Received: from qvarsx.er.usgs.GOV (qvarsx.er.usgs.gov [130.11.51.82]) by python.org (8.6.12/8.6.12) with ESMTP id UAA13300 for ; Wed, 30 Aug 1995 20:39:11 -0400 Received: from localhost (dsjfqvarsa [130.11.51.73]) by qvarsx.er.usgs.GOV (EMAIL 1.2.1) with ESMTP id AAA28101 for ; Thu, 31 Aug 1995 00:35:07 GMT Resent-Message-Id: <199508310035.AAA28101@qvarsx.er.usgs.GOV> Path: qvarsx.er.usgs.gov!rsg1.er.usgs.gov!stc06.ctd.ornl.gov!fnnews.fnal.gov!usenet.eel.ufl.edu!news.mathworks.com!uhog.mit.edu!grapevine.lcs.mit.edu!usenet@lcs.mit.edu From: jjh@ling-ling.lcs.mit.edu (James Hugunin) Newsgroups: comp.lang.python Subject: [PYTHON MATRIX-SIG] The Matrix Object Proposal (very long) Date: 18 Aug 1995 17:16:55 GMT Organization: MIT Laboratory for Computer Science Lines: 292 Message-ID: <412hu7$92c@GRAPEVINE.LCS.MIT.EDU> Reply-To: jjh@ling-ling.lcs.mit.edu NNTP-Posting-Host: ling-ling.lcs.mit.edu Keywords: numeric, matrix Resent-To: matrix-sig@python.org Resent-Date: Wed, 30 Aug 1995 20:39:51 -0400 Resent-From: "Jim Fulton, U.S. Geological Survey" Apparently-To: Sender: owner-matrix-sig@python.org Precedence: bulk I apologize for the length of this posting, but you were warned in the subject header. There seems to be a fair amount of interest in the python community concerning the addition of numeric operations to python. My own desire is to have as large a library of matrix based functions available as possible (linear algebra, eigenfunctions, signal processing, statistics, etc.). In order to ensure that all of these libraries interoperate, there needs to be agreement on a basic matrix object that can be used to represent arrays of numbers. The following is a proposal for such an object. It can be viewed as an extension of the current array object to higher dimensions and numerical operations. The wheels have already been set in motion to create a Matrix SIG mailing-list for further discussion of this proposal, as well as other python numerics related issues. This should come on-line sometime next week and serious discussion of this proposal should probably be deferred until it can be done on that list. Summaries of that discussion will be posted to the main list as consensus develops. Below I specify the behavior of a Matrix object within python. The C (or FORTRAN) API to such an object is obviously of significant importance as this object is primarily intended as an interface to existing numerical libraries. I'm currently working on that proposal (at least the C part) as well, but I think everyone will agree that this post is long enough as it is. Note: Parts of this proposal are stolen from the python documentation for the array object, and from discussions on the net. I'd like to particularly thank Jim Fulton for letting me review (and steal ideas from) his current Matrix object and for providing feedback on the first draft of this proposal (though he certainly doesn't agree with everything in it, even now). Matrix Object This defines a new object type which can efficiently represent a multidimensional array of basic values (char, unsigned byte, signed byte, int, short, long, float, double, complex float and complex double). Matrices are defined as both a sequence type and a mapping type, and usually (unless it's a matrix of char's) as a number type. It should be noted that allowing a type to be both a sequence and a number and to behave properly for addition and multiplication, will require some very small patches to the ceval.c code. All of the examples use the following matrices: Assume A = [1,2,3], B = [11,12,13], C = [[1,4,9],[16,25,36]], M=[[ 1, 2, 3, 4, 5],[11,12,13,14,15],[21,22,23,24,25],[31,32,33,34,35]] **Numeric operations** "+", "-", "*", "/", "^"(as power), abs, unary"-", and unary"+" are defined as the corresponding element-wise arithmetic operations. Because these assume element-wise correspondence between the two matrices, the operandi must be of the same dimensions (however, note the dimensions-coercion section below). A+B = [12,14,16] A*B = [11,24,39] C*C = [[2,8,18],[32,50,72]] -A = [-1,-2,-3] Possible suggestions for the other numeric operations: "%" represents matrix multiplication (or vector dot product). "~" represents transposition. Other suggestions? **Sequence operations** Concatenation and multiplication sequence operations are not defined, but instead the arithmetic operations are invoked for the "+" and "*" operators. Indexing by both single indices and by slices is supported. All returned matrices are returned by reference rather than by value. len(M) is defined to return the size of the first dimension in in matrix. examples: len(M) -> 4 D = M[0] D[0] = 66 print M[0] [66,2,3,4,5] The semantics of returning a slice by reference is consistent with the mapping operations given below, however, this is inconsistent with the list objects semantics. This is open to debate. D = M[0:2] D[0][0] = 66 print M[0] [66,2,3,4,5] **Mapping operations** Recent newsgroup discussions have offered some convincing arguments for treating a matrix as a mapping type where sequences of ints are used as the keys. The sequence must be of length less than or equal to the numer of dimensions in the matrix. Each element of the sequence is either an integer or a sequence. If it is an integer, it returns the corresponding element for that dimension, if it is a sequence then it returns all of the elements given in the sequence. ie. A[0] -> 1, A[((1,2))] -> [2,3] M[0] -> [1,2,3,4,5] M[(0,3)] -> 4 M[((0,2),0)] -> [1,21] M[(range(1,3),range(2,4))] -> [[13,14],[23,24]] In order to simplify the syntax of these references, it would be valuable to change the syntax of python so that M[0,3] is equivalent to M[(0,3)]. This is a small change to the current indexing semantics which will give a reasonable meaning to a currently illegal syntactic item. When used as a setvalue, the corresponding elements of the given array will be set to the left hand side. All of these operations that return a matrix will return it by-reference, meaning that the elements of the new array will be references to the old one and changes to either array will effect the other. **Instantiation** The following function will be used to instantiate a matrix: Matrix(datasource(optional), d1, d2, ..., typecode='d') The typecodes are as follows: c - char (non-numeric) 1 - unsigned char b - signed char h - short i - int l - long f - float d - double F - complex float D - complex double The optional datasource can be a file object or a string object and will fill the matrix with the raw data contained therein. In this case the dimensions are required. The first dimension can be input as -1, in which case the first dimension will be set to be as large as necessary to hold the entire contents of the file or string. The optional datasource can also be a sequence, in which case the dimensions of the array do not need to be passed in and are instead determined by the hierarchical structure of the sequence. examples: Matrix([range(3),(4,5,6)], 'i') -> [[0,1,2],[4,5,6]] Matrix(2,3) -> [[0,0,0],[0,0,0]] Matrix('abcdef', 2,3, 'u') -> [[97,98,99],[100,101,102]] Matrix('abcdef', -1,2, 'c') -> ['ab', 'cd', 'ef'] **Coercion** Dimension coercion:The following is a proposal to make the matrix object behave as closely as possible to matrices in matlab (to make it easy to convert matlab code to python), and to generalize this behavior to arbitrarily high dimensional objects. If a matrix of fewer dimensions than that required is provided, then the matrix can be converted to more dimensions by making copies of itself. For purposes of dimension coercion, an int or a float is considered a 0-d matrix. ie. A+1 -> [2,3,4], A+C -> [[2,6,12],[17,27,39]] B^2 -> [121,144,169] vs. B^A -> [11, 144, 2197] Simliar coercion should be performed for most functions acting on matrices. Helper functions will be provided to make this easy, however there is no way to enforce this. The following is a proposed standard to allow functions to be applied to higher-dimensional objects in a uniform way. If a function is defined to operate on scalars (say sqrt()), then when applied to a matrix it should apply itself to every element in the matrix. ie. sqrt(C) -> [[1,2,3],[4,5,6]] This is generalized to higher dimensions so that if a function is provided with a matrix of more dimensions that it requires, that function will apply itself to each appropriately sized unit in the matrix. ie. sum(A) -> 6 whereas sum(C) -> [14,77] Type coercion: No type coercion will be performed. However, it is possible to have a function defined to work on multiple types (operator overloading). So sqrt might be defined on matrices of both floats and doubles in which case it will work properly when given either matrix as input, but will return a type error if applied to a matrix of ints. **Basic Data Items and Methods (non-numeric)** typecode - The typecode character used to create the matrix itemsize - The length in bytes of one matrix item in the internal representation dimensions - A sequence containing the size of the matrix along each of its dimensions copy - Returns a copy of the matrix. This is a good way to take an array returned by reference and turn it into one returned by value. transpose - The transpose of the matrix (not needed if "~" is used for transpose) byteswap - Useful for reading data from a file written on a machine with a different byte order typecast(t) - Returns a new matrix of the same dimensions where each item is cast (using C typecasting rules) from the matrix's type to the new type t. tofile(f) - Write all items (as machine values) to the file object f. tolist() - Convert the matrix to an ordinary (possibly nested) list with the same items. tostring() - Convert the array to an array of machine values and return the string representation (the same sequence of bytes that would be written to a file by the tofile() method.) Note: the append, fromlist, etc. methods are not present because it is assumed that a matrix object is of constant size (there are some strong efficiency reasons for this assumption but this is likely to be another source of arguments). **Basic functions taking matrix arguments (non-numeric)** Since it isn't possible to append to a matrix (or efficient if it's decided it should be possible) I thought that some sort of function for producing a new matrix from a list of matrices would be useful. So I'm proposing the following function which is very similar in spirit to the string.join operation. concat(l) - l must be a list of matrices, and each matrix in l must have the same number of dimensions, as well as the same size for every dimension except for the first. This will return a new matrix M whose first dimension is the sum of the first dimensions of all the matrices in the list, and whose data is filled from the data in all of these matrices. ie. concat([A,B]) -> [1,2,3,11,12,13] This is different from Matrix([A,B]) -> [[1,2,3],[11,12,13]] **Basic functions taking matrix arguments (numeric)** It is hoped that there will be a large variety of special purpose modules to do fancy numeric operations on matrices. There should also be a basic set of operations provided in the Matrix module (the module with the instantiation function). Because type coercion is not going to be performed, it is important to decide both on the list of functions and on the basic types they should support. The following is a very preliminary list of the functions I consider most fundamental. rng(start, stop, step(optional), typecode='d') where all points in the range must be strictly less than the stop point, according to the semantics of range(). This function should possibly be added to the instantiation options. ie. rng(0,0.4) = [0,0.1,0.2,0.3]; rng(0,0.1,0.41) = [0,0.1,0.2,0.3,0.4] scalar operations (defined on floats, doubles, and all complex): everything in mathmodule.c round scalar operations (defined on all numeric types): sign vector to scalar operations (defined on all numeric types): max, min - Should these return the index of the max or min? sum, prod vector to vector operations (defined on all numeric types): cumsum, cumprod ----- Jim Hugunin - hugunin@mit.edu Laboratory for Computer Science Massachusetts Institute of Technology ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Sep 11 19:03:26 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id TAA01503 for matrix-sig-people; Mon, 11 Sep 1995 19:03:26 -0400 Received: from servrcolkr.cr.usgs.gov (servrcolkr.cr.usgs.gov [136.177.112.5]) by python.org (8.6.12/8.6.12) with ESMTP id TAA01494 for ; Mon, 11 Sep 1995 19:03:20 -0400 Received: from localhost (dsjfqvarsa.er.usgs.gov [130.11.51.73]) by servrcolkr.cr.usgs.gov (EMAIL 1.2.1) with ESMTP id XAA13015 for ; Mon, 11 Sep 1995 23:04:40 GMT Message-Id: <199509112304.XAA13015@servrcolkr.cr.usgs.gov> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Let's get going In-reply-to: <412hu7$92c@GRAPEVINE.LCS.MIT.EDU> Date: Mon, 11 Sep 1995 19:04:39 -0400 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk Well, now that we're all here, let's get started. :-) As you know, the purpose of this SIG is to develop a proposal (and possible an implementation) for a new Python matrix type. I'd like to use Jim Hugunin's proposal as a starting point. I give my response to his proposal below. Before beginning this, however, I thought I'd summarize my involvement to date, so you know where I'm coming from. At the python workshop, I presented some work in progress on a Fortran (77) Interface Description Language (FIDL) for Python. This tool allows one to very quickly create Python modules that call Fortran libraries given brief high-level descriptions of the library routines to be called. The primary data structure in Fortran 77 is the fortran array, which is a multi-dimensional block of homogenous data elements. Of course, many interesting C routines, especially in mathematic and scientific domains, use this same data structure. To facilitate interfacing to these sorts of routines, I wanted an an efficient Python implementation of the same sort of data structure. In particular: o I wanted a python built-in data type containing a data structure that I could pass directly to Fortran or C routines. I did not want to have to do alot of data conversion or copying on each call. For example, if a Fortran routine returns a matrix, then it should be possible to pass this matrix to another Fortran routine without reallocating a data structure or converting the entire matrix. o I wanted multi-dimensional access as would be familiar to a python programmer. That is, if I have two-dimensional arrays, a and b, then: a[i][j] = b[k][l] should work as expected and should work efficiently. In general, given an n-dimensional matrix, m, if n > 1, then m[i] should return an n-1 dimensional matrix, and if n = 1, then m[i] should return a scalar. Note that for "m[i][j] = spam" to work correctly, assignment to the jth element of m[i] must be reflected in m. o I need to support *all* data types supported by Fortran 77, including strings. Not only do I have to interface to mathematical routines, but I need to interface to "legacy" systems (Some of which are still being written :). What I came up with was a fairly simple implementation of a matrix type that was a pure container type. My matrix implementation currently provides no numeric behavior, although a matrix type cries out for numeric behavior. My matrix implementation includes the following features: o Matrix objects have an internal data pointer that points to a homogenous block of memory. This pointer (or some offset from it, of course) can be passed directly to C or Fortran. o The internal data structure may be shared among multiple matrix objects. In particular, a __getitem__ from a n-dimensional matrix, where n>1, returns a matrix object that shares the same data (at an appropriate offset) as the original matrix. This shared data structure is reference counted, so if you have: a=Matrix([[1,2,3],[4,5,6],[7,8,9]]) b=a[1] # b == Matrix([4,5,6]) del a b is not invalidated by the third line. Note that if you have: a=Matrix([[1,2,3],[4,5,6],[7,8,9]]) b=Matrix([11,22,33]) a[1]=b a[1][1]=99 You end up with: a == Matrix([[1,2,3],[11,99,33],[7,8,9]]) b == Matrix([11,22,33]) That is, assigning to a matrix copies data. So assignment is by copy, even though access is by reference. (As you can see, matrices can be created from sequences of sequences. Matrices can also be assigned from arbitrary sequences of the right dimension or level of nesting.) o I chose to preserve the copy semantics of slices from lists. So in: a=Matrix([[1,2,3],[4,5,6],[7,8,9]]) b=a[1][:] # b == Matrix([4,5,6]) b[1]=99 # does not affect a The third line does not change a. So that's where I'm coming from. I think this type is generally useful, which is why I helped start this SIG. I'm pretty open on how and if matrices should have numeric behavior. With that, here are my comments on Jim's proposal. On 18 Aug 1995 17:16:55 GMT James Hugunin said: > I apologize for the length of this posting, but you were warned in the > subject header. > > There seems to be a fair amount of interest in the python community > concerning the addition of numeric operations to python. My own desire is > to have as large a library of matrix based functions available as possible > (linear algebra, eigenfunctions, signal processing, statistics, etc.). In > order to ensure that all of these libraries interoperate, there needs to > be agreement on a basic matrix object that can be used to represent arrays > of numbers. The following is a proposal for such an object. It can be > viewed as an extension of the current array object to higher dimensions > and numerical operations. > > The wheels have already been set in motion to create a Matrix SIG > mailing-list for further discussion of this proposal, as well as other > python numerics related issues. This should come on-line sometime next > week and serious discussion of this proposal should probably be deferred > until it can be done on that list. Summaries of that discussion will be > posted to the main list as consensus develops. > > Below I specify the behavior of a Matrix object within python. The C (or > FORTRAN) API to such an object is obviously of significant importance as > this object is primarily intended as an interface to existing numerical > libraries. I'm currently working on that proposal (at least the C part) > as well, but I think everyone will agree that this post is long enough as > it is. > > Note: Parts of this proposal are stolen from the python documentation for > the array object, and from discussions on the net. I'd like to > particularly thank Jim Fulton for letting me review (and steal ideas from) > his current Matrix object and for providing feedback on the first draft of > this proposal (though he certainly doesn't agree with everything in it, > even now). We aren't that far apart now, at least not on things that matter to me. :-) > > Matrix Object > > This defines a new object type which can efficiently represent a > multidimensional array of basic values (char, unsigned byte, signed byte, > int, short, long, float, double, complex float and complex double). > Matrices are defined as both a sequence type and a mapping type, and > usually (unless it's a matrix of char's) as a number type. It should be > noted that allowing a type to be both a sequence and a number and to > behave properly for addition and multiplication, will require some very > small patches to the ceval.c code. While I think the patches are OK, I don't think they're necessary, so if anyone objects to this proposal on the grounds that changes are required to ceval.c, you should not be concerned. > All of the examples use the following matrices: > Assume A = [1,2,3], B = [11,12,13], C = [[1,4,9],[16,25,36]], > M=[[ 1, 2, 3, 4, 5],[11,12,13,14,15],[21,22,23,24,25],[31,32,33,34,35]] > > > **Numeric operations** > > "+", "-", "*", "/", "^"(as power), abs, unary"-", and unary"+" are defined > as the corresponding element-wise arithmetic operations. Because these I'd rather see multiplication and division not be elementwise. But I don't feel strongly about this. > assume element-wise correspondence between the two matrices, the operandi > must be of the same dimensions (however, note the dimensions-coercion > section below). > > A+B = [12,14,16] > A*B = [11,24,39] > C*C = [[2,8,18],[32,50,72]] > -A = [-1,-2,-3] > > Possible suggestions for the other numeric operations: > "%" represents matrix multiplication (or vector dot product). If we go with elementwise interpretation for * and /, then I think that all operators that make sense for floating point numbers should have an elementwise interpretation, so m%spam should compute a matrix of mods. Perhaps one would carry this same argument to bitwise operators, since you could have matrices of integers. If we go with elementwise interpretation of numeric operations, then I'm inclined to use functional, rather than matrix notation for matrix multiplication, dot products, and so on. > "~" represents transposition. > Other suggestions? Perhaps, as a compromize, there could be a special method or operator that creates a reference copied version of a matrix that provides a different semantics. For example, perhaps the default matrix type could provide standard matrix interpretation for * and /, but there could be a method, elementwise that would allow: m.elementwise() + a to do elementwise arithmentic. IMPORTANT NOTE: Mixed-mode arithmetic between matrices and scalars should be defined. > > **Sequence operations** > > Concatenation and multiplication sequence operations are not defined, but > instead the arithmetic operations are invoked for the "+" and "*" > operators. Right. Note if we don't need elementwise bitwise operators, then we could concatinate with "|", which I find most natural in a matrix context: spam = foo | bar I suppose one could even repeat with >> or <<, but this is not so natural. > Indexing by both single indices and by slices is supported. > All returned matrices are returned by reference rather than by value. > len(M) is defined to return the size of the first dimension in in matrix. > > > examples: > len(M) -> 4 > > D = M[0] > D[0] = 66 > print M[0] > [66,2,3,4,5] Right, for my matrix type. > The semantics of returning a slice by reference is consistent with the > mapping operations given below, however, this is inconsistent with the > list objects semantics. This is open to debate. > > D = M[0:2] > D[0][0] = 66 > print M[0] > [66,2,3,4,5] I'm not sure what you were trying to say here, but this example holds only if M is a list of lists. If M is one of my matrices, then M is unchanged. Perhaps that is your point. Note that I tried to maintain the spirit of list semantics, in which the slice operator has copy semantics, but with a list, the things you are copying are, themselves, references. Anything is open to debate, but how else could this work, and satisfy other requirements at the same time? > > **Mapping operations** > > Recent newsgroup discussions have offered some convincing arguments for > treating a matrix as a mapping type where sequences of ints are used as > the keys. The sequence must be of length less than or equal to the numer > of dimensions in the matrix. Each element of the sequence is either an > integer or a sequence. If it is an integer, it returns the corresponding > element for that dimension, if it is a sequence then it returns all of the > elements given in the sequence. > > ie. A[0] -> 1, A[((1,2))] -> [2,3] > M[0] -> [1,2,3,4,5] > M[(0,3)] -> 4 > M[((0,2),0)] -> [1,21] > M[(range(1,3),range(2,4))] -> [[13,14],[23,24]] A way of visualizing this is as multi-dimensional slices that let you, for example, take a rectangular slice out of a 2-d matrix. > In order to simplify the syntax of these references, it would be valuable > to change the syntax of python so that M[0,3] is equivalent to M[(0,3)]. > This is a small change to the current indexing semantics which will give a > reasonable meaning to a currently illegal syntactic item. > > When used as a setvalue, the corresponding elements of the given array > will be set to the left hand side. All of these operations that return a > matrix will return it by-reference, meaning that the elements of the new > array will be references to the old one and changes to either array will > effect the other. Note that if one wants a reference, not a copy, one can always apply a copy operator, [:] as one often does not with lists. > > > **Instantiation** > > The following function will be used to instantiate a matrix: > Matrix(datasource(optional), d1, d2, ..., typecode='d') > > The typecodes are as follows: > c - char (non-numeric) > 1 - unsigned char > b - signed char > h - short > i - int > l - long > f - float > d - double > F - complex float > D - complex double > > The optional datasource can be a file object or a string object and will > fill the matrix with the raw data contained therein. In this case the > dimensions are required. The first dimension can be input as -1, in which > case the first dimension will be set to be as large as necessary to hold > the entire contents of the file or string. > > The optional datasource can also be a sequence, in which case the > dimensions of the array do not need to be passed in and are instead > determined by the hierarchical structure of the sequence. > > examples: > Matrix([range(3),(4,5,6)], 'i') -> [[0,1,2],[4,5,6]] > Matrix(2,3) -> [[0,0,0],[0,0,0]] > Matrix('abcdef', 2,3, 'u') -> [[97,98,99],[100,101,102]] > Matrix('abcdef', -1,2, 'c') -> ['ab', 'cd', 'ef'] Note that I have found it convenient to implement matrices so that 1-d matrices of chars behave alot like strings, as I often pass these to or get these back from Fortran routines in which they are just that. > > **Coercion** > > Dimension coercion:The following is a proposal to make the matrix object > behave as closely as possible to matrices in matlab (to make it easy to > convert matlab code to python), and to generalize this behavior to > arbitrarily high dimensional objects. > > If a matrix of fewer dimensions than that required is provided, then the > matrix can be converted to more dimensions by making copies of itself. > For purposes of dimension coercion, an int or a float is considered a 0-d > matrix. > > ie. A+1 -> [2,3,4], A+C -> [[2,6,12],[17,27,39]] > B^2 -> [121,144,169] vs. B^A -> [11, 144, 2197] I agree wrt scalars. > Simliar coercion should be performed for most functions acting on > matrices. Helper functions will be provided to make this easy, however > there is no way to enforce this. The following is a proposed standard to > allow functions to be applied to higher-dimensional objects in a uniform > way. > > If a function is defined to operate on scalars (say sqrt()), then when > applied to a matrix it should apply itself to every element in the matrix. > > ie. sqrt(C) -> [[1,2,3],[4,5,6]] > > This is generalized to higher dimensions so that if a function is provided > with a matrix of more dimensions that it requires, that function will > apply itself to each appropriately sized unit in the matrix. > > ie. sum(A) -> 6 whereas sum(C) -> [14,77] I am a bit concerned that this will lead to hard to find bugs. This also put's extra burden on the function implementor. I realize (since you explained it to me in a separate note :) your concern that this is needed for efficiency, otherwise the map operator, or perhaps a specialized matrix map operator could be used. > > Type coercion: No type coercion will be performed. However, it is > possible to have a function defined to work on multiple types (operator > overloading). So sqrt might be defined on matrices of both floats and > doubles in which case it will work properly when given either matrix as > input, but will return a type error if applied to a matrix of ints. > > > **Basic Data Items and Methods (non-numeric)** > > typecode - The typecode character used to create the matrix > > itemsize - The length in bytes of one matrix item in the internal > representation > > dimensions - A sequence containing the size of the matrix along each of ^^^^^^^^^^ tuple > its dimensions > > copy - Returns a copy of the matrix. This is a good way to take an array > returned by reference and turn it into one returned by value. This is not needed. Use [:]. > transpose - The transpose of the matrix (not needed if "~" is used for > transpose) I'm thinking that this (or ~) should return by reference. > byteswap - Useful for reading data from a file written on a machine with a > different byte order Hm. Does this operate in-place or return a value? You know, there really *should* be a matrix map operator. This should function like the standard map operator, but it will generate a matrix and it will probably want to be told at what level/dimension to apply the mapping function. In fact, this would give you arbitrary elementwise operations given scalar functions. > > typecast(t) - Returns a new matrix of the same dimensions where each item > is cast (using C typecasting rules) from the matrix's type to the new type > t. Our constructor already gives you this: new_matrix=Matrix(old_matrix,new_type) > tofile(f) - Write all items (as machine values) to the file object f. > > tolist() - Convert the matrix to an ordinary (possibly nested) list with > the same items. > > tostring() - Convert the array to an array of machine values and return > the > string representation (the same sequence of bytes that would > be written to a file by the tofile() method.) > > Note: the append, fromlist, etc. methods are not present because it is > assumed that a matrix object is of constant size (there are some strong > efficiency reasons for this assumption but this is likely to be another > source of arguments). > > **Basic functions taking matrix arguments (non-numeric)** > > Since it isn't possible to append to a matrix (or efficient if it's > decided it should be possible) I thought that some sort of function for > producing a new matrix from a list of matrices would be useful. So I'm > proposing the following function which is very similar in spirit to the > string.join operation. > > concat(l) - l must be a list of matrices, and each matrix in l must have > the same number of dimensions, as well as the same size for every > dimension except for the first. This will return a new matrix M whose > first dimension is the sum of the first dimensions of all the matrices in > the list, and whose data is filled from the data in all of these matrices. > > ie. concat([A,B]) -> [1,2,3,11,12,13] > > This is different from Matrix([A,B]) -> [[1,2,3],[11,12,13]] > > > **Basic functions taking matrix arguments (numeric)** > > It is hoped that there will be a large variety of special purpose modules > to do fancy numeric operations on matrices. There should also be a basic > set of operations provided in the Matrix module (the module with the > instantiation function). Because type coercion is not going to be > performed, it is important to decide both on the list of functions and on > the basic types they should support. The following is a very preliminary > list of the functions I consider most fundamental. > > rng(start, stop, step(optional), typecode='d') where all points in the > range must be strictly less than the stop point, according to the > semantics of range(). This function should possibly be added to the > instantiation options. > > ie. rng(0,0.4) = [0,0.1,0.2,0.3]; rng(0,0.1,0.41) = [0,0.1,0.2,0.3,0.4] > Hm. Why not: Matrix.range(start=1,stop,step=1, typecode='d') and Matrix.range([(start=1,stop,step=1), ...], typecode='d') That is: call it range, and give it same semantics, and include a multi-dimensional range operator, mrange, that takes a sequence of range specs. > > scalar operations (defined on floats, doubles, and all complex): > everything in mathmodule.c > round > > scalar operations (defined on all numeric types): > sign > > vector to scalar operations (defined on all numeric types): > max, min - Should these return the index of the max or min? > sum, prod > > vector to vector operations (defined on all numeric types): > cumsum, cumprod Comments anyone? -- Jim Fulton jfulton@usgs.gov (703) 648-5622 U.S. Geological Survey, Reston VA 22092 This message is being posted to obtain or provide technical information relating to my duties at the U.S. Geological Survey. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Sep 12 03:43:15 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id DAA03226 for matrix-sig-people; Tue, 12 Sep 1995 03:43:15 -0400 Received: from big.fishnet.net (root@big.fishnet.net [205.216.133.3]) by python.org (8.6.12/8.6.12) with ESMTP id DAA03222 for ; Tue, 12 Sep 1995 03:43:11 -0400 Received: from port32.fishnet.net (port32.fishnet.net [205.216.133.232]) by big.fishnet.net (8.6.12/8.6.9) with SMTP id AAA31805 for ; Tue, 12 Sep 1995 00:46:03 GMT Message-Id: <199509120046.AAA31805@big.fishnet.net> X-Sender: graham@fishnet.net X-Mailer: Windows Eudora Pro Version 2.1.2 Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Tue, 12 Sep 1995 00:46:50 -0700 To: matrix-sig@python.org From: Graham Hughes Subject: Re: [PYTHON MATRIX-SIG] Let's get going Sender: owner-matrix-sig@python.org Precedence: bulk At 07:04 PM 9/11/95 -0400, you wrote: > o I wanted a python built-in data type containing a data structure > that I could pass directly to Fortran or C routines. I did not > want to have to do alot of data conversion or copying on each > call. For example, if a Fortran routine returns a matrix, then > it should be possible to pass this matrix to another Fortran > routine without reallocating a data structure or converting the > entire matrix. Instant problem right there. To the best of my knowledge (and I've read this in several books), Fortran and C use completely different ways of storing multidimensional arrays. To wit: given the array m, like this: 3 4 5 6 7 8 9 0 3 -2 4 1 There are two ways of storing this array in a continuous memory location; by rows or by columns. By rows looks like this: '3 4 5 6 7 8 9 0 3 -2 4 1', and is used by both C and Pascal. By columns looks like this: '3 7 3 4 8 -2 5 9 4 6 0 1', and is used by Fortran. In other words, to pass to either Fortran or C functions you're going to need to know which type of function you have, and then convert the internal representation. This can be done, but it greatly complicates things. >> "+", "-", "*", "/", "^"(as power), abs, unary"-", and unary"+" are defined >> as the corresponding element-wise arithmetic operations. Because these > >I'd rather see multiplication and division not be elementwise. But I >don't feel strongly about this. Ah, but I'd prefer it be elementwise because it allows me to do wonderful APLish things with it :) A good reason for having multiplication/division (same thing on a mathematical level, although much stickier anywhere else) elementwise is that addition and subtraction already are, and it would be confusing to do it otherwise. However, a better reason is the inherent complications with matrix multiplications and divisions; specifically, multiplications require that the matricies have their rows and columns match in specific ways that are not immediately intuitive, and division is worse; not only does one need to be square, but the square matrix must be invertible, a condition not always fulfilled. Simple elementwise multiplication and division eliminates these non-intutive requirements in exchange for the same requirements that you have with addition and subtraction; both matricies must simply be of equal dimension. I'm not by any means suggesting that we not incorporate the matrixwise multiplication and division, but perhaps it would be better to have these as separate functions, both to avoid the above problems and to highlight their non-intuitive natures (I mean, really, there *still* isn't an easy method to invert a matrix, and the dimension requirements for matrix multiplication probably took us all a while to get used to)... >> Possible suggestions for the other numeric operations: >> "%" represents matrix multiplication (or vector dot product). > >If we go with elementwise interpretation for * and /, then I think >that all operators that make sense for floating point numbers should >have an elementwise interpretation, so m%spam should compute a matrix >of mods. Perhaps one would carry this same argument to bitwise >operators, since you could have matrices of integers. I agree. From experience with several kinds of matricies, I'm inclined to think that something similar to APL would be a really good idea... Given that, we *could* implement matrix multiplication, dot products, etc. as something similar to APL's outer and inner product functions... This would make it more general than simply matrix multiplication and therefore more useful, and the matrix multiplication could simply be a function that hardcodes a certain inner product operation... For those that haven't been exposed to APL, the inner and outer product functions are ways of implementing element-by-element functions to generalized tensors (although they're usually used on vectors and matricies only). An outer product returns a table of results of an operation foo on all pairs of one from matrix bar and one from matrix baz. That is, it does foo(bar[i],baz[j]) for each 0 <= i < len(bar) and 0 <= j < len(baz). Naturally, it works best in this form on vectors, but there is nothing preventing it being implemented on matricies; mail me for additional information on how to do this, as I need time to grovel over the manuals again so I can remember how to do it :) An inner product is similar; it takes two functions, foo and bar, and two matricies, baz and quux. It performs reduce(foo, (SUBSET1)) where SUBSET1 is bar(baz[i],quux[i]) for each 0 <= i < len(baz). The entirety can be formulated more like reduce(foo, map(bar, baz[0][0:], quux[0][0:])) but it is a little less clear IMHO. Plus, I'm not really sure I got those slices right; they're supposed to be the first row... One intriguing possiblity with using the APL stuff is it allows mixing and matching between matrix dimensions; there is nothing that says that you cannot perform an outer product on a matrix and a vector. This is even occasionally useful. (Some may ask, why the APL stuff? This is because I am convinced that APL is doing the Right Thing when it comes to matricies. J supports an even richer set of 'adverbs', but I have no intention of infecting Python with that kind of complexity ;) > >If we go with elementwise interpretation of numeric operations, then >I'm inclined to use functional, rather than matrix notation for matrix >multiplication, dot products, and so on. This matches with what I said *waaay* back before the inner and outer product stuff. >Perhaps, as a compromize, there could be a special method or operator >that creates a reference copied version of a matrix that provides a >different semantics. For example, perhaps the default matrix type >could provide standard matrix interpretation for * and /, but there >could be a method, elementwise that would allow: > > m.elementwise() + a > >to do elementwise arithmentic. I suppose if you use standard matrix interpretation for * and /, but if we don't and use elementwise stuff instead for the reasons already mentioned, than we can completely avoid this problem... >IMPORTANT NOTE: Mixed-mode arithmetic between matrices and scalars > should be defined. Here, we can stand on math; it is similar to elementwise arithmetic, i.e. m + s where m is a matrix and s is a scalar adds s to every element of m. This seems acceptable, and even intutive, to me. >> Concatenation and multiplication sequence operations are not defined, but >> instead the arithmetic operations are invoked for the "+" and "*" >> operators. > >Right. Note if we don't need elementwise bitwise operators, then we >could concatinate with "|", which I find most natural in a matrix >context: > > spam = foo | bar I can see your point... Dear me, looks like what we need are some APLlike functions... (kicks self) I happen to like having the "|" operator do bitwise stuff, simply because somebody's going to ask "well, what does & do?". They complement each other nicely. >I suppose one could even repeat with >> or <<, but this is not so natural. Yeah, plus << can give you a really *quick* multiplication by 2. >> Recent newsgroup discussions have offered some convincing arguments for >> treating a matrix as a mapping type where sequences of ints are used as >> the keys. The sequence must be of length less than or equal to the numer >> of dimensions in the matrix. Each element of the sequence is either an >> integer or a sequence. If it is an integer, it returns the corresponding >> element for that dimension, if it is a sequence then it returns all of the >> elements given in the sequence. >> >> ie. A[0] -> 1, A[((1,2))] -> [2,3] >> M[0] -> [1,2,3,4,5] >> M[(0,3)] -> 4 >> M[((0,2),0)] -> [1,21] >> M[(range(1,3),range(2,4))] -> [[13,14],[23,24]] > >A way of visualizing this is as multi-dimensional slices that let you, >for example, take a rectangular slice out of a 2-d matrix. Definitely useful; the most intutive (although not necessarily the fastest) way of calculating a determinant is to use multi-dimensional slices; I was stymied by the 'normal' Python list-of-lists implementation on this note when trying to implement a matrix inverter. >> **Instantiation** >> >> The following function will be used to instantiate a matrix: >> Matrix(datasource(optional), d1, d2, ..., typecode='d') >> >> The typecodes are as follows: >> c - char (non-numeric) >> 1 - unsigned char >> b - signed char >> h - short >> i - int >> l - long >> f - float >> d - double >> F - complex float >> D - complex double I'm not fond of this, as it breaks normal Python typing; let the matrix figure out what's there. Matter of fact, might be a good idea to make the matrix a general container like lists are now, so you can store, say, a tuple representing a complex number in there? As it stands, with these restrictions you can forget it. However, this makes it slightly more work for those wonderful Fortran library routines. >> **Coercion** >> >> Dimension coercion:The following is a proposal to make the matrix object >> behave as closely as possible to matrices in matlab (to make it easy to >> convert matlab code to python), and to generalize this behavior to >> arbitrarily high dimensional objects. >> >> If a matrix of fewer dimensions than that required is provided, then the >> matrix can be converted to more dimensions by making copies of itself. >> For purposes of dimension coercion, an int or a float is considered a 0-d >> matrix. >> >> ie. A+1 -> [2,3,4], A+C -> [[2,6,12],[17,27,39]] >> B^2 -> [121,144,169] vs. B^A -> [11, 144, 2197] > >I agree wrt scalars. > Interesting... I like it. So what happens if I convert a matrix of [1,2,3] into a 4x2 ([[x,x,x,x],[x,x,x,x]])? Do we get it wrapping around like [[1,2,3,1],[2,3,1,2]]? >> Simliar coercion should be performed for most functions acting on >> matrices. Helper functions will be provided to make this easy, however >> there is no way to enforce this. The following is a proposed standard to >> allow functions to be applied to higher-dimensional objects in a uniform >> way. >> >> If a function is defined to operate on scalars (say sqrt()), then when >> applied to a matrix it should apply itself to every element in the matrix. >> >> ie. sqrt(C) -> [[1,2,3],[4,5,6]] >> >> This is generalized to higher dimensions so that if a function is provided >> with a matrix of more dimensions that it requires, that function will >> apply itself to each appropriately sized unit in the matrix. >> >> ie. sum(A) -> 6 whereas sum(C) -> [14,77] Definitely. Back to the elementwise operations. >I am a bit concerned that this will lead to hard to find bugs. This >also put's extra burden on the function implementor. I realize (since >you explained it to me in a separate note :) your concern that this is >needed for efficiency, otherwise the map operator, or perhaps a >specialized matrix map operator could be used. I see your point. It might be better to use that matrix map operator... Best of both worlds, I guess. Oh, for APL's parallelism ;) That might not be a bad idea, but I think it would be a nasty bit of work for the interpreter and underlying source code. >> >> Type coercion: No type coercion will be performed. However, it is >> possible to have a function defined to work on multiple types (operator >> overloading). So sqrt might be defined on matrices of both floats and >> doubles in which case it will work properly when given either matrix as >> input, but will return a type error if applied to a matrix of ints. Well, if we simply let the caller figure out what it's got in it, like what I suggested above, this would be unnecessary. Let sqrt() (or indirectly, Python) figure out whether it's a double, int, or a complex. >> byteswap - Useful for reading data from a file written on a machine with a >> different byte order > >Hm. Does this operate in-place or return a value? > >You know, there really *should* be a matrix map operator. This should >function like the standard map operator, but it will generate a matrix >and it will probably want to be told at what level/dimension to apply >the mapping function. In fact, this would give you arbitrary >elementwise operations given scalar functions. Or, we can always use inner/outer products... :) >> typecast(t) - Returns a new matrix of the same dimensions where each item >> is cast (using C typecasting rules) from the matrix's type to the new type >> t. > >Our constructor already gives you this: > > new_matrix=Matrix(old_matrix,new_type) > Unnecessary if the matrix is a container. >> concat(l) - l must be a list of matrices, and each matrix in l must have >> the same number of dimensions, as well as the same size for every >> dimension except for the first. This will return a new matrix M whose >> first dimension is the sum of the first dimensions of all the matrices in >> the list, and whose data is filled from the data in all of these matrices. >> >> ie. concat([A,B]) -> [1,2,3,11,12,13] Hm. Seems to run into difficulty with >1D matricies. Ideally, concat should take 2 3 4 5 6 7 and 5 6 7 8 9 10 and return 2 3 4 5 6 7 5 6 7 8 9 10 But the given behaviour of concat depends greatly on whether [[x],[y]] is stored by rows or by columns, ie. is it like this: [ | , | ] V V [x] [y] or like this: [ -> [x] , -> [y] ] This is a conceptual difficulty. >> >> This is different from Matrix([A,B]) -> [[1,2,3],[11,12,13]] >> >> >> **Basic functions taking matrix arguments (numeric)** >> >> It is hoped that there will be a large variety of special purpose modules >> to do fancy numeric operations on matrices. There should also be a basic >> set of operations provided in the Matrix module (the module with the >> instantiation function). Because type coercion is not going to be >> performed, it is important to decide both on the list of functions and on >> the basic types they should support. The following is a very preliminary >> list of the functions I consider most fundamental. >> >> rng(start, stop, step(optional), typecode='d') where all points in the >> range must be strictly less than the stop point, according to the >> semantics of range(). This function should possibly be added to the >> instantiation options. >> >> ie. rng(0,0.4) = [0,0.1,0.2,0.3]; rng(0,0.1,0.41) = [0,0.1,0.2,0.3,0.4] >> > >Hm. Why not: > > Matrix.range(start=1,stop,step=1, typecode='d') > >and > > Matrix.range([(start=1,stop,step=1), ...], typecode='d') > >That is: > > call it range, and give it same semantics, and > > include a multi-dimensional range operator, mrange, that takes a > sequence of range specs. > Sounds good to me. >Comments anyone? Ditto. Anyone see something they'd like to pull apart (besides APL...)? Graham Graham Hughes Home page http://www.fishnet.net/~graham/ ``I think it would be a good idea.'' -- Mahatma Ghandi, when asked what he thought of Western civilization finger for PGP public key. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Sep 12 05:16:32 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id FAA03398 for matrix-sig-people; Tue, 12 Sep 1995 05:16:32 -0400 Received: from tarzan.math.jyu.fi (ji@tarzan.math.jyu.fi [130.234.32.1]) by python.org (8.6.12/8.6.12) with ESMTP id FAA03394 for ; Tue, 12 Sep 1995 05:16:15 -0400 Received: by tarzan.math.jyu.fi (1.37.109.16/16.2) id AA077177434; Tue, 12 Sep 1995 12:17:14 +0300 X-Charset: ISO_8859-1 Date: Tue, 12 Sep 1995 12:17:13 +0300 (EETDST) From: Jonne Itkonen To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Let's get going In-Reply-To: <199509120046.AAA31805@big.fishnet.net> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk On Tue, 12 Sep 1995, Graham Hughes wrote: > >I'd rather see multiplication and division not be elementwise. But I > >don't feel strongly about this. > > Ah, but I'd prefer it be elementwise because it allows me to do wonderful > APLish things with it :) I prefer the mathematical, not elementwise multiplication. We are handling matrixes here (not tables!). Besides, one doesn't want to define the * operator for complex numbers to do 'elementwise' multiplication, does she? It would be meaningless! In my opinion, the overloading of operators is to make things more clear than to mess them up. The elementwise multiplication by * operator would be clear to anyone with no knowledge of matrices, but not to someone who knows what matrices are. Perhaps a table object should be developed for elementwise calculation? > A good reason for having multiplication/division (same thing on a > mathematical level, although much stickier anywhere else) elementwise is > that addition and subtraction already are, and it would be confusing to do That is not a good reason. It's more confusing to use mathematical definition for some, and non-mathematical definitions for other operators, as in your '+ vs. *' example. -Jonne URL: http://www.math.jyu.fi/~ji/ ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Sep 12 06:26:47 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id GAA03532 for matrix-sig-people; Tue, 12 Sep 1995 06:26:47 -0400 Received: from sunic.sunet.se (sunic.sunet.se [192.36.125.2]) by python.org (8.6.12/8.6.12) with ESMTP id GAA03528 for ; Tue, 12 Sep 1995 06:26:19 -0400 From: Michal.Spalinski@fuw.edu.pl Received: from cocos.fuw.edu.pl by sunic.sunet.se (8.6.8/2.03) id MAA28154; Tue, 12 Sep 1995 12:26:07 +0200 Received: from albert4.fuw.edu.pl by cocos.fuw.edu.pl (4.1/SMI-4.1) id AA17210; Tue, 12 Sep 95 12:25:44 +0200 Received: by albert4.fuw.edu.pl (4.1/SMI-4.1) id AA01195; Tue, 12 Sep 95 12:27:57 +0200 Date: Tue, 12 Sep 95 12:27:57 +0200 Message-Id: <9509121027.AA01195@albert4.fuw.edu.pl> To: graham@fishnet.net Cc: matrix-sig@python.org In-Reply-To: <199509120046.AAA31805@big.fishnet.net> (message from Graham Hughes on Tue, 12 Sep 1995 00:46:50 -0700) Subject: Re: [PYTHON MATRIX-SIG] Let's get going Sender: owner-matrix-sig@python.org Precedence: bulk If this is going to be useful to people like me who are very fond of Python and have concrete numerical work to do in the field of Physics or whatever, the `*`, `+` etc really should be reserved for matrix operations (not elementwise). This is because one would like to be able to type a complicated formula and have it parsed correctly, and at the same time one should be able to look at it a week later and see immediately whether a sign is wrong or a factor is missing. This is the problem with using lisp for this kind of thing - if the notation is different from that which one is used to (by writing on paper) it makes it laborious to check whether the formulae are really correct. Regarding the APL type of stuff, outer products and so on, I think it is worth having and potentially useful, but it should use a different syntax. Maybe new infix operators, or just function calls - but not `+`, `*` etc. -- Michal S. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Sep 12 09:33:32 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA04037 for matrix-sig-people; Tue, 12 Sep 1995 09:33:32 -0400 Received: from rnet.rose.rsoc.rockwell.com (rose.rsoc.rockwell.com [161.40.39.100]) by python.org (8.6.12/8.6.12) with SMTP id JAA04033 for ; Tue, 12 Sep 1995 09:33:29 -0400 Received: by rnet.rose.rsoc.rockwell.com (4.1/SMI-4.1) id AA18295; Tue, 12 Sep 95 08:34:43 CDT Received: from sunrise(161.40.50.11) by rnet via smap (V1.3) id sma018293; Tue Sep 12 08:34:41 1995 Received: from feynman.rsoc.rockwell.com by rose.rsoc.rockwell.com (5.0/SMI-SVR4) id AA29273; Tue, 12 Sep 1995 08:34:40 -0500 Received: by feynman.rsoc.rockwell.com (5.0/SMI-SVR4) id AA24449; Tue, 12 Sep 1995 08:34:38 -0500 Date: Tue, 12 Sep 1995 08:34:38 -0500 From: forrest@rose.rsoc.rockwell.com (Dave Forrest) Message-Id: <9509121334.AA24449@feynman.rsoc.rockwell.com> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] * and / operators (was Re: Let's get going) X-Sun-Charset: US-ASCII Content-Length: 1422 Sender: owner-matrix-sig@python.org Precedence: bulk [Heartwarming personal intro - skip if you're not interested] Hi, I'm Dave Forrest with Rockwell Space Ops. Co. A few of you have met my colleagues Robin Friedrich, Greg Boes, and Charlie Fly. We are working on ground-based simulation, monitoring, and analysis software for the Space Shuttle. So, we have mostly an engineering/physics/math bias in our use of Python - although we're watching the python GUI work with keen interest. We love this language! [Real comments - start reading here] First off I'd like to add another vote for * being a matrix multiply, rather than elementwise. As for /, however, that doesn't really have meaning for matrices. When we built a C++ Matrix class, we only defined / for element-wise division by a scalar. It works quite nicely. We end up with something like this: Matrix operator/(Matrix, scalar) Matrix operator*(Matrix, Matrix) Matrix operator*(Matrix, scalar) <- elementwise muliplication by scalar Note that this avoids the sticky situation with inversion of matrices - we found it better to leave that for derived classes so that people could implement different inversion routines that capitalize on any a_priori knowledge of what's in the Matrix. e.g. class SquareMatrix( Matrix ): def inv(): ... That being said, I would like to see elementwise multiplication, etc. in there somewhere - but we use them far less frequently than real matrix operations. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Sep 12 14:08:12 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA05437 for matrix-sig-people; Tue, 12 Sep 1995 14:08:12 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id OAA05433 for ; Tue, 12 Sep 1995 14:08:06 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id OAA19276 (8.6.11/IDA-1.6 for ); Tue, 12 Sep 1995 14:08:19 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA21978; Tue, 12 Sep 1995 14:08:16 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA03216; Tue, 12 Sep 1995 14:08:18 -0400 Date: Tue, 12 Sep 1995 14:08:18 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509121808.OAA03216@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Python, matrices, and everything Sender: owner-matrix-sig@python.org Precedence: bulk [Yet another introduction - skip the next paragraph if you want to read something about matrices...] My name is Konrad Hinsen (*not* Hinsen Konrad, as given in my address), and I am a computational physicist, currently working as a postdoc at the University of Montreal. My main field of work is computer simulations of various kinds, and I use Python for everything that it not time-critical. My opinion about Python is a mixture of admiration for its elegant and efficient structure and frustration about its lack of numerical features, which is why I joined the Matrix-SIG. [Here comes the real stuff.] Before jumping into technical details, it might be a good idea to define what we are aiming at. As it has already become clear from the dicussion, matrices are used in two different (but not entirely distinct) ways: 1) in linear algebra applications 2) as containers of values that are structured as tables In linear algebra, one needs mainly one- and two-dimensional matrices of real or complex numbers. Important operations are addition and multiplication, factorization, inversion, eigenvalues etc. There are efficient Fortran and C libraries for these operations, so what is needed is a suitable representation in Python that allows interfacing to these libraries and some convenient way of accessing them, i.e. operators (+ - *, maybe / for scalar division) and tons of operations that are best packaged into a matrix class. The second application is just as important (in my view) and much more complex. As APLers know, there are lots of useful operations on tables of values, which can be of any dimension and contain any data type, although the numerical are the most important ones (Python already provides good string handling). It makes sense to have all mathematical functions and operators acting by default elementwise, including of course multiplication, but also user-defined functions. Basically arrays would be extensions of numerical scalars, the latter becoming arrays of rank zero. The by far most elegant and powerful array concept I know of is the one implemented in J, a relatively new language that can be regarded as an improved version of APL. I certainly don't want to take over J's other features, but its concept of array and operator ranks is extremely useful. I'll give a short description below. But first let's see how the two applications can be reconciled in a single implementation. Many of the linear algebra operations make sense only for two-dimensional matrices, but the J approach of operator rank takes care of the nicely. And even without that approach, this would only mean some more error checking for such operations. The only point of conflict is the implementation of multiplication (there is no difference for addition and subtraction). Linear-algebra style matrix multiplication is regarded as an inner product with addition and multiplication operations from an APL/J point of view, so it is available, but arguably not in the most convenient form. On the other hand, once the * symbol means matrix multiplication, the total general structure of elementwise operations is ruined. I therefore propose to introduce some other symbol for matrix multiplication. One could choose some combination like ".*" or "*.", which looks similar enough to plain multiplication to make its meaning evident. And now I'll shortly describe J's rank system. Every object's rank is identical to the number of its dimensions - a scalar has rank 0, a vector rank 1, and so on. When used in operations however, a rank N array is viewed as a rank A array of rank B cells, where A+B=N. A rank 3 array, for example, can be used as a three-dimensional array of scalars, as a two-dimensional matrix of vectors, as a vector of two-dimensional matrices, or as a "scalar" containing a three-dimensional array. How it is used depends on the rank of the operation acting on it. Most scalar operations have infinite rank, which means that use their operators at the highest rank possible. Effectively that means elementwise operation in the classical sense, except that there are also rules that cover the combination of object of different rank, such as multiplication of an array by a scalar. But there are also other operations. Matrix inversion, for example, by default operates on rank 2 cells. So if you "invert" a three-dimensional array, this operation will be interpreted as inversion of each rank-2 cell in the array. The result is again a rank 3 array. The beauty of the system is that the rank of each operation can be changed at will, not only for the built-in operations, but also for user-defined functions. If, for example, you want to add a rank 2 matrix to each rank 2 cell of a rank 3 array, you just specify rank 2 addition and get what you want. Nothing could be simpler or more flexible. Unfortunately, I dislike many of J's other features as a programming language, which is why I am still a Python user... So that's my point of view in this discussion: let's concentrate on APLish (or Jish) arrays and then add the operations that are needed in linear algebra. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Sep 12 15:33:33 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA05906 for matrix-sig-people; Tue, 12 Sep 1995 15:33:33 -0400 Received: from servrcolkr.cr.usgs.gov (servrcolkr.cr.usgs.gov [136.177.112.5]) by python.org (8.6.12/8.6.12) with ESMTP id PAA05902 for ; Tue, 12 Sep 1995 15:33:28 -0400 Received: from localhost ([130.11.51.73]) by servrcolkr.cr.usgs.gov (EMAIL 1.2.1) with ESMTP id TAA12230; Tue, 12 Sep 1995 19:22:12 GMT Message-Id: <199509121922.TAA12230@servrcolkr.cr.usgs.gov> To: Graham Hughes cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Let's get going In-reply-to: <199509120046.AAA31805@big.fishnet.net> Date: Tue, 12 Sep 1995 15:21:49 -0400 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk On Tue, 12 Sep 1995 00:46:50 -0700 Graham Hughes said: > At 07:04 PM 9/11/95 -0400, you wrote: > > o I wanted a python built-in data type containing a data structure > > that I could pass directly to Fortran or C routines. I did not > > want to have to do alot of data conversion or copying on each > > call. For example, if a Fortran routine returns a matrix, then > > it should be possible to pass this matrix to another Fortran > > routine without reallocating a data structure or converting the > > entire matrix. > > Instant problem right there. To the best of my knowledge (and I've read this > in several books), Fortran and C use completely different ways of storing > multidimensional arrays. To wit: given the array m, like this: > > 3 4 5 6 > > 7 8 9 0 > > 3 -2 4 1 > > There are two ways of storing this array in a continuous memory location; by > rows or by columns. By rows looks like this: '3 4 5 6 7 8 9 0 3 -2 4 1', and > is used by both C and Pascal. By columns looks like this: '3 7 3 4 8 -2 5 9 > 4 6 0 1', and is used by Fortran. In other words, to pass to either Fortran > or C functions you're going to need to know which type of function you have, > and then convert the internal representation. > > This can be done, but it greatly complicates things. Not really. Fortran stores data in multidimensional arrays with the indexes changing most rapidly in the left. Some people give this the interpretation that Fortran stores data "by column". C multi-dimensional arrays are stored with the indexes changing most rapidly on the right. Some people give this the interpretation that C stores data "by row". The difference is mainly one of interpretation. Both C and Fortran store mult-dimensional data pretty much the same way, as a big homogenous "rectangulish" glob of data. They differ only in the way they order the indexes to this data, and this ordering is only important for a certain set of interpretations. I choose to interpret C and Python multidimensional indexes, especially in the context of matrices, in a way in which this difference in indexing does not present a problem. I think of n-dimensional matrices as sequences of n-1-dimensional matrices. That is, matrices are stored, "by sub-matrix". Now, in the case of 2-d matrices, I was taught to think of a matrix as a collection of column vectors. With this interpretation, C and python 2-d matrices are, indeed, stored by column. Consider m[i][j]. Since 2-d matrices are a collection of column vectors, m[i] is the ith column. That is, we give the column index *first*, and then the row index. With this interpretation, there is no difference between C and Fortran storage formats. (snip) > > >> **Instantiation** > >> > >> The following function will be used to instantiate a matrix: > >> Matrix(datasource(optional), d1, d2, ..., typecode='d') > >> > >> The typecodes are as follows: > >> c - char (non-numeric) > >> 1 - unsigned char > >> b - signed char > >> h - short > >> i - int > >> l - long > >> f - float > >> d - double > >> F - complex float > >> D - complex double > > I'm not fond of this, as it breaks normal Python typing; let the matrix > figure out what's there. Matter of fact, might be a good idea to make the > matrix a general container like lists are now, so you can store, say, a > tuple representing a complex number in there? As it stands, with these > restrictions you can forget it. However, this makes it slightly more work > for those wonderful Fortran library routines. But the data must be stored in a homogenous format that can be passed to C and Fotran libraries. Also, libraries often require specific types, so you want control over the types used. I suppose that the routines could be made smart enough to sniff at the first element and check for numeric, complex (objects with re and im attributes), or string data and pick the default type accordingly. Note that the routines are already smart enough to convert arbitrary compatible objects to the necessary format. (snip) > >> concat(l) - l must be a list of matrices, and each matrix in l must have > >> the same number of dimensions, as well as the same size for every > >> dimension except for the first. This will return a new matrix M whose > >> first dimension is the sum of the first dimensions of all the matrices in > >> the list, and whose data is filled from the data in all of these matrices. > >> > >> ie. concat([A,B]) -> [1,2,3,11,12,13] > > Hm. Seems to run into difficulty with >1D matricies. Ideally, concat should take > > 2 3 4 5 6 7 > and > 5 6 7 8 9 10 > > and return > > 2 3 4 5 6 7 > > 5 6 7 8 9 10 > > But the given behaviour of concat depends greatly on whether [[x],[y]] is > stored by rows or by columns, ie. is it like this: > > [ | , | ] > V V > [x] [y] > > or like this: > > [ > -> [x] > , > -> [y] > ] > > This is a conceptual difficulty. No, it is not a problem as long as you use the interpretation that n-dimensional matrices are sequences of n-1-dimensional matrices. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Sep 12 15:56:29 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA06051 for matrix-sig-people; Tue, 12 Sep 1995 15:56:29 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id PAA06047 for ; Tue, 12 Sep 1995 15:56:26 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id PAA22875 (8.6.11/IDA-1.6 for ); Tue, 12 Sep 1995 15:56:30 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA00846; Tue, 12 Sep 1995 15:56:28 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA08479; Tue, 12 Sep 1995 15:56:30 -0400 Date: Tue, 12 Sep 1995 15:56:30 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509121956.PAA08479@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org In-reply-to: <199509121922.TAA12230@servrcolkr.cr.usgs.gov> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] Let's get going Sender: owner-matrix-sig@python.org Precedence: bulk Consider m[i][j]. Since 2-d matrices are a collection of column vectors, m[i] is the ith column. That is, we give the column index Of course you could also say that 2-d matrices are a collection of row vectors, which then give you the other interpretation. Anyway, I agree that this is just an interpretation problem and not serious, although users will have to be aware of how Python matrices correspond to Fortran/C matrices. I suppose that the routines could be made smart enough to sniff at the first element and check for numeric, complex (objects with re and im attributes), or string data and pick the default type accordingly. Speaking of complex numbers: has it ever been considered to make them built-in objects in Python? This would simplify the matrix implementation, and also make the math functions behave more reasonably (i.e. sqrt(-1.) would return i instead of causing an exception). It can't be much work and would be very useful for numerical work. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Sep 12 17:04:53 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA06311 for matrix-sig-people; Tue, 12 Sep 1995 17:04:53 -0400 Received: from retro.jhuapl.edu (retro.jhuapl.edu [128.244.146.239]) by python.org (8.6.12/8.6.12) with ESMTP id RAA06307 for ; Tue, 12 Sep 1995 17:04:50 -0400 Message-Id: <199509122104.RAA06307@python.org> Received: by retro.jhuapl.edu (1.37.109.16/16.2) id AA140499955; Tue, 12 Sep 1995 17:05:55 -0400 Date: Tue, 12 Sep 1995 17:05:55 -0400 From: Chris Chase S1A To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Vector-style operations vs. matrix operations In-Reply-To: <9509121027.AA01195@albert4.fuw.edu.pl> References: <199509120046.AAA31805@big.fishnet.net> <9509121027.AA01195@albert4.fuw.edu.pl> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk Hello, This is a long post. Self-introduction: I am an electrical engineer PhD working at the Applied Physics Lab of Johns Hopkins University. I do a lot of remote sensor data analysis requiring alot of rapidly developed numerical and visualization software. I would like to express some of my opinions regarding the elementwise versus matrix operators discussion. Elementwise operations for the proposed matrix objects fall under the category of vector operations in many math libriaries. When the matrix object is allocated in a single contiguous block of memory as in C or Fortran (and has been proposed in this forum) vector operations view the matrix as a one dimensional vector. These types of libraries support as a minimum '+','-','*' (and maybe '/') in an elementwise fashion. Each is invaluable in code that wishes to avoid for loops. Many popular interactive array oriented languages support vector operations. Below I list how a few of these languages implement vector and matrix operations. In Matlab, vector operations (i.e. elementwise) are called array operations and use the operators "+", "-", ".*" "./" and ".^". "*", "/", and "^" are used for matrix multiplication, right inverse, and matrix powers, respectively. In IDL (which I use more frequently than Matlab), "+", "-", "*", and "/" are all vector operators. "#" is matrix multiplication (actually "#" is used for Fortran-style arrays and "##" is used for C style arrays which can be very confusing). There is no operator performing matrix division. In Mathematica, elementwise multiplication is a space or "*" and matrix multiplication is given by the "." operator. There is not a matrix division operator. In APL (http://www.acm.org/sigapl/) "+", "-", "*", and "/" are vector-style operations. They are even more general in that they can be applied to specific ranks (as posted by Hinsen Konrad). Special operators are used for matrix inversion. Matrix multiplication is expressed using a combination of operators with the inner product operator (specifically, "+.x"). As much as I like the generality and completeness of the functional-style operators in APL and its descendant J, I think that the operator explosion makes the code extremely unreadable for the uninitiated. The following are all the same in the use of vector and matrix operations to Matlab and may be of interest for their other features regarding arrays: scilab http://www.inria.fr/Logiciels/SCILAB-eng.html octave http://www.che.utexas.edu/octave.html (maintained by the FSF - the GNU people that produce gcc, g++, emacs, etc.) tela http://www.geo.fmi.fi/prog/tela.html rlab ftp://evans.ee.adfa.oz.au/pub/RLaB/ or ftp://csi.jpl.nasa.gov/pub/matlab/RLaB/ (also GNU). tela is tensor oriented (allowing >2 dimensional arrays), so it uses generalizations of matrix multiplication and inner products and is a nice improvenment on that point over Matlab. ----- My preferences: In short, I agree with the proposal by Hinsen Konrad , i.e., "+", "-", "*", and "/" as vector style operations but applied rank-wise. I prefer to have arbitrary dimensioned arrays [i.e., tensors or tables as hinsenk@ere.umontreal.ca (Hinsen Konrad) refered to them] rather than only one dimensional vectors or two dimensional matrices. I prefer that "+", "-", "*", "/" all perform vector style operations. This makes their behavior consistent among each other. (The posted comment about "*" not applying to complex valued arrays was incorrect.) In most of my scientific programming, I encounter these vector style operators much more often than matrix multiplication and division. They replace the many "for" loops that are so common in numerical algorithms. They are displayed compactly and execute much faster than the loops in an interpreted language. When using higher dimensional arrays, matrix multiplication is rather specialized even when generalized to tensors. Matrix multiplication is actually a specialization of a general inner product (e.g., the implementation in APL). As such, if matrix multiplication had to have an operator I would choose a new one. Or, we would be better served by a generalized inner product operator. But if we don't want to add new operators I would let "*" be the more common vector (elementwise) operator. The Matlab approach is good, but it requires additional operators like ".*" and "./". Matlab also suffers from limitations to two dimensions (this is the main motivation behind the development of Tela). "/" does not generalize easily to the higher dimensions and the generalization of "*" is useful but limited when applied (as in Tela) only along the inner dimensions of two tensors. At the higher dimensions the elementwise operators applied at specific ranks are much more common and usefule then a generalized matrix multiplication operator. If it is desirable to avoid the proliferation of operators as in APL and J then matrix multiplication and division would be best left as function calls. There are additional reasons to advise against a "/" operator for matrix division. First there is the issue of whether a left or right inverse is desired (this is handled in Matlab by using two different operators "/" and "\"). Second, many matrix inversion problems are best solved using decomposition algorithms, such as LU decomposition or SVD, that use multiple steps wherein the intermediate results of the decomposition may saved for solving the same problem (e.g. with many different parameter matrices.) Additionally, the user often would like to pick which type of inversion algorithm to use and may want to override default parameters for these algorithms. I would prefer that matrix inversion be left to a group of explicit function calls. I suppose what is implemented for an array extension to Python depends on the end goal: 1) a Matlab-like array extension for Python is oriented toward 2-dimensional matrix linear algebra. Although limited in scope, this is useful and in particular allows very compact readable notation for linear algebra equations. However, it also tends to be stuck on a row/column view of arrays. 2) an array extension to handle higher dimension arrays (call them tensors or tables) that can easily handle generalized rank arithmetic (APL or J style as briefly set forth in Hinsen Konrad's post) and with specialized functions for matrix multiplication and division. Generalized inner and outer products (taking user specified binary operators) would also be very helpful. This view is more compatible with Jim Fulton's "n-dimensional matrices are sequences of n-1-dimensional matrices." In either case, I think it would be a mistake to add too many new operators in the APL or J fashion. Such clutter would reduce the readability of Python programs. One also runs the risk of trying to make a subset of Python look like a functional programming language which could further confuse users. Sincerely, Chris Chase =============================== Bldg 24-E188 The Applied Physics Laboratory The Johns Hopkins University Laurel, MD 20723-6099 (301)953-6000 x8529 chris.chase@jhuapl.edu ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Sep 12 17:28:06 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA06443 for matrix-sig-people; Tue, 12 Sep 1995 17:28:06 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id RAA06439 for ; Tue, 12 Sep 1995 17:28:03 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id RAA25669 (8.6.11/IDA-1.6 for ); Tue, 12 Sep 1995 17:28:16 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA23125; Tue, 12 Sep 1995 17:28:14 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA12993; Tue, 12 Sep 1995 17:28:16 -0400 Date: Tue, 12 Sep 1995 17:28:16 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509122128.RAA12993@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org In-reply-to: <199509122104.RAA06307@python.org> (message from Chris Chase S1A on Tue, 12 Sep 1995 17:05:55 -0400) Subject: Re: [PYTHON MATRIX-SIG] Vector-style operations vs. matrix operations Sender: owner-matrix-sig@python.org Precedence: bulk In either case, I think it would be a mistake to add too many new operators in the APL or J fashion. Such clutter would reduce the readability of Python programs. One also runs the risk of trying to make a subset of Python look like a functional programming language which could further confuse users. Let me add that I totally agree about that. There is no point in re-creating APL or J, especially since Python is already a better language in many respects (especially when it comes to legibility). All I would like to take over from J is its array and rank concept. In terms of array-specific operations, I suggest inner and outer product, transpose (in the general sense), and reduction (I hope I haven't forgotten anything). These would be implemented as methods, not as special operators as in APL or J. Together with the elementwise operations this would give an array system superior to most languages (except for APL/J of course) and extremely useful both for linear algebra and all kinds of data handling. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Sep 12 21:23:03 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id VAA07211 for matrix-sig-people; Tue, 12 Sep 1995 21:23:03 -0400 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id VAA07207 for ; Tue, 12 Sep 1995 21:23:01 -0400 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa21291; 12 Sep 95 21:22 EDT Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id VAA23857; Tue, 12 Sep 1995 21:22:17 -0400 Message-Id: <199509130122.VAA23857@monty> To: Hinsen Konrad cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Let's get going In-reply-to: Your message of "Tue, 12 Sep 1995 15:56:30 EDT." <199509121956.PAA08479@cyclone.ERE.UMontreal.CA> References: <199509121956.PAA08479@cyclone.ERE.UMontreal.CA> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Tue, 12 Sep 1995 21:22:17 -0400 Sender: owner-matrix-sig@python.org Precedence: bulk > Speaking of complex numbers: has it ever been considered to make > them built-in objects in Python? This would simplify the matrix > implementation, and also make the math functions behave more > reasonably (i.e. sqrt(-1.) would return i instead of causing > an exception). It can't be much work and would be very useful > for numerical work. Hm. I think it *would* be much work, since the standard C math library doesn't have any functions working on complex numbers. I have a specific objection against making sqrt(-1) returning a complex number. In Python, as a rule, the *type* of a function's return value may depend on the *type* of its argument(s), but not in their *value*. Even though there is nothing (or at least very little) in the language's implementation that enforces this, it is a consistent convention throughout the language. It even has one important visible consequence: 1/3 is calculated as truncating, as in C (though it truncates towards -infinity rather than towards 0), rather than returning a floating point value if the result cannot be represented as an integer. I believe this is an important property -- it makes reasoning about the types of variables/arguments/functions in a Python program easier (and sooner or later, this reasoning is going to be done by programs as well as by people). --Guido van Rossum URL: (I have more to say about the matrix discussion but I need some more time to digest everything that's been said.) ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 00:17:59 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id AAA07616 for matrix-sig-people; Wed, 13 Sep 1995 00:17:59 -0400 Received: from big.fishnet.net (root@big.fishnet.net [205.216.133.3]) by python.org (8.6.12/8.6.12) with ESMTP id AAA07612 for ; Wed, 13 Sep 1995 00:17:55 -0400 Received: from port26.fishnet.net (port26.fishnet.net [205.216.133.226]) by big.fishnet.net (8.6.12/8.6.9) with SMTP id VAA22737; Tue, 12 Sep 1995 21:20:12 GMT Message-Id: <199509122120.VAA22737@big.fishnet.net> X-Sender: graham@fishnet.net X-Mailer: Windows Eudora Pro Version 2.1.2 Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Tue, 12 Sep 1995 21:21:25 -0700 To: Jonne Itkonen From: Graham Hughes Subject: Re: [PYTHON MATRIX-SIG] Let's get going Cc: matrix-sig@python.org Sender: owner-matrix-sig@python.org Precedence: bulk At 12:17 PM 9/12/95 +0300, you wrote: >I prefer the mathematical, not elementwise multiplication. We are handling >matrixes here (not tables!). Besides, one doesn't want to define the * >operator for complex numbers to do 'elementwise' multiplication, does she? >It would be meaningless! Ah, but would it be? Part of the reason why I wanted to do elementwise multiplication and division is that matrix multiplication is nearly trivial with inner and outer products. E.g.; you want your `normal' complex multiplication? Try outer_product(lambda x,y: x+y,lambda x,y: x*y, v1, v2) which can easily be all a member function calls. Similarly, (BTW, this works with APL's data parallelism; depending on how we implement outer_product it may be a bit nastier), matrix multiplication is this: outer_product(lambda x,y: x+y, lambda x,y: x*y, m1, m2.transpose()) (Note the transposition.) Finally, we reach the question of efficiency. No matter how you implement it, matrix multiplication will be slow(er). Divison, besides which it won't always work, will take roughly forever; APL is quick on this note, but that's because to do matrix multiplication you use a special purpose function. This, and the odd dimension requirements for matrix multiplication, are why I'm not fond of having the `mathematical' methods be the default. Besides, if we do it that way, we can treat one-dimensional vectors consisting of only one element (i.e. [3] or some such) essentially as scalars, which is a parallelism I like; whereas matrix multiplication and division will take 10 times as long to get the same answer. > >In my opinion, the overloading of operators is to make things more clear >than to mess them up. The elementwise multiplication by * operator would >be clear to anyone with no knowledge of matrices, but not to someone who >knows what matrices are. Correct me if I'm wrong, but doesn't matrix multiplication use a special syntax all its own? All the matricies are either boxed or in boldface, some times they even use 'X' instead of a dot, all to make absolutely sure that the reader gets the point. Great efforts are taken to make sure that anyone reading knows that the multiplication does weird stuff here (because weird stuff happens; where else do you take two things of unequal size and come up with something else with a possibly completely different size?) I understand the engineering tendency to think in terms of matrix multiplication, but there is a precedent for divorcing it from the matrix addition; the two are radically different operations. > >Perhaps a table object should be developed for elementwise calculation? Probably not; too much common code to make it worthwhile. Besides, since Python is typeless, we *cannot* distinguish between multiplication by a scalar or by a matrix. There is no typeof() function to distinguish them. To avoid bizarrities concerning *scalar* multiplication, we must use elementwise multiplication. If this were C++, we could use two seperate operator *()s, but it's not. And we're pleased about that most of the time :) >That is not a good reason. It's more confusing to use mathematical >definition for some, and non-mathematical definitions for other operators, >as in your '+ vs. *' example. To you. To the person coming from APL, J or some other similar languages, it's the other way around. Plus it raises problems dealing with scalar multiplication. Graham Graham Hughes Home page http://www.fishnet.net/~graham/ ``I think it would be a good idea.'' -- Mahatma Ghandi, when asked what he thought of Western civilization finger for PGP public key. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 09:20:16 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA08901 for matrix-sig-people; Wed, 13 Sep 1995 09:20:16 -0400 Received: from servrcolkr.cr.usgs.gov (servrcolkr.cr.usgs.gov [136.177.112.5]) by python.org (8.6.12/8.6.12) with ESMTP id JAA08897 for ; Wed, 13 Sep 1995 09:20:11 -0400 Received: from localhost (dsjfqvarsa.er.usgs.gov [130.11.51.73]) by servrcolkr.cr.usgs.gov (EMAIL 1.2.1) with ESMTP id NAA07565; Wed, 13 Sep 1995 13:21:01 GMT Message-Id: <199509131321.NAA07565@servrcolkr.cr.usgs.gov> To: Graham Hughes cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Let's get going In-reply-to: <199509122120.VAA22737@big.fishnet.net> Date: Wed, 13 Sep 1995 09:21:09 -0400 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk These are good discussions. I'm going to have to find some time to digest what's been said. Here are a few quick comments that don't require much thought. ;-) On Tue, 12 Sep 1995 21:21:25 -0700 Graham Hughes said: > At 12:17 PM 9/12/95 +0300, you wrote: (snip) > Finally, we reach the question of efficiency. No matter how you implement > it, matrix multiplication will be slow(er). Divison, besides which it won't > always work, A number of people (not to pick on you) have made the agument that "such and such shouldn't be done because it won't always work". I really don't think this is a valid argument. Most, if not all, of the operations and functions have some preconditions. You can't do elementwise arithmetic unless both operands have the same dimensions. You can't do elementwise division if *any* elements of the divisor are zero. Heck, scalar division fails if a divisor is zero, but Guido was still kind enough to include "/" in teh language. (Whew. :-) > Probably not; too much common code to make it worthwhile. Besides, since > Python is typeless, Python is not typeless, only it's variables are. > we *cannot* distinguish between multiplication by a > scalar or by a matrix. Not necessarily by reading the code. > There is no typeof() function to distinguish them. No, it's called type(). There's also hasattr, which can be used to check basic object signatures. > To > avoid bizarrities concerning *scalar* multiplication, we must use > elementwise multiplication. If this were C++, we could use two seperate > operator *()s, but it's not. > It is easy enough to handle this with a check in the single * operator. In fact, I think that whathever we do, numeric operations involving matrices should work with: o Matrices and matrices, o Matrices and scalars, and o Matrices and sequences of sequences (non-Matrix objects that support similar multi-dimensional access). Supporting this does not really depend on whether a element-wise interpretation is used for *. Note that applying the mapping operator to take matrix slices will *not* return matrix objects, since matrix objects are supposed to contain contiguous globs of memory, so that their data can be passed to C and Fortran libraries directly. These matrix slices will reference matrix objects so that modification of slice elements will be reflected in the matrices from which they are derived, but for mathematical operations, they will probably use different code. My suspicion is that there will be code that: o Operates on two matrices (and perhaps a matrix and a scalar), using knowledge of internal data structures to optimize performance, and separate code that o Operates on sequences of sequences. These will be written in C, for speed, but will use PySequence_GetItem calls to access operand data. > > >That is not a good reason. It's more confusing to use mathematical > >definition for some, and non-mathematical definitions for other operators, > >as in your '+ vs. *' example. Actually, the "mathematical" interpretation of + happens to be the same as the elementwise interpretation. Still, most of the people who call for a mathematical interpretation of * seem only to want that operator to be non-elementwise. This seems to me to stengthen the case of those who want an elementwise interpretation for all operations. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 09:45:11 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA09047 for matrix-sig-people; Wed, 13 Sep 1995 09:45:11 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id JAA09043 for ; Wed, 13 Sep 1995 09:45:08 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id JAA08870 (8.6.11/IDA-1.6); Wed, 13 Sep 1995 09:44:59 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA06209; Wed, 13 Sep 1995 09:44:57 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA26202; Wed, 13 Sep 1995 09:45:03 -0400 Date: Wed, 13 Sep 1995 09:45:03 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509131345.JAA26202@cyclone.ERE.UMontreal.CA> To: guido@CNRI.Reston.VA.US CC: matrix-sig@python.org In-reply-to: <199509130122.VAA23857@monty> (message from Guido van Rossum on Tue, 12 Sep 1995 21:22:17 -0400) Subject: Re: [PYTHON MATRIX-SIG] Let's get going Sender: owner-matrix-sig@python.org Precedence: bulk Hm. I think it *would* be much work, since the standard C math library doesn't have any functions working on complex numbers. True, but complex libraries are easily available (e.g. GNU-C comes with one). I have a specific objection against making sqrt(-1) returning a complex number. In Python, as a rule, the *type* of a function's return value may depend on the *type* of its argument(s), but not in their *value*. Even though there is nothing (or at least very little) There is no need to make real and complex numbers different types. A real number would just be a complex number with an imaginary part of zero. Of course internally there should be a difference for efficiency reasons, but the user need not see it. Contrary to integers, reals numbers need not have a special type since any operation on complex numbers with zero imaginary part is exactly equivalent to an operation on real numbers (for integers there is a difference in division). consistent convention throughout the language. It even has one important visible consequence: 1/3 is calculated as truncating, as in C (though it truncates towards -infinity rather than towards 0), rather than returning a floating point value if the result cannot be represented as an integer. And I am sure this feature has already led to some surprises for people who thought they could use Python like a pocket calculator. For languages like Python, where numerical efficiency does not have top priority, I'd consider it more useful to have a number model that reflects mathematical categories instead of internal storage format categories. There should be only one type "number" with representations for integers, fractions, floating point numbers, and complex numbers, the latter having real and imaginary parts that can be of any of the basic representations. Then 1/3 would return a fraction. This is what anyone without programming experience would expect. Of course I realize that it is much too late to introduce such a change... I believe this is an important property -- it makes reasoning about the types of variables/arguments/functions in a Python program easier (and sooner or later, this reasoning is going to be done by programs as well as by people). I certainly agree on that. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 11:57:55 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA09738 for matrix-sig-people; Wed, 13 Sep 1995 11:57:55 -0400 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id LAA09734 for ; Wed, 13 Sep 1995 11:57:52 -0400 Received: from nineveh.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA00677; Wed, 13 Sep 95 11:58:13 EDT Received: by nineveh.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA04152; Wed, 13 Sep 95 12:04:44 -0400 Message-Id: <9509131604.AA04152@nineveh.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Wed, 13 Sep 95 12:04:44 -0400 To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Vector-style operations vs. matrix operations References: <199509120046.AAA31805@big.fishnet.net> <9509121027.AA01195@albert4.fuw.edu.pl> <199509122104.RAA06307@python.org> Sender: owner-matrix-sig@python.org Precedence: bulk Short personal intro, feel free to skip: Hi, my name's Jim Hugunin and I'm a PhD student working in MIT's Laboratory for Computer Science. My current work involves designing speech recognition systems. My Master's thesis involved the fabrication and computer modeling of superconducting transistors (using the Bogoliubov deGennes equations if anyone cares) so I'm reasonably familiar with numerical modelling in both the EE and the physics communities. I'm the one who wrote the initial proposal for the matrix object that Jim Fulton posted to start this discussion, so you already know my opinions on most of these issues. I've also implemented a matrix object (on top of Jim Fulton's) which provides most of the capabilities described in the proposal. Real comments begin here: First, some posts seem to be placing too much stock in the name "matrix" object. I propose that the name should be changed to either table or tensor (as mentioned by Chris Chase) if this is necessary to quell the arguments that a "matrix" should behave like a mathematical "matrix". (Note: I have no objections to arguments in favor of matrix-multiplication for "*", I just don't think they should be based on something as arbitrary as the object's name). Next, I'd like to add my vote to those in favor of element-wise operations. This just seems to me to be the most general purpose sort of functionality that could be provided by an object that stores contiguous blocks of homogeneous data. I think that all of the same element-wise operations should be provided as are provided for the int and float objects (note this is a small change to my proposal). This makes a table object simply a way of storing a multi-dimensional block of homogeneous data-elements, that can be operated on in exactly the same way as the equivalent scalars in python with greatly improved performance. I also completely agree with Konrad Hinsen's suggestion that J's rank system be used for table objects. This is in fact a very minor change to my original proposal. J's rank system is simply a clearer way of expressing my dimensionality coercion ideas. Chric Chase suggests: Generalized inner and outer products (taking user specified binary operators) would also be very helpful. I'm curious to better understand this proposal. If it means defining an outer product that can take python binary functions as it's arguments, I would argue that it would be incredibly inefficient (compared to native C), and therefore might as well be implemented in python as a set of "helper" functions. If it means specifying only primitive binary arithmetic operations (like "+" and "*") then I think that it could be implemented efficiently, but I would really like to see some examples of where this would be useful (other than matrix multiplication which will probably be done with special purpose code for efficiency reasons anyway). All of the above comments also relate to proposals for specialized "map" and "reduce" functions for tables/matrices. Unless there is some large efficiency gain possible, I'd rather see these implemented in python than C. -Jim Hugunin hugunin@mit.edu ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 12:32:25 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA09881 for matrix-sig-people; Wed, 13 Sep 1995 12:32:25 -0400 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id MAA09877 for ; Wed, 13 Sep 1995 12:32:23 -0400 Received: from nineveh.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA00875; Wed, 13 Sep 95 12:32:42 EDT Received: by nineveh.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA04160; Wed, 13 Sep 95 12:39:14 -0400 Message-Id: <9509131639.AA04160@nineveh.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Wed, 13 Sep 95 12:39:13 -0400 To: "Jim Fulton, U.S. Geological Survey" Subject: Re: [PYTHON MATRIX-SIG] Let's get going Cc: matrix-sig@python.org References: <199509112304.XAA13015@servrcolkr.cr.usgs.gov> Sender: owner-matrix-sig@python.org Precedence: bulk In my previous post I comment on what appears to be the primary source of controversy for the matrix object, whether it should be treated as a table of numbers, or as a true mathematical matrix. I'm afraid that that issue is likely to be the source of substantial more discussion before anything is decided. I would also like to comment on some of the less controversial points raised by Jim Fulton in his first post regarding basic methods for matrix objects. >> transpose - The transpose of the matrix (not needed if "~" is used for >> transpose) > I'm thinking that this (or ~) should return by reference. I agree completely. In fact, when I went about implementing matrix slices (so that m[((1,3),)] can be properly returned by reference, I noticed that transposing a matrix was trivially implemented by converting it to a matrix slice. This of course raises the issue of how to properly deal with these matrix slices. For example, the way my code is written now, if I try to add a slice to a matrix, I can perform the operation quickly without needing to convert the slice to a matrix, as long as the slice meets a fairly obscure list of requirements. Basically these can be summarized as the fact that there must be a constant offset between the indices for each dimensions. ie. M[((1,3,5),)] would be easy to deal with (as fast as a matrix) whereas: M[((1,3,6),)] would not be What bothers me about this gets back to your earlier complaints regarding automatic coercion as being a possible source of errors (which I've come around to agreeing with in general). On the other hand, conceptually, it would be nice to be able to treat a slice exactly the same as a matrix whenever possible. I see three options, none of which seems perfect, but I am inclined towards the third. 1) Don't allow slices to be passed to functions which require a matrix argument, and don't allow numeric operations on slices 2) Treat a slice just like any other non-matrix sequence, and allow numeric operations, and allow it to be passed to matrix functions using the generic sequence to matrix functions (there is an interesting issue here having to do with copying data back from the matrix to the input sequence if it turns out that the function modifies it's input data. I'm curious as to how you deal with this, or whether you just don't allow this to happen). 3) Whenever a slice is passed to a function requiring a matrix argument, automatically copy the data from the slice to a contiguous block of memory. Allow slices to be used in numeric operations, doing this efficiently where possible, and copying the slice to a contiguous block of memeory where not possible. This would be the most efficient solution. >> byteswap - Useful for reading data from a file written on a machine with a >> different byte order > Hm. Does this operate in-place or return a value? In-place. In fact, I am tempted to try and make all of the methods on matrices operate "in-place" in order to be in line with list objects. >> typecast(t) - Returns a new matrix of the same dimensions where each item >> is cast (using C typecasting rules) from the matrix's type to the new type >> t. >Our constructor already gives you this: > new_matrix=Matrix(old_matrix,new_type) True. My problem is that I am frequently having to cast between types (I deal with samples of sound that usually have to come in and go out as shorts, but which I usually want to operate on as floats in the meantime). So, I agree that this should be done using the matrix constructor, and all that I'd add is that the matrix constructor should be written so that this typecasting is done efficiently. -Jim Hugunin hugunin@mit.edu ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 12:34:33 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA09881 for matrix-sig-people; Wed, 13 Sep 1995 12:32:25 -0400 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id MAA09877 for ; Wed, 13 Sep 1995 12:32:23 -0400 Received: from nineveh.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA00875; Wed, 13 Sep 95 12:32:42 EDT Received: by nineveh.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA04160; Wed, 13 Sep 95 12:39:14 -0400 Message-Id: <9509131639.AA04160@nineveh.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Wed, 13 Sep 95 12:39:13 -0400 To: "Jim Fulton, U.S. Geological Survey" Subject: Re: [PYTHON MATRIX-SIG] Let's get going Cc: matrix-sig@python.org References: <199509112304.XAA13015@servrcolkr.cr.usgs.gov> Sender: owner-matrix-sig@python.org Precedence: bulk In my previous post I comment on what appears to be the primary source of controversy for the matrix object, whether it should be treated as a table of numbers, or as a true mathematical matrix. I'm afraid that that issue is likely to be the source of substantial more discussion before anything is decided. I would also like to comment on some of the less controversial points raised by Jim Fulton in his first post regarding basic methods for matrix objects. >> transpose - The transpose of the matrix (not needed if "~" is used for >> transpose) > I'm thinking that this (or ~) should return by reference. I agree completely. In fact, when I went about implementing matrix slices (so that m[((1,3),)] can be properly returned by reference, I noticed that transposing a matrix was trivially implemented by converting it to a matrix slice. This of course raises the issue of how to properly deal with these matrix slices. For example, the way my code is written now, if I try to add a slice to a matrix, I can perform the operation quickly without needing to convert the slice to a matrix, as long as the slice meets a fairly obscure list of requirements. Basically these can be summarized as the fact that there must be a constant offset between the indices for each dimensions. ie. M[((1,3,5),)] would be easy to deal with (as fast as a matrix) whereas: M[((1,3,6),)] would not be What bothers me about this gets back to your earlier complaints regarding automatic coercion as being a possible source of errors (which I've come around to agreeing with in general). On the other hand, conceptually, it would be nice to be able to treat a slice exactly the same as a matrix whenever possible. I see three options, none of which seems perfect, but I am inclined towards the third. 1) Don't allow slices to be passed to functions which require a matrix argument, and don't allow numeric operations on slices 2) Treat a slice just like any other non-matrix sequence, and allow numeric operations, and allow it to be passed to matrix functions using the generic sequence to matrix functions (there is an interesting issue here having to do with copying data back from the matrix to the input sequence if it turns out that the function modifies it's input data. I'm curious as to how you deal with this, or whether you just don't allow this to happen). 3) Whenever a slice is passed to a function requiring a matrix argument, automatically copy the data from the slice to a contiguous block of memory. Allow slices to be used in numeric operations, doing this efficiently where possible, and copying the slice to a contiguous block of memeory where not possible. This would be the most efficient solution. >> byteswap - Useful for reading data from a file written on a machine with a >> different byte order > Hm. Does this operate in-place or return a value? In-place. In fact, I am tempted to try and make all of the methods on matrices operate "in-place" in order to be in line with list objects. >> typecast(t) - Returns a new matrix of the same dimensions where each item >> is cast (using C typecasting rules) from the matrix's type to the new type >> t. >Our constructor already gives you this: > new_matrix=Matrix(old_matrix,new_type) True. My problem is that I am frequently having to cast between types (I deal with samples of sound that usually have to come in and go out as shorts, but which I usually want to operate on as floats in the meantime). So, I agree that this should be done using the matrix constructor, and all that I'd add is that the matrix constructor should be written so that this typecasting is done efficiently. -Jim Hugunin hugunin@mit.edu ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 12:35:42 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA09902 for matrix-sig-people; Wed, 13 Sep 1995 12:35:42 -0400 Received: from rnet.rose.rsoc.rockwell.com (rose.rsoc.rockwell.com [161.40.39.100]) by python.org (8.6.12/8.6.12) with SMTP id MAA09898 for ; Wed, 13 Sep 1995 12:35:39 -0400 Received: by rnet.rose.rsoc.rockwell.com (4.1/SMI-4.1) id AA22321; Wed, 13 Sep 95 11:36:36 CDT Received: from sunrise(161.40.50.11) by rnet via smap (V1.3) id sma022319; Wed Sep 13 11:36:07 1995 Received: from feynman.rsoc.rockwell.com by rose.rsoc.rockwell.com (5.0/SMI-SVR4) id AA05890; Wed, 13 Sep 1995 11:36:05 -0500 Received: by feynman.rsoc.rockwell.com (5.0/SMI-SVR4) id AA00821; Wed, 13 Sep 1995 11:36:03 -0500 Date: Wed, 13 Sep 1995 11:36:03 -0500 From: forrest@rose.rsoc.rockwell.com (Dave Forrest) Message-Id: <9509131636.AA00821@feynman.rsoc.rockwell.com> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Are we talking about one thing or two? X-Sun-Charset: US-ASCII Content-Length: 1555 Sender: owner-matrix-sig@python.org Precedence: bulk When we started talking about the use of operator* being elementwise or "mathematical", I thought that was the extent of it - a disagreement over one operator. But now I think we may actually be asking for two completely different animals. Mr. Hugunin suggested that "matrix" is, in effect, just a name. Well, maybe not. For what we do here we have definite semantics assigned to the word "matrix" and they are the "mathematical" semantics described earlier. The "elementwise" stuff is more like what we would call an "array" or "table" - although "tensor" sounds good to me. So, I'd like to suggest that we pursue not one, but two new entities. A "matrix" would be limited to two dimensions and support the "mathematical" notion that most of us engineers had beat into our brains in college. A "tensor" or "table" or even "array" would fit the more general interpretation and have element-wise operations. Why? For practical reasons. We here - and I would submit that many other organizations are like this - do more Matlab-like work. The way Matlab defines a matrix is not a problem but a huge advantage for us. Several of the rest of you apparently deal with more abstract notions either as data storage elements, set theoretic elements, or mathematical elements. You need the elementwise stuff just as badly as we need our "mathematical" stuff. Rather than trying to make a one-size-fits all that isn't exactly what anyone wants, why not make two interfaces (maybe one implementation?) that give each camp what they want to work with? ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 13:44:53 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA10214 for matrix-sig-people; Wed, 13 Sep 1995 13:44:53 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id NAA10210 for ; Wed, 13 Sep 1995 13:44:48 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id NAA17425 (8.6.11/IDA-1.6); Wed, 13 Sep 1995 13:43:30 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA02220; Wed, 13 Sep 1995 13:43:29 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA09394; Wed, 13 Sep 1995 13:43:37 -0400 Date: Wed, 13 Sep 1995 13:43:37 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509131743.NAA09394@cyclone.ERE.UMontreal.CA> To: forrest@rose.rsoc.rockwell.com CC: matrix-sig@python.org In-reply-to: <9509131636.AA00821@feynman.rsoc.rockwell.com> (forrest@rose.rsoc.rockwell.com) Subject: Re: [PYTHON MATRIX-SIG] Are we talking about one thing or two? Sender: owner-matrix-sig@python.org Precedence: bulk When we started talking about the use of operator* being elementwise or "mathematical", I thought that was the extent of it - a disagreement over one operator. But now I think we may actually be asking for two completely different animals. Mr. Hugunin suggested that "matrix" is, in effect, just a name. Well, maybe not. For what we do here we have But the one operator is the only difference - apart from the interpretation of multiplication, "mathematical" matrices are just a special case of general arrays. "mathematical" semantics described earlier. The "elementwise" stuff is more like what we would call an "array" or "table" - although "tensor" sounds good to me. Not to me though. A tensor is a mathematical object representing a physical quantity in space that exists indepently of a choice of a coordinate system. A tensor is thus not just a table of numbers, but a set of numbers with a well-defined transformation property when changing from one coordinate system to another. I vote for "array" as the name for what we are trying to do. Why? For practical reasons. We here - and I would submit that many other organizations are like this - do more Matlab-like work. The way Matlab defines a matrix is not a problem but a huge advantage for us. But in what way would a more general concept be a disadvantage? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 14:07:42 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA10335 for matrix-sig-people; Wed, 13 Sep 1995 14:07:42 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id OAA10330 for ; Wed, 13 Sep 1995 14:07:37 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id OAA18220 (8.6.11/IDA-1.6); Wed, 13 Sep 1995 14:07:16 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA08197; Wed, 13 Sep 1995 14:07:14 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA10603; Wed, 13 Sep 1995 14:07:22 -0400 Date: Wed, 13 Sep 1995 14:07:22 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509131807.OAA10603@cyclone.ERE.UMontreal.CA> To: jjh@mama-bear.lcs.mit.edu CC: matrix-sig@python.org In-reply-to: <9509131639.AA04160@nineveh.LCS.MIT.EDU> (message from James Hugunin on Wed, 13 Sep 95 12:39:13 -0400) Subject: Re: [PYTHON MATRIX-SIG] Let's get going Sender: owner-matrix-sig@python.org Precedence: bulk 3) Whenever a slice is passed to a function requiring a matrix argument, automatically copy the data from the slice to a contiguous block of memory. Allow slices to be used in numeric operations, doing this efficiently where possible, and copying the slice to a contiguous block of memeory where not possible. This would be the most efficient solution. Why would it have to be copied whenever it is passed to a function? It would be sufficient to make a copy whenever the matrix is changed. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 14:23:38 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA10388 for matrix-sig-people; Wed, 13 Sep 1995 14:23:38 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id OAA10383 for ; Wed, 13 Sep 1995 14:23:26 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id OAA18709 (8.6.11/IDA-1.6); Wed, 13 Sep 1995 14:23:22 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA13737; Wed, 13 Sep 1995 14:23:21 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA11504; Wed, 13 Sep 1995 14:23:29 -0400 Date: Wed, 13 Sep 1995 14:23:29 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509131823.OAA11504@cyclone.ERE.UMontreal.CA> To: jjh@mama-bear.lcs.mit.edu CC: matrix-sig@python.org In-reply-to: <9509131604.AA04152@nineveh.LCS.MIT.EDU> (message from James Hugunin on Wed, 13 Sep 95 12:04:44 -0400) Subject: Re: [PYTHON MATRIX-SIG] Vector-style operations vs. matrix operations Sender: owner-matrix-sig@python.org Precedence: bulk Chric Chase suggests: Generalized inner and outer products (taking user specified binary operators) would also be very helpful. I'm curious to better understand this proposal. If it means defining an outer product that can take python binary functions as it's arguments, I would argue that it would be incredibly inefficient (compared to native C), and therefore might as well be implemented in python as a set of "helper" functions. If it means specifying only primitive binary arithmetic operations (like "+" and "*") then I think that it could be implemented efficiently, but I would really like to see some examples of where this would be useful (other than matrix multiplication which will probably be done with special purpose code for efficiency reasons anyway). Of course general inner and outer products, as well as reduction, should work with any binary operation, including user-defined functions. And I agree that this is best done in Python. However, the most important applications use only the built-in operations (arithmetic, comparison, logical), so it would make sense to provide a more efficient solution for them. There are lots of applications for these things, as any APLer will confirm. Unfortunately I can't include any APL idioms here due to character set restrictions :-( But let me give a simple example: suppose you have an array of data values x (one-dimensional). You want to make a cumulative histogram, i.e. an array of integers n in which each entry n[i] indicates the number of values in x that are larger than b[i], where b[i] is the value of each "bin" in the histogram. This is just an outer product using >, followed by a reduction with +: n = reduce(lambda a,b: a+b, 1, outer_product(lambda a,b: a>b, x, b)) in Pseudo-Python, where the second argument of "reduce" indicates the rank of the cells to be reduced. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 14:23:44 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA10394 for matrix-sig-people; Wed, 13 Sep 1995 14:23:44 -0400 Received: from servrcolkr.cr.usgs.gov (servrcolkr.cr.usgs.gov [136.177.112.5]) by python.org (8.6.12/8.6.12) with ESMTP id OAA10390 for ; Wed, 13 Sep 1995 14:23:40 -0400 Received: from localhost (dsjfqvarsa.er.usgs.gov [130.11.51.73]) by servrcolkr.cr.usgs.gov (EMAIL 1.2.1) with ESMTP id SAA17108; Wed, 13 Sep 1995 18:24:21 GMT Message-Id: <199509131824.SAA17108@servrcolkr.cr.usgs.gov> To: hinsenk@ere.umontreal.ca (Hinsen Konrad) cc: jjh@mama-bear.lcs.mit.edu cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Let's get going In-reply-to: <199509131807.OAA10603@cyclone.ERE.UMontreal.CA> Date: Wed, 13 Sep 1995 14:24:18 -0400 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk On Wed, 13 Sep 1995 14:07:22 -0400 Hinsen Konrad said: > > 3) Whenever a slice is passed to a function requiring a matrix argument, > automatically copy the data from the slice to a contiguous block of memory. > Allow slices to be used in numeric operations, doing this efficiently where > possible, and copying the slice to a contiguous block of memeory where not > possible. This would be the most efficient solution. > > Why would it have to be copied whenever it is passed to a function? > It would be sufficient to make a copy whenever the matrix is > changed. This gets back to my earlier comment that slices, as proposed cannot be matrices because they store a reference to the data from which they were created and a bunch of offsets for accessing the orinal data properly. When passed to math/sci libraries, they'll have to be converted to real matrices so that their data are contiguous. JIm ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 14:28:26 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA10439 for matrix-sig-people; Wed, 13 Sep 1995 14:28:26 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id OAA10435 for ; Wed, 13 Sep 1995 14:28:22 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id OAA18842 (8.6.11/IDA-1.6); Wed, 13 Sep 1995 14:28:19 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA14777; Wed, 13 Sep 1995 14:28:18 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA11695; Wed, 13 Sep 1995 14:28:26 -0400 Date: Wed, 13 Sep 1995 14:28:26 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509131828.OAA11695@cyclone.ERE.UMontreal.CA> To: jfulton@usgs.gov CC: matrix-sig@python.org In-reply-to: <199509131824.SAA17108@servrcolkr.cr.usgs.gov> (jfulton@wrdmail.er.usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] Let's get going Sender: owner-matrix-sig@python.org Precedence: bulk > Why would it have to be copied whenever it is passed to a function? > It would be sufficient to make a copy whenever the matrix is > changed. This gets back to my earlier comment that slices, as proposed cannot be matrices because they store a reference to the data from which they were created and a bunch of offsets for accessing the orinal data properly. When passed to math/sci libraries, they'll have to be converted to real matrices so that their data are contiguous. I see. "Function" means "function in an external library", not "Python function" as I had assumed. BTW, there is a good book called "Scientific and Engineering C++" that appeared recently, unfortunately I don't remember the names of the authors (but I can check at home). It contains a very good discussion of how a matrix class in C++ could be designed, and this covers many of the topics we are discussing here. Unfortunately they treat only one- and two-dimensional matrices. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 14:33:36 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA10492 for matrix-sig-people; Wed, 13 Sep 1995 14:33:36 -0400 Received: from servrcolkr.cr.usgs.gov (servrcolkr.cr.usgs.gov [136.177.112.5]) by python.org (8.6.12/8.6.12) with ESMTP id OAA10487 for ; Wed, 13 Sep 1995 14:33:31 -0400 Received: from localhost (dsjfqvarsa.er.usgs.gov [130.11.51.73]) by servrcolkr.cr.usgs.gov (EMAIL 1.2.1) with ESMTP id SAA17250; Wed, 13 Sep 1995 18:34:14 GMT Message-Id: <199509131834.SAA17250@servrcolkr.cr.usgs.gov> To: hinsenk@ere.umontreal.ca (Hinsen Konrad) cc: "James L Fulton, Hydrologist, Reston, VA " cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Let's get going In-reply-to: <199509131828.OAA11695@cyclone.ERE.UMontreal.CA> Date: Wed, 13 Sep 1995 14:34:11 -0400 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk On Wed, 13 Sep 1995 14:28:26 -0400 Hinsen Konrad said: > > > Why would it have to be copied whenever it is passed to a function? > > It would be sufficient to make a copy whenever the matrix is > > changed. > > This gets back to my earlier comment that slices, as proposed cannot > be matrices because they store a reference to the data from which they > were created and a bunch of offsets for accessing the orinal data > properly. When passed to math/sci libraries, they'll have to be > converted to real matrices so that their data are contiguous. > > I see. "Function" means "function in an external library", not > "Python function" as I had assumed. Python functions won't really care whether they get a "true" matrix, or a sequence of sequences, as they will use []s to access data in either case. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 15:44:48 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA10861 for matrix-sig-people; Wed, 13 Sep 1995 15:44:48 -0400 Received: from rnet.rose.rsoc.rockwell.com (rose.rsoc.rockwell.com [161.40.39.100]) by python.org (8.6.12/8.6.12) with SMTP id PAA10857 for ; Wed, 13 Sep 1995 15:44:45 -0400 Received: by rnet.rose.rsoc.rockwell.com (4.1/SMI-4.1) id AA23046; Wed, 13 Sep 95 14:45:44 CDT Received: from sunrise(161.40.50.11) by rnet via smap (V1.3) id sma023042; Wed Sep 13 14:45:13 1995 Received: from feynman.rsoc.rockwell.com by rose.rsoc.rockwell.com (5.0/SMI-SVR4) id AA06888; Wed, 13 Sep 1995 14:45:10 -0500 Received: by feynman.rsoc.rockwell.com (5.0/SMI-SVR4) id AA01068; Wed, 13 Sep 1995 14:45:10 -0500 Date: Wed, 13 Sep 1995 14:45:10 -0500 From: forrest@rose.rsoc.rockwell.com (Dave Forrest) Message-Id: <9509131945.AA01068@feynman.rsoc.rockwell.com> To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Are we talking about one thing or two? X-Sun-Charset: US-ASCII Content-Length: 951 Sender: owner-matrix-sig@python.org Precedence: bulk > From owner-matrix-sig@python.org Wed Sep 13 12:46 CDT 1995 [snip] > > Why? For practical reasons. We here - and I would submit that many > other organizations are like this - do more Matlab-like work. The way > Matlab defines a matrix is not a problem but a huge advantage for us. > > But in what way would a more general concept be a disadvantage? Easy - a more general concept might require me to worry about extra things that I'm not interested in (like the number of dimensions, or the "rank" that J uses). These things are not interesting and can only cause problems by being there to make mistakes on. We will end up writing a wrapper that gives us the interface we want and hides the things that we're not interested in - but since we're not the only ones who want this and it would be more efficiently done as an intrinsic part of the language I think that's the right thing to do. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 16:38:20 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA11112 for matrix-sig-people; Wed, 13 Sep 1995 16:38:20 -0400 Received: from servrcolkr.cr.usgs.gov (servrcolkr.cr.usgs.gov [136.177.112.5]) by python.org (8.6.12/8.6.12) with ESMTP id QAA11108 for ; Wed, 13 Sep 1995 16:38:17 -0400 Received: from localhost (dsjfqvarsa.er.usgs.gov [130.11.51.73]) by servrcolkr.cr.usgs.gov (EMAIL 1.2.1) with ESMTP id UAA22390; Wed, 13 Sep 1995 20:38:58 GMT Message-Id: <199509132038.UAA22390@servrcolkr.cr.usgs.gov> To: James Hugunin cc: "James L Fulton, Hydrologist, Reston, VA " cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Let's get going In-reply-to: <9509131639.AA04160@nineveh.LCS.MIT.EDU> Date: Wed, 13 Sep 1995 16:38:56 -0400 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk On Wed, 13 Sep 95 12:39:13 -0400 James Hugunin said: > (snip) > > I would also like to comment on some of the less controversial points > raised by Jim Fulton in his first post regarding basic methods for matrix > objects. > > >> transpose - The transpose of the matrix (not needed if "~" is used for > >> transpose) > > I'm thinking that this (or ~) should return by reference. > > I agree completely. In fact, when I went about implementing matrix slices > (so that m[((1,3),)] can be properly returned by reference, I noticed that > transposing a matrix was trivially implemented by converting it to a matrix > slice. This of course raises the issue of how to properly deal with these > matrix slices. > > For example, the way my code is written now, if I try to add a slice to a > matrix, I can perform the operation quickly without needing to convert the > slice to a matrix, as long as the slice meets a fairly obscure list of > requirements. Basically these can be summarized as the fact that there must > be a constant offset between the indices for each dimensions. ie. > > M[((1,3,5),)] would be easy to deal with (as fast as a matrix) whereas: > M[((1,3,6),)] would not be I think this is too strange. > What bothers me about this gets back to your earlier complaints regarding > automatic coercion as being a possible source of errors (which I've come > around to agreeing with in general). On the other hand, conceptually, it > would be nice to be able to treat a slice exactly the same as a matrix > whenever possible. Right. This is polymorphism, not coercion. > I see three options, none of which seems perfect, but I am inclined towards > the third. > > 1) Don't allow slices to be passed to functions which require a matrix > argument, and don't allow numeric operations on slices Yuck. > 2) Treat a slice just like any other non-matrix sequence, and allow numeric > operations, and allow it to be passed to matrix functions using the generic > sequence to matrix functions (there is an interesting issue here having to > do with copying data back from the matrix to the input sequence if it turns > out that the function modifies it's input data. I'm curious as to how you > deal with this, or whether you just don't allow this to happen). > 3) Whenever a slice is passed to a function requiring a matrix argument, > automatically copy the data from the slice to a contiguous block of memory. > Allow slices to be used in numeric operations, doing this efficiently where > possible, and copying the slice to a contiguous block of memeory where not > possible. This would be the most efficient solution. I can't tell 2 and three apart. Currently, if FIDL calls a function that modifies one of it's arguments, the modified value is returned as an element of the return list. Non-scalar output/modify values are always matrices. If a modify argument *is* a matrix, then it gets modified, otherwise it gets copied. This is a bit gross but it handles most cases. > > >> byteswap - Useful for reading data from a file written on a machine with a > >> different byte order > > > Hm. Does this operate in-place or return a value? > > In-place. In fact, I am tempted to try and make all of the methods on > matrices operate "in-place" in order to be in line with list objects. I'm not sure what you mean by this. Surely, you aren't trying to make: m=[[1,2,3],[4,5,6],[7,8,9]] b=[11,22,33] m=[1]=b b[1]=99 cause m[1][1] to equal 99? Are you? Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 16:46:28 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA11173 for matrix-sig-people; Wed, 13 Sep 1995 16:46:28 -0400 Received: from retro.jhuapl.edu (retro.jhuapl.edu [128.244.146.239]) by python.org (8.6.12/8.6.12) with ESMTP id QAA11169 for ; Wed, 13 Sep 1995 16:46:25 -0400 Message-Id: <199509132046.QAA11169@python.org> Received: by retro.jhuapl.edu (1.37.109.16/16.2) id AA180845239; Wed, 13 Sep 1995 16:47:19 -0400 Date: Wed, 13 Sep 1995 16:47:19 -0400 From: Chris Chase S1A To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Are we talking about one thing or two? In-Reply-To: <9509131636.AA00821@feynman.rsoc.rockwell.com> References: <9509131636.AA00821@feynman.rsoc.rockwell.com> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk >>>>> "Dave" == Dave Forrest writes: Dave> Rather than trying to make a one-size-fits all that isn't exactly what Dave> anyone wants, why not make two interfaces (maybe one implementation?) Dave> that give each camp what they want to work with? By two interfaces do you mean having different meanings for an operator such as "*" depending on the arguments classes? I think it would be too confusing to have an operation "A*B" that performs matrix multiplication when A and B are matrices (two dimensional arrays) but performs elementwise multiplication for the a general class of arrays (whatever you want to call them - arrays, tensors, tables - I prefer arrays). I think that supporting the Matlab-like matrix linear algebra such as matrix multiplication and matrix division is a high priority for many people. One question is can the new operators be added to the language without difficulty, in a logical fashion, and without cluttering the language with operators that are only used for this "matrix" class? The Matlab approach uses "+", "-", ".*", "./" ".^" for elementwise operations (called array operations in Matlab. I call them vector operations.) "*", "/", "\", "^" for matrix operations. ("^" is a Matrix power operator). Python is much more general than a matrix language, so I think that using all of these would give Python operator bloat. Is adding new operators to the language even a possibility? New methods or functions would have to be defined anyway to implement the operators, e.g. matrixmultiply(a,b), vectormultiply(a,b), leftdivision(a,b), etc. Without operators that call these functions we would probably want shorter names. >> >> Why? For practical reasons. We here - and I would submit that many >> other organizations are like this - do more Matlab-like work. The way >> Matlab defines a matrix is not a problem but a huge advantage for us. >> >> But in what way would a more general concept be a disadvantage? Dave> Easy - a more general concept might require me to worry about extra Dave> things that I'm not interested in (like the number of dimensions, or Dave> the "rank" that J uses). These things are not interesting and can only Dave> cause problems by being there to make mistakes on. We will end up Dave> writing a wrapper that gives us the interface we want and hides the Dave> things that we're not interested in - but since we're not the only ones Dave> who want this and it would be more efficiently done as an intrinsic Dave> part of the language I think that's the right thing to do. Even if you limit yourself to only two dimensions you still have to worry about rank. In Matlab you can make a mistake using one-dimensional vectors when a matrix is required. Tela is a perfect example of a language that took the Matlab syntax and added higher dimensional arrays without breaking any of the basic Matlab matrix features. IDL and APL are additional examples with higher dimensional arrays that don't limit their capability to do matrix linear algebra. Matlab is two-dimensional more because of its limited scope and computer resources from historical beginnings. I even seem to recall being informed by Mathworks representatives at a tradeshow that higher dimensional arrays would be added in a future release to Matlab. The natural implemenation would be a multi-dimensional array class as Jim is proposing with a matrix linear algebra module to specialize this class [for implementing matrix multiplication, matrix inverses/adjoints (square and non-square "matrix division") , linear equation solvers, column rank, factorization, spectral/eigenvalue computations, special matrices (diagonal, Hankel, toeplitz, Hilbert, sparse, etc.), norms, matrix exponential, and on and on]. This would easily be a whole additional SIG - the Matrix Linear Algebra (Matlab clone) SIG. Such a SIG wouldn't have to implement another object type and external interface except for subclassing to special matrices (sparse in particular). Is the purpose of this SIG only for implementing a two-dimensional matrix linear algebra system within Python or for implementing a more general purpose multi-dimensional array class with homogeneous elements for easy interface to external scientific and mathematical libraries? Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 16:54:44 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA11209 for matrix-sig-people; Wed, 13 Sep 1995 16:54:44 -0400 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id QAA11205 for ; Wed, 13 Sep 1995 16:54:42 -0400 Received: from nineveh.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA02516; Wed, 13 Sep 95 16:54:59 EDT Received: by nineveh.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA04255; Wed, 13 Sep 95 17:01:32 -0400 Message-Id: <9509132101.AA04255@nineveh.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Wed, 13 Sep 95 17:01:31 -0400 To: "Jim Fulton, U.S. Geological Survey" Subject: Re: [PYTHON MATRIX-SIG] Let's get going Cc: matrix-sig@python.org References: <199509132038.UAA22390@servrcolkr.cr.usgs.gov> Sender: owner-matrix-sig@python.org Precedence: bulk I've got to run to a meeting, but I wanted to reply to your points. I'm not at all sure that we disagree on the implementation of slices. Let me ask two simple questions. Let's say A is a slice taken from a matrix, and B is a contiguous matrix. 1) Should I be able to say C = A+B, and have the obvious thing happen (assuming that dimensions line up, etc.)? I'd say yes, and I have a few performance optimizations to make this fast in certain cases and nobody needs to worry about those. 2) Should I be able to say fft(A) where fft is an in-place FFT routine? Should it be expected to modify the memory referred to by A, or only to return a brand-new matrix which corresponds to the fft of A? I'm not sure what the right answer to this one is. > > > Hm. Does this operate in-place or return a value? > > > > In-place. In fact, I am tempted to try and make all of the methods on > > matrices operate "in-place" in order to be in line with list objects. > > I'm not sure what you mean by this. Surely, you aren't trying to > make: > > m=[[1,2,3],[4,5,6],[7,8,9]] > b=[11,22,33] > m=[1]=b > b[1]=99 > > cause m[1][1] to equal 99? Are you? Not at all! All I meant by this is that methods on list objects (like insert or append) actually change the list object that they are operating on, whereas operations like concatenation return a new object. In that vein, I feel that m.byteswap() should operate in-place on m and cause the memory associated with m to be byte-swapped. This is as opposed to having it return a new matrix in with the same dimensions as m, but with all values byte-swapped. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 16:54:49 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA11220 for matrix-sig-people; Wed, 13 Sep 1995 16:54:49 -0400 Received: from servrcolkr.cr.usgs.gov (servrcolkr.cr.usgs.gov [136.177.112.5]) by python.org (8.6.12/8.6.12) with ESMTP id QAA11215 for ; Wed, 13 Sep 1995 16:54:46 -0400 Received: from localhost (dsjfqvarsa.er.usgs.gov [130.11.51.73]) by servrcolkr.cr.usgs.gov (EMAIL 1.2.1) with ESMTP id UAA23220; Wed, 13 Sep 1995 20:55:30 GMT Message-Id: <199509132055.UAA23220@servrcolkr.cr.usgs.gov> To: chris.chase@jhuapl.edu cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Are we talking about one thing or two? In-reply-to: <199509132046.QAA11169@python.org> Date: Wed, 13 Sep 1995 16:55:28 -0400 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk On Wed, 13 Sep 1995 16:47:19 -0400 Chris Chase S1A said: > >>>>> "Dave" == Dave Forrest writes: > > Dave> Rather than trying to make a one-size-fits all that isn't exactly what > Dave> anyone wants, why not make two interfaces (maybe one implementation?) > Dave> that give each camp what they want to work with? > > By two interfaces do you mean having different meanings for an > operator such as "*" depending on the arguments classes? > > I think it would be too confusing to have an operation "A*B" that > performs matrix multiplication when A and B are matrices (two > dimensional arrays) but performs elementwise multiplication for the > a general class of arrays (whatever you want to call them - arrays, > tensors, tables - I prefer arrays). Hm. Well this is what happens now with numbers. Math on integers is different from math on numbers. > I think that supporting the Matlab-like matrix linear algebra such as > matrix multiplication and matrix division is a high priority for many > people. One question is can the new operators be added to the > language without difficulty, in a logical fashion, and without > cluttering the language with operators that are only used for this > "matrix" class? > > > The Matlab approach uses > > "+", "-", ".*", "./" ".^" for elementwise operations (called array > operations in Matlab. I call them vector operations.) > > "*", "/", "\", "^" for matrix operations. ("^" is a Matrix power operator). > > Python is much more general than a matrix language, so I think that > using all of these would give Python operator bloat. > > Is adding new operators to the language even a possibility? I'm 99% sure that the answer is no. I think that we should try to keep this proposal to things that can be implemented with a new module withion the bounds of the existing language. > New methods or functions would have to be defined anyway to implement > the operators, e.g. matrixmultiply(a,b), vectormultiply(a,b), > leftdivision(a,b), etc. Without operators that call these functions we > would probably want shorter names. > > >> > >> Why? For practical reasons. We here - and I would submit that many > >> other organizations are like this - do more Matlab-like work. The way > >> Matlab defines a matrix is not a problem but a huge advantage for us. > >> > >> But in what way would a more general concept be a disadvantage? > > Dave> Easy - a more general concept might require me to worry about extra > Dave> things that I'm not interested in (like the number of dimensions, or > Dave> the "rank" that J uses). These things are not interesting and can only > Dave> cause problems by being there to make mistakes on. We will end up > Dave> writing a wrapper that gives us the interface we want and hides the > Dave> things that we're not interested in - but since we're not the only ones > Dave> who want this and it would be more efficiently done as an intrinsic > Dave> part of the language I think that's the right thing to do. > > Even if you limit yourself to only two dimensions you still have to > worry about rank. In Matlab you can make a mistake using > one-dimensional vectors when a matrix is required. Tela is a perfect > example of a language that took the Matlab syntax and added higher > dimensional arrays without breaking any of the basic Matlab matrix > features. IDL and APL are additional examples with higher dimensional > arrays that don't limit their capability to do matrix linear algebra. > Matlab is two-dimensional more because of its limited scope and > computer resources from historical beginnings. I even seem to recall > being informed by Mathworks representatives at a tradeshow that higher > dimensional arrays would be added in a future release to Matlab. > > The natural implemenation would be a multi-dimensional array class as > Jim is proposing with a matrix linear algebra module to specialize > this class [for implementing matrix multiplication, matrix > inverses/adjoints (square and non-square "matrix division") , linear > equation solvers, column rank, factorization, spectral/eigenvalue > computations, special matrices (diagonal, Hankel, toeplitz, Hilbert, > sparse, etc.), norms, matrix exponential, and on and on]. This would > easily be a whole additional SIG - the Matrix Linear Algebra (Matlab > clone) SIG. Such a SIG wouldn't have to implement another object type > and external interface except for subclassing to special matrices > (sparse in particular). > > Is the purpose of this SIG only for implementing a two-dimensional > matrix linear algebra system within Python or for implementing a more > general purpose multi-dimensional array class with homogeneous > elements for easy interface to external scientific and mathematical > libraries? The latter. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 17:18:20 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA11353 for matrix-sig-people; Wed, 13 Sep 1995 17:18:20 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id RAA11349 for ; Wed, 13 Sep 1995 17:18:14 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id RAA24718 (8.6.11/IDA-1.6); Wed, 13 Sep 1995 17:17:46 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA08973; Wed, 13 Sep 1995 17:17:45 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA20989; Wed, 13 Sep 1995 17:17:54 -0400 Date: Wed, 13 Sep 1995 17:17:54 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509132117.RAA20989@cyclone.ERE.UMontreal.CA> To: chris.chase@jhuapl.edu CC: matrix-sig@python.org In-reply-to: <199509132046.QAA11169@python.org> (message from Chris Chase S1A on Wed, 13 Sep 1995 16:47:19 -0400) Subject: Re: [PYTHON MATRIX-SIG] Are we talking about one thing or two? Sender: owner-matrix-sig@python.org Precedence: bulk The natural implemenation would be a multi-dimensional array class as Jim is proposing with a matrix linear algebra module to specialize this class [for implementing matrix multiplication, matrix inverses/adjoints (square and non-square "matrix division") , linear ... Why specialise? That only limits the flexibility of the whole package. Why shouldn't it be possible to have "inverse" invert each two-dimensional slice of a three-dimensional array? There are applications for it, it doesn't cost any extra effort to do it, so why not do it? Why invent artificial restrictions? Is the purpose of this SIG only for implementing a two-dimensional matrix linear algebra system within Python or for implementing a more general purpose multi-dimensional array class with homogeneous elements for easy interface to external scientific and mathematical libraries? I'd say both. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 17:23:26 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA11380 for matrix-sig-people; Wed, 13 Sep 1995 17:23:26 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id RAA11376 for ; Wed, 13 Sep 1995 17:23:23 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id QAA22452 (8.6.11/IDA-1.6); Wed, 13 Sep 1995 16:11:20 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA09390; Wed, 13 Sep 1995 16:11:19 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA17017; Wed, 13 Sep 1995 16:10:49 -0400 Date: Wed, 13 Sep 1995 16:10:49 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509132010.QAA17017@cyclone.ERE.UMontreal.CA> To: forrest@rose.rsoc.rockwell.com CC: matrix-sig@python.org In-reply-to: <9509131945.AA01068@feynman.rsoc.rockwell.com> (forrest@rose.rsoc.rockwell.com) Subject: Re: [PYTHON MATRIX-SIG] Are we talking about one thing or two? Sender: owner-matrix-sig@python.org Precedence: bulk Easy - a more general concept might require me to worry about extra things that I'm not interested in (like the number of dimensions, or the "rank" that J uses). These things are not interesting and can only cause problems by being there to make mistakes on. We will end u writing a wrapper that gives us the interface we want and hides the But it doesn't, as several decades of APL experience have shown. I know several APL users who are doing linear-algebra type things and are not even aware of the existence of higher-dimensional arrays. Your "wrapper" would only map some operations on "matrices" on exactly the same operations on "arrays" and leave out some others. That's just an unnecessary complication. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 17:30:18 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA11429 for matrix-sig-people; Wed, 13 Sep 1995 17:30:18 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id RAA11425 for ; Wed, 13 Sep 1995 17:30:14 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id RAA25097 (8.6.11/IDA-1.6); Wed, 13 Sep 1995 17:30:13 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA14823; Wed, 13 Sep 1995 17:30:12 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA21754; Wed, 13 Sep 1995 17:30:21 -0400 Date: Wed, 13 Sep 1995 17:30:21 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509132130.RAA21754@cyclone.ERE.UMontreal.CA> To: jjh@mama-bear.lcs.mit.edu CC: matrix-sig@python.org In-reply-to: <9509132101.AA04255@nineveh.LCS.MIT.EDU> (message from James Hugunin on Wed, 13 Sep 95 17:01:31 -0400) Subject: Re: [PYTHON MATRIX-SIG] Let's get going Sender: owner-matrix-sig@python.org Precedence: bulk 2) Should I be able to say fft(A) where fft is an in-place FFT routine? Should it be expected to modify the memory referred to by A, or only to return a brand-new matrix which corresponds to the fft of A? I'm not sure what the right answer to this one is. Both operations are useful in different contexts, so why not have both? One possibility would be to have a function fft(A) that returns a new array, and a method a.fft() to do an in-place FFT. Implementationally the function would just copy A and call the method. The same applies to inversion, factorization etc. It would be nice to have some consistency here - always functions for copies, always methods for in-place modifications. > I'm not sure what you mean by this. Surely, you aren't trying to > make: > > m=[[1,2,3],[4,5,6],[7,8,9]] > b=[11,22,33] > m=[1]=b > b[1]=99 > > cause m[1][1] to equal 99? Are you? Actually this is not such a silly question, because Python's lists behave exactly in this way. So the question is whether we want arrays to have reference semantice (like Python lists) or value semantics (like Python integers, floats, and complex numbers). I strongly recommend the latter - reference semantics for arrays quickly produce confusion, as I had to find out while playing with such an implementation in Smalltalk. on, whereas operations like concatenation return a new object. In that vein, I feel that m.byteswap() should operate in-place on m and cause the memory associated with m to be byte-swapped. This is as opposed to having it return a new matrix in with the same dimensions as m, but with all values byte-swapped. This depends on what you would use byte swapping for. In fact, I am not at all convinced of its utility. What would happen when such a byte-swapped matrix is accessed? Would all operations on it still be allowed? That would create an enormous implementation effort. If not, then a byte-swapped matrix would no longer be a matrix, because none of the matrix operations could be used on it. The only reason I can see for a byte swapping operation is during I/O, so that's where it should go (as a flag to I/O functions). In general, I have the impression that we get too much lost in implementational details. Let's first define arrays as an abstract data type in terms of the operations allowed on it from a user's point of view and *then* worry about implementational details. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 17:39:59 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA11481 for matrix-sig-people; Wed, 13 Sep 1995 17:39:59 -0400 Received: from servrcolkr.cr.usgs.gov (servrcolkr.cr.usgs.gov [136.177.112.5]) by python.org (8.6.12/8.6.12) with ESMTP id RAA11477 for ; Wed, 13 Sep 1995 17:39:53 -0400 Received: from localhost (dsjfqvarsa.er.usgs.gov [130.11.51.73]) by servrcolkr.cr.usgs.gov (EMAIL 1.2.1) with ESMTP id VAA24360; Wed, 13 Sep 1995 21:40:43 GMT Message-Id: <199509132140.VAA24360@servrcolkr.cr.usgs.gov> To: James Hugunin cc: "James L Fulton, Hydrologist, Reston, VA " cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Let's get going In-reply-to: <9509132101.AA04255@nineveh.LCS.MIT.EDU> Date: Wed, 13 Sep 1995 17:40:42 -0400 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk On Wed, 13 Sep 95 17:01:31 -0400 James Hugunin said: > > I've got to run to a meeting, but I wanted to reply to your points. > > I'm not at all sure that we disagree on the implementation of slices. Let > me ask two simple questions. Let's say A is a slice taken from a matrix, > and B is a contiguous matrix. > > 1) Should I be able to say C = A+B, and have the obvious thing happen > (assuming that dimensions line up, etc.)? Yes. > I'd say yes, and I have a few performance optimizations to make this fast > in certain cases and nobody needs to worry about those. Cool. > > 2) Should I be able to say fft(A) where fft is an in-place FFT routine? > Should it be expected to modify the memory referred to by A, or only to > return a brand-new matrix which corresponds to the fft of A? > > I'm not sure what the right answer to this one is. My opinion, in general is that functions should return new objects, however, reasonable performance arguments can be made for at least some cases of having functions modify large matrices rather than creating new ones. In the case above, even if I had fft modify it's argument, I would still have it return the argument as a return value. The problem is that you want to be able to call routines that expect contigous data. Slice data, even for the cases you mentioned aren't like that. Slices could be implemented with their own contiguous data area that they keep in sync with the original matrices data, but I think this would complicate the implementation of slices beyound their benefit. Of course, this is only a problem when calling Fortran or C. Python functions (or C functions written for Python, which are much faster than python functions, but have many of the benefits) should nor present a problem. I suppose when calling a library function that wants to modify it's data, one could (within the glue code) do something equivalent to: def spam_glue(m): if type(m) is MatrixType: return spam(m) else: t=Matrix(m) r=spam(t) m[:]=t return r That is the (automagically generated) glue code could create a temporrary matrix as copy of the original data, call the function with the copy, and then slice-assign the modified copy back to the argument. Note this should work work with any sequence, such as a slice, that allows assignment from any arbitrary sequence. (Grrrrrr. Unfortunately, this won't work with list arguments, as the following code fails: a=[1,2,3] a[:]=(7,8,9) # This fails, but shouldn't, IMHO This should work!) I think I could work this behavior into FIDL, without much trouble. Hm. So perhaps allowing slices to be used as modifyable arguments might not be too bad, as long as the implementor is willing to do a little extra work. (In my case, the implementor is a program, so I don't mind if it works hard. ;) You still pay the performance penalty of making a copy, but at least functions that provide performance wins when used with matrices can still function correctly with slices. > > > > > Hm. Does this operate in-place or return a value? > > > > > > In-place. In fact, I am tempted to try and make all of the methods on > > > matrices operate "in-place" in order to be in line with list objects. > > > > I'm not sure what you mean by this. Surely, you aren't trying to > > make: > > > > m=[[1,2,3],[4,5,6],[7,8,9]] > > b=[11,22,33] > > m=[1]=b > > b[1]=99 > > > > cause m[1][1] to equal 99? Are you? > > Not at all! All I meant by this is that methods on list objects (like > insert or append) actually change the list object that they are operating > on, whereas operations like concatenation return a new object. In that > vein, I feel that m.byteswap() should operate in-place on m and cause the > memory associated with m to be byte-swapped. This is as opposed to having > it return a new matrix in with the same dimensions as m, but with all values > byte-swapped. Cool. Actually, I would use a naming convention to make this clear, so I might have both "byteswap" which modifies in place, and "byteswapped" which returns a new objects. (BTW Guido, I wish lists has a "sorted" operation that returned a new sorted list.) Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 17:52:21 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA11516 for matrix-sig-people; Wed, 13 Sep 1995 17:52:21 -0400 Received: from servrcolkr.cr.usgs.gov (servrcolkr.cr.usgs.gov [136.177.112.5]) by python.org (8.6.12/8.6.12) with ESMTP id RAA11512 for ; Wed, 13 Sep 1995 17:52:18 -0400 Received: from localhost (dsjfqvarsa.er.usgs.gov [130.11.51.73]) by servrcolkr.cr.usgs.gov (EMAIL 1.2.1) with ESMTP id VAA25125; Wed, 13 Sep 1995 21:52:18 GMT Message-Id: <199509132152.VAA25125@servrcolkr.cr.usgs.gov> To: hinsenk@ere.umontreal.ca (Hinsen Konrad) cc: jjh@mama-bear.lcs.mit.edu cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Let's get going In-reply-to: <199509132130.RAA21754@cyclone.ERE.UMontreal.CA> Date: Wed, 13 Sep 1995 17:52:16 -0400 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk On Wed, 13 Sep 1995 17:30:21 -0400 Hinsen Konrad said: > > 2) Should I be able to say fft(A) where fft is an in-place FFT routine? > Should it be expected to modify the memory referred to by A, or only to > return a brand-new matrix which corresponds to the fft of A? > > I'm not sure what the right answer to this one is. > > Both operations are useful in different contexts, so why not > have both? One possibility would be to have a function fft(A) that > returns a new array, and a method a.fft() to do an in-place > FFT. Implementationally the function would just copy A and call > the method. The same applies to inversion, factorization etc. > It would be nice to have some consistency here - always functions > for copies, always methods for in-place modifications. > > > I'm not sure what you mean by this. Surely, you aren't trying to > > make: > > > > m=[[1,2,3],[4,5,6],[7,8,9]] > > b=[11,22,33] > > m=[1]=b > > b[1]=99 > > > > cause m[1][1] to equal 99? Are you? > > Actually this is not such a silly question, because Python's lists > behave exactly in this way. I didn't say it was a silly question. I know that lists behave this way, but making matrices behave this way complicates things alot. > So the question is whether we want > arrays to have reference semantice (like Python lists) or value > semantics (like Python integers, floats, and complex numbers). > I strongly recommend the latter - reference semantics for > arrays quickly produce confusion, as I had to find out while > playing with such an implementation in Smalltalk. You have to have some reference semantics to make m[i][j] work. Actuall, the proposal has reference semantics for reference (ie __getitem__) and copy semantics for element/slice assignment (__setitem__). > on, whereas operations like concatenation return a new object. In that > vein, I feel that m.byteswap() should operate in-place on m and cause the > memory associated with m to be byte-swapped. This is as opposed to having > it return a new matrix in with the same dimensions as m, but with all values > byte-swapped. > > This depends on what you would use byte swapping for. In fact, I am > not at all convinced of its utility. What would happen when such > a byte-swapped matrix is accessed? Would all operations on it > still be allowed? That would create an enormous implementation effort. > If not, then a byte-swapped matrix would no longer be a matrix, > because none of the matrix operations could be used on it. > The only reason I can see for a byte swapping operation is during > I/O, so that's where it should go (as a flag to I/O functions). > > > In general, I have the impression that we get too much lost in > implementational details. Let's first define arrays as an abstract > data type in terms of the operations allowed on it from a > user's point of view and *then* worry about implementational > details. I agree that we should try to focus on requirements, however, a very basic requirement of this type is it's ability to support interfacing with existing numerical routines, which imposes some significant restrictions on it's implementation. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 18:33:19 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA11754 for matrix-sig-people; Wed, 13 Sep 1995 18:33:19 -0400 Received: from retro.jhuapl.edu (retro.jhuapl.edu [128.244.146.239]) by python.org (8.6.12/8.6.12) with ESMTP id SAA11749 for ; Wed, 13 Sep 1995 18:33:16 -0400 Message-Id: <199509132233.SAA11749@python.org> Received: by retro.jhuapl.edu (1.37.109.16/16.2) id AA186081648; Wed, 13 Sep 1995 18:34:08 -0400 Date: Wed, 13 Sep 1995 18:34:08 -0400 From: Chris Chase S1A To: matrix-sig@python.org Cc: hinsenk@ere.umontreal.ca (Hinsen Konrad) Subject: [PYTHON MATRIX-SIG] Re: Are we talking about one thing or two? In-Reply-To: <199509132117.RAA20989@cyclone.ERE.UMontreal.CA> References: <199509132046.QAA11169@python.org> <199509132117.RAA20989@cyclone.ERE.UMontreal.CA> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk Hinsen> Why specialise? That only limits the flexibility of the whole Hinsen> package. Why shouldn't it be possible to have "inverse" invert Hinsen> each two-dimensional slice of a three-dimensional array? There Hinsen> are applications for it, it doesn't cost any extra effort to Hinsen> do it, so why not do it? Why invent artificial restrictions? I did not mean that this could not be done. Inverse() in this context is an operation on square matrices (R^NxN). It is not a concept that I have experience generalizing to higher dimensions. Your example maps the inverse() function onto the two-dimensional slices. I think of this as a mapping operator that is applied to an indexed set of projections followed by application of inverse(). In general, you could apply any matrix function in this manner. You could do this for spectral factorization, diagonalization, etc. There is no need to redefine your matrix linear algebra functions. Instead you use a projection mapping/iterator (rank selector or whatever you want to call it) to apply the matrix function to each two-dimensional slice. This would seem the natural way to implement it. Many matrix functions such as inverse() would be best implemented by a call to an already existing and efficient C or FORTRAN library function. Perhaps it would be more elegant (and like APL) if the rank selection was a part of the inverse() parameter list but it can just as easily be a part of a projection mapping function that does the rank selection and then applies inverse(), i.e., a variation of the builtin map(): map(function, array, rank selection parameters) This avoids having to recode the same mechanism for each matrix function. It also provides a method for applying already coded lower dimensional operators (1D, 2D, 3D, whatever) to higher dimensional arrays without having to recode. James Hugunin made a comment about this - many of the mapping and reduction operations are generalizations that can be implemented in Python with calls to the appropriate external matrix-based function (or whatever the lower dimensional function is). Chris Chase ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 21:15:56 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id VAA12404 for matrix-sig-people; Wed, 13 Sep 1995 21:15:56 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id VAA12400 for ; Wed, 13 Sep 1995 21:15:52 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id VAA29547 (8.6.11/IDA-1.6); Wed, 13 Sep 1995 21:15:49 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id VAA15104; Wed, 13 Sep 1995 21:15:47 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id VAA03573; Wed, 13 Sep 1995 21:15:57 -0400 Date: Wed, 13 Sep 1995 21:15:57 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509140115.VAA03573@cyclone.ERE.UMontreal.CA> To: jfulton@usgs.gov CC: matrix-sig@python.org In-reply-to: <199509132152.VAA25125@servrcolkr.cr.usgs.gov> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] Let's get going Sender: owner-matrix-sig@python.org Precedence: bulk You have to have some reference semantics to make m[i][j] work. Actuall, the proposal has reference semantics for reference (ie __getitem__) and copy semantics for element/slice assignment (__setitem__). Indexing is really just syntactic sugar for getitem() and setitem(), so I wouldn't call that reference semantics. I agree that we should try to focus on requirements, however, a very basic requirement of this type is it's ability to support interfacing with existing numerical routines, which imposes some significant restrictions on it's implementation. Sure. But that's just one point on the list of requirements ;-) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 13 21:46:18 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id VAA12507 for matrix-sig-people; Wed, 13 Sep 1995 21:46:18 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id VAA12503 for ; Wed, 13 Sep 1995 21:46:12 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id VAA00006 (8.6.11/IDA-1.6); Wed, 13 Sep 1995 21:46:06 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id VAA20831; Wed, 13 Sep 1995 21:46:05 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id VAA04451; Wed, 13 Sep 1995 21:46:15 -0400 Date: Wed, 13 Sep 1995 21:46:15 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509140146.VAA04451@cyclone.ERE.UMontreal.CA> To: chris.chase@jhuapl.edu CC: matrix-sig@python.org In-reply-to: <199509132233.SAA27339@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Wed, 13 Sep 1995 18:34:08 -0400) Subject: [PYTHON MATRIX-SIG] Re: Are we talking about one thing or two? Sender: owner-matrix-sig@python.org Precedence: bulk I did not mean that this could not be done. Inverse() in this context is an operation on square matrices (R^NxN). It is not a concept that I have experience generalizing to higher dimensions. Your example maps the inverse() function onto the two-dimensional slices. I think of this as a mapping operator that is applied to an indexed set of projections followed by application of inverse(). In the rank concept that I outlined before, matrix inversion (and similar operations) is an operation with an intrinsic rank of 2. Its meaning for rank 3 arrays is thus an automatic consequence of the general rank rules and *not* an additional rule. You could specify an explicit rank of 1, reducing matrix inversion to scalar inversion. Specifying a higher rank would lead to an error message. Nevertheless, the possibility of specifying explicit ranks is important, since they can also be variables to be determined at runtime. There are applications where this is useful. And I would like to stress again that this generalization comes at no disadvantage to "two-dimensional only" users. slice. This would seem the natural way to implement it. Many matrix functions such as inverse() would be best implemented by a call to an already existing and efficient C or FORTRAN library function. Implmentation is a different story. I agree that using existing libraries makes sense. I would nevertheless prefer C libraries to Fortran libraries, as otherwise a Fortran compiler would become necessary to install Python, and not every Unix system comes with a Fortran compiler. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Sep 14 08:34:59 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id IAA14107 for matrix-sig-people; Thu, 14 Sep 1995 08:34:59 -0400 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id IAA14100 for ; Thu, 14 Sep 1995 08:34:56 -0400 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa18624; 14 Sep 95 8:27 EDT Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) for id IAA01106; Thu, 14 Sep 1995 08:27:04 -0400 Message-Id: <199509141227.IAA01106@monty> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Why I won't add complex numbers to the base language From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Thu, 14 Sep 1995 08:27:04 -0400 Sender: owner-matrix-sig@python.org Precedence: bulk [This is the first of a series of small essays that treat several issues that have been brought up in the matrix sig. I'd like to make an effort to at least separate the threads by subject, if not putting some issues to rest altogether.] It's been proposed to add complex numbers to the base language, and when I muttered I didn't like that the response was "there's no mathematical reason not to." Let me rebutt. 1. Mathematically, there is at least one difference: complex numbers can't be compared (e.g. z1 < z2 is undefined when either has a nonzero imaginary party). Python will have to define this operation anyhow, since deep inside it requires that each object type defines comparison -- but it leads me to believe that there should be plenty of algorithms out there that wouldn't work if some of their parameters were complex numbers. (At least if the designer of the code didn't think about the possibility of complex numbers, the correctness of the code can't be guaranteed in general.) As long as real and complex are distinct types in Python, there is no fear that complex numbers will be introduced into the computation accidentally (e.g. by taking the sqrt() of a negative number or by a rounding error) -- they are passed in or used explicitly. 2. The available of GNU or other pseudo-free packages for transcendental functions on complex numbers is still a big restriction for Python's portability. Everything that is part of the *core* language has to be portable to every new platform. Since some of these platforms are non Unix, portability of code written by a crowd who assume gcc is available "everywhere" is questionable. There's another problem with GNU code that's only relevant for a certain group but which I care about anyway: people want to be able to embed Python (at least it's core) in commercial applications, and if there's a GNU licence attached to any part of that core, this is a problem. (Don't get me wrong -- I don't mind having Python extensions that use GNU stuff, as long as they can be taken out cleanly and easily by those who can't use GNU stuff, for whatever reason.) 3. The implementation of complex numbers as the only floating-point type in Python would add considerable complexity (no pun intended :-). I take that this should mean that if x is a real number (e.g. 0.0) and z is a complex number (e.g. sqrt(-1.0)), type(x) == type(z). This means that the representation in memory of Python complex number objects either has to contain the imaginary part at all times, or it has to contain a flag telling whether there's an imaginary part or not. Since there's no space in the standard object header for such a flag (at least not without changes that would have repercussions at many other places), it would have to be an additional byte (or more) in the complex number object itself. Say an object header plus malloc overhead is 12 bytes and a double precision float is 8 bytes. If we choose to have an additional flag it gets rounded up to 4 bytes (these are typical and optimistic numbers -- on some platforms your mileage may vary). So we have two choices: either add a flag, so numbers without an imaginary part are 12 (header) + 8(real) + 4 (flag) == 24 bytes and numbers with one are 12 + 8 + 4 + 8(imaginary) == 32 bytes, or all numbers are 12 + 8 + 8 == 28 bytes. Since most Python programs won't be using numbers with imaginary parts (Python being a general programming language), we may want to choose the version with a flag -- but this add at least one if statement to each C function operating on complex numbers (at least two for binary operators). Timewise, the fastest solution would be to always have the imaginary part there -- this would also make the code considerably cleaner. Conclusion: there's a big price to be paid by every Python user for having complex numbers as the only floating point type. Consider the alternative: write an extension module ("complex") that defines arithmetic on complex numbers (there are already examples of how to do this in Python; a C version is totally plausible) and another extensions module ("cmath") that defines transcendental functions on complex number. Now the price is only paid by those who use the feature. I think this solution can still lead to very clean code. I would even sanction a coding style where one writes "from math import *" to use the standard math functions, so it would require only a one-line change to user the complex versions instead ("from cmath import *"). --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Sep 14 08:46:38 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id IAA14155 for matrix-sig-people; Thu, 14 Sep 1995 08:46:38 -0400 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id IAA14151 for ; Thu, 14 Sep 1995 08:46:35 -0400 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa19003; 14 Sep 95 8:44 EDT Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) for id IAA01172; Thu, 14 Sep 1995 08:44:21 -0400 Message-Id: <199509141244.IAA01172@monty> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Forget adding new operators to Python From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Thu, 14 Sep 1995 08:44:20 -0400 Sender: owner-matrix-sig@python.org Precedence: bulk [The second in a series of short essays on subjects raised in the Matrix discussion.] Let me be brief on this one: adding new operators (like ".*", "./" or "\") to the language is no-no. I'm very fond of the fact that nearly all graphical elements of the Python language correspond pretty closely to their use in "everyday life" (with the C language considered to be part of everyday life :-). I should point out that even though the semantics of operators are (almost) entirely defined by their operands, their syntax (including priorities) is not -- the parser doesn't know the operand types. Another, more practical reason is that adding a new operator requires changes to many components of the language implementation -- e.g. if ".*" were to be added as a new numeric operator, I'd have to make changes to every module that implements numbers, if only to add a NULL pointer. The only thing I regret is not having added "**" as an exponentiation operator -- this may happen someday (contributions accepted!). Oh, and there's also agreement that operators like "+=", "*=" should eventually be added; though not "++" and "--". --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Sep 14 09:27:08 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA14334 for matrix-sig-people; Thu, 14 Sep 1995 09:27:08 -0400 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id JAA14330 for ; Thu, 14 Sep 1995 09:27:05 -0400 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa19772; 14 Sep 95 9:20 EDT Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) for id JAA01256; Thu, 14 Sep 1995 09:20:28 -0400 Message-Id: <199509141320.JAA01256@monty> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] A problem with slicing From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Thu, 14 Sep 1995 09:20:28 -0400 Sender: owner-matrix-sig@python.org Precedence: bulk [The third in a series of short essays on subjects raised in the Matrix discussion.] Here's a problem where I have neither a strong opinion nor a perfect solution... Jim Fulton proposes an elegant indexing syntax for matrix objects which doesn't require any changes to the language: M[i][j] references the element at column i and row j (or was that column j and row i? Never mind...). This nicely generalizes to slicing, so you can write: M[i][j1:j2] meaning the column vector at column i with row indices j1...j2-1. Unfortunately, the analogous expression for a row vector won't work: M[i1:i2][j] The reason for this is that it works by interpreting M as a sequence of columns (and it's all evaluated one thing at a time -- M[i][j] means (M[i])[j], and so on). M[i] is column i, so M[i][j] is the element at row j thereof. But slice semantics imply that of M is a sequence of X'es, then M[i1:j1] is still a sequence of X'es -- just shorter. So M[p:q][r] is really the same as M[p+r] (assuming r URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Sep 14 09:39:19 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA14406 for matrix-sig-people; Thu, 14 Sep 1995 09:39:19 -0400 Received: from rnet.rose.rsoc.rockwell.com (rose.rsoc.rockwell.com [161.40.39.100]) by python.org (8.6.12/8.6.12) with SMTP id JAA14402 for ; Thu, 14 Sep 1995 09:39:16 -0400 Received: by rnet.rose.rsoc.rockwell.com (4.1/SMI-4.1) id AA24882; Thu, 14 Sep 95 08:40:05 CDT Received: from sunrise(161.40.50.11) by rnet via smap (V1.3) id sma024875; Thu Sep 14 08:39:43 1995 Received: from feynman.rsoc.rockwell.com by rose.rsoc.rockwell.com (5.0/SMI-SVR4) id AA10685; Thu, 14 Sep 1995 08:39:40 -0500 Received: by feynman.rsoc.rockwell.com (5.0/SMI-SVR4) id AA04618; Thu, 14 Sep 1995 08:39:37 -0500 Date: Thu, 14 Sep 1995 08:39:37 -0500 From: forrest@rose.rsoc.rockwell.com (Dave Forrest) Message-Id: <9509141339.AA04618@feynman.rsoc.rockwell.com> To: matrix-sig@python.org, guido@CNRI.Reston.VA.US Subject: Re: [PYTHON MATRIX-SIG] Forget adding new operators to Python X-Sun-Charset: US-ASCII Content-Length: 525 Sender: owner-matrix-sig@python.org Precedence: bulk This is a strong reason to provide two different interfaces - matrix and array. You can implement them both with the same code, but just provide a simple interface to matrix that restricts itself to two dimensions and does matrix multiplication with operator *. Array then acts "element-wise" and generalizes to any dimension,etc. Result: People who want stuff like my folks want are happy, people who want the more general stuff are happy, it's only implemented once, and no new operators are added to the language. dF ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Sep 14 10:15:25 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA14570 for matrix-sig-people; Thu, 14 Sep 1995 10:15:25 -0400 Received: from servrcolkr.cr.usgs.gov (servrcolkr.cr.usgs.gov [136.177.112.5]) by python.org (8.6.12/8.6.12) with ESMTP id KAA14561 for ; Thu, 14 Sep 1995 10:15:22 -0400 Received: from localhost (dsjfqvarsa.er.usgs.gov [130.11.51.73]) by servrcolkr.cr.usgs.gov (EMAIL 1.2.1) with ESMTP id OAA16059; Thu, 14 Sep 1995 14:15:44 GMT Message-Id: <199509141415.OAA16059@servrcolkr.cr.usgs.gov> To: Guido van Rossum cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] A problem with slicing In-reply-to: <199509141320.JAA01256@monty> Date: Thu, 14 Sep 1995 10:15:42 -0400 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk On Thu, 14 Sep 1995 09:20:28 -0400 Guido van Rossum said: > [The third in a series of short essays on subjects raised in the > Matrix discussion.] > > Here's a problem where I have neither a strong opinion nor a perfect > solution... > > Jim Fulton proposes an elegant indexing syntax for matrix objects > which doesn't require any changes to the language: > > M[i][j] > > references the element at column i and row j (or was that column j and > row i? Never mind...). Actually, it's element j of sub-matrix i. If M is a 2-d matrix, then you may choose to call submatrices either "rows" or "columns". I prefer "columns". > This nicely generalizes to slicing, so you can write: > > M[i][j1:j2] > > meaning the column vector at column i with row indices j1...j2-1. > > Unfortunately, the analogous expression for a row vector won't work: > > M[i1:i2][j] > > The reason for this is that it works by interpreting M as a sequence > of columns (and it's all evaluated one thing at a time -- M[i][j] > means (M[i])[j], and so on). M[i] is column i, so M[i][j] is the > element at row j thereof. But slice semantics imply that of M is a > sequence of X'es, then M[i1:j1] is still a sequence of X'es -- just > shorter. So M[p:q][r] is really the same as M[p+r] (assuming r > > One way out of this is to adopt the syntax > > M[i, j] > > for simple indexing. This would require only a minor tweaking of the > grammar I believe. In fact, this could be as simple as saying that the comma operator generates tuples inside of []s. This is: M[i,j] is equivalent to M[(i,j)]. or even: M[i,] is equivalent to M[(i,)] > This could be extended to support > > M[i1:i2, j] > M[i1:i2, j1:j2] > M[i, j1:j2] > > (and of course higher-dimensional equivalents). > > This would require considerable changes of the run-time architecture > of slicing and indexing, and since currently everything is geared > towards one-dimensional indexing/slicing, but I suppose it would be > doable. I agree. > (Funny how I'm accepting this possibility of changing the language > here, while I'm violently opposed to it for operator definitions. I Yeah. Strange even! ;-) > guess with adding operators there is no end to the number of new > operators you could dream up, so there would be no end to the change; > while here there's a clear-cut one-time change.) Hm. I really don't think this is a good idea. I don't really think we need M[i1:i2, j1:j2]. M[range(i1,i2),range(j1,j2)] is fine for me. Plus, it also allows: M[(1,3,5),(2,4,6)], in other words, we can simply allow a sequence of indexes for a dimension and then let range generate the desired sequence when we want a range. > > Of course adopting such a change would completely ruin any possbility > of using things like > > M[3, 4, 7] = [1, 10, 100] > > as roughly equivalent to > > M[3] = 1 > M[4] = 10 > M[7] = 100 > > but then again I'm not too fond of that anyway (as a matter of fact, > I'd oppose it strongly). > > > Some other things that I haven't completely followed through, and that > may cause complications for the theoretical foundation of it all: > > - Allowing M[i, j] for (multidimensional) sequence types would also > meaning that D[i, j] would be equivalent to D[(i, j)] for > dictionaries. I see no reason to support M[i,j] for arbitrary sequence types. I'd say that if a type wants to support multiple arguments to [], then it should provide mapping behavior and have the mapping implementation sniff for either an integer or a tuple argument and do the right thing. I am *vary much* against a language change to support this. > - Should M[i][j] still be equivalent to M[i, j]? Yes. M[i,j] is really a compact form of M[((i),(j))]. > - Now we have multidimensional sequence types, should be have a > multidimensional equivalent of len()? Some ideas: I'm against multi-dimension sequence types. 8-> > - len(a, i) would return a's length in dimension i; len(a, i) == len(a) > > - dim(a) (or rank(a)?) would return the number of dimensions > > - shape(a) would return a tuple giving a's dimensions, e.g. for a > 3x4 matrix it would return (3, 4), and for a one-dimensional > sequence such as a string or list, it would return a singleton > tuple: (len(a),). Unnecessary. Matrices can provide special methods for this. > - How about multidimensional map(), filter(), reduce()? > > - map() seems easy (except there seems to be no easy way to specify > the rank of the operator): it returns a similarly shaped > multidimensional object whose elements are the function results for > the corresponding elements of the input matrix > > - filter() is problematic since the columns won't be of the same > length > > - reduce()??? -- someone who knows APL tell me what it should mean There have been a number of proposals for generic functions that operate over matrices in some fashion. I have not had time to digest them yet. Stay tuned. :-) (Geez, I really need to get back to my day job.) > - Multidimensional for loops? Or should for iterate over the first > dimension only? What is wrong with nested for loops. Ee-gads, what's gotten into you? :-] > One sees there are many potential consequences of a seemingly simple > change -- Simple? I agree that enabling the tuplefication opertor, ",", in []s is simple, but not adding mult-dimensional behavior to sequences. > that's why I insist that language changes be thought through > in extreme detail before being introduced... I really don't see any reason why the matrix type should require language changes (aside from the minor impact of the tuplefication operator). Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Sep 14 10:40:28 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA14734 for matrix-sig-people; Thu, 14 Sep 1995 10:40:28 -0400 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id KAA14728 for ; Thu, 14 Sep 1995 10:40:24 -0400 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa21971; 14 Sep 95 10:39 EDT Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id KAA01464; Thu, 14 Sep 1995 10:39:00 -0400 Message-Id: <199509141439.KAA01464@monty> To: "Jim Fulton, U.S. Geological Survey" cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] A problem with slicing In-reply-to: Your message of "Thu, 14 Sep 1995 10:15:42 EDT." <199509141415.OAA16059@servrcolkr.cr.usgs.gov> References: <199509141415.OAA16059@servrcolkr.cr.usgs.gov> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Thu, 14 Sep 1995 10:39:00 -0400 Sender: owner-matrix-sig@python.org Precedence: bulk > I really don't see any reason why the matrix type should require > language changes (aside from the minor impact of the tuplefication > operator). I guess one of my points was that, given the desire for a fairly consistent design, allowing tuples as indices is *not* a minor change... (More later.) --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Sep 14 10:49:51 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA14797 for matrix-sig-people; Thu, 14 Sep 1995 10:49:51 -0400 Received: from servrcolkr.cr.usgs.gov (servrcolkr.cr.usgs.gov [136.177.112.5]) by python.org (8.6.12/8.6.12) with ESMTP id KAA14793 for ; Thu, 14 Sep 1995 10:49:47 -0400 Received: from localhost (dsjfqvarsa.er.usgs.gov [130.11.51.73]) by servrcolkr.cr.usgs.gov (EMAIL 1.2.1) with ESMTP id OAA17703; Thu, 14 Sep 1995 14:50:27 GMT Message-Id: <199509141450.OAA17703@servrcolkr.cr.usgs.gov> To: Guido van Rossum cc: "James L Fulton, Hydrologist, Reston, VA " cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] A problem with slicing In-reply-to: <199509141439.KAA01464@monty> Date: Thu, 14 Sep 1995 10:50:25 -0400 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk On Thu, 14 Sep 1995 10:39:00 -0400 Guido van Rossum said: > > I really don't see any reason why the matrix type should require > > language changes (aside from the minor impact of the tuplefication > > operator). > > I guess one of my points was that, given the desire for a fairly > consistent design, allowing tuples as indices is *not* a minor > change... I'm not suggesting that tuples should be allowed as indexes to sequence types. Tuples are *already* allowed as indexes to mapping types. I propose that matrices should provide *both* sequence and mapping behavior, where the mapping behavior is used to support matrix slicing. This requires *no* change to the language. The only *minor* change proposed, which I could live without is to allow the "," to generate tuples inside of []s, just as it does outside of []. In fact, I view the current non-recognition of tuples in []s as an inconsistency. For example: a=1,2,3 is equivalent to: a=(1,2,3) and: for spam in 1,2,3: ... is equivalent to: for spam in (1,2,3): ... so why isn't: a[1,2,3] equivalent to: a[(1,2,3) Note that this change is cosmetic (like making modules callable ;-), and is not *needed* for the matrix proposal. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Sep 14 11:00:34 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA14891 for matrix-sig-people; Thu, 14 Sep 1995 11:00:34 -0400 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id LAA14887 for ; Thu, 14 Sep 1995 11:00:32 -0400 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa22339; 14 Sep 95 10:55 EDT Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id KAA01555; Thu, 14 Sep 1995 10:55:32 -0400 Message-Id: <199509141455.KAA01555@monty> To: "Jim Fulton, U.S. Geological Survey" cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] A problem with slicing In-reply-to: Your message of "Thu, 14 Sep 1995 10:50:25 EDT." <199509141450.OAA17703@servrcolkr.cr.usgs.gov> References: <199509141450.OAA17703@servrcolkr.cr.usgs.gov> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Thu, 14 Sep 1995 10:55:32 -0400 Sender: owner-matrix-sig@python.org Precedence: bulk > The only *minor* change proposed, which I could live without is to > allow the "," to generate tuples inside of []s, just as it does > outside of []. In fact, I view the current non-recognition of tuples > in []s as an inconsistency. For example: [...] > so why isn't: > > a[1,2,3] > > equivalent to: > > a[(1,2,3)] Because there's also a[1:2] while there is no equivalent a(1:2) I could either tweak the priorities so that a[1,2:3,4] is parsed as a[(1,2) : (3,4)] or so that it is parsed as a[1, (2:3), 4] but neither appears very natural to me. I guess my problem is that ":" and "," have "fuzzy" priorities, and while everybody agrees that e.g. "*" binds tighter than "+", if you ask a few people in the street, or even computer programmers, you'd get confused answers. --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Sep 14 11:12:30 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA14951 for matrix-sig-people; Thu, 14 Sep 1995 11:12:30 -0400 Received: from servrcolkr.cr.usgs.gov (servrcolkr.cr.usgs.gov [136.177.112.5]) by python.org (8.6.12/8.6.12) with ESMTP id LAA14947 for ; Thu, 14 Sep 1995 11:12:25 -0400 Received: from localhost (dsjfqvarsa.er.usgs.gov [130.11.51.73]) by servrcolkr.cr.usgs.gov (EMAIL 1.2.1) with ESMTP id PAA18008; Thu, 14 Sep 1995 15:13:06 GMT Message-Id: <199509141513.PAA18008@servrcolkr.cr.usgs.gov> To: Guido van Rossum cc: "James L Fulton, Hydrologist, Reston, VA " cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] A problem with slicing In-reply-to: <199509141455.KAA01555@monty> Date: Thu, 14 Sep 1995 11:13:03 -0400 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk On Thu, 14 Sep 1995 10:55:32 -0400 Guido van Rossum said: > > The only *minor* change proposed, which I could live without is to > > allow the "," to generate tuples inside of []s, just as it does > > outside of []. In fact, I view the current non-recognition of tuples > > in []s as an inconsistency. For example: > [...] > > so why isn't: > > > > a[1,2,3] > > > > equivalent to: > > > > a[(1,2,3)] > > Because there's also > > a[1:2] > > while there is no equivalent > > a(1:2) > > I could either tweak the priorities so that > > a[1,2:3,4] > > is parsed as > > a[(1,2) : (3,4)] Can't be, ":" wants integers. > or so that it is parsed as > > a[1, (2:3), 4] Can't be, (2:3) is not a valid expression, so it can't yield a valid element of the tuple. > but neither appears very natural to me. Good. :-) > I guess my problem is that ":" and "," have "fuzzy" priorities, and > while everybody agrees that e.g. "*" binds tighter than "+", if you > ask a few people in the street, or even computer programmers, you'd > get confused answers. But ":" only makes sense for sequences and "," only makes sense for mappings, so ":" and "," should never appear together in []s. I don't think this is a precedence issue. I see your point that ":" complicates things a bit. You not only have to recognize ",", but you have to make sure that it is not used in conjunction with ":". Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Sep 14 12:43:14 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA15313 for matrix-sig-people; Thu, 14 Sep 1995 12:43:14 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id MAA15305 for ; Thu, 14 Sep 1995 12:43:04 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id MAA15050 (8.6.11/IDA-1.6); Thu, 14 Sep 1995 12:42:08 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA12527; Thu, 14 Sep 1995 12:42:07 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA06980; Thu, 14 Sep 1995 12:42:21 -0400 Date: Thu, 14 Sep 1995 12:42:21 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509141642.MAA06980@cyclone.ERE.UMontreal.CA> To: guido@CNRI.Reston.VA.US CC: matrix-sig@python.org In-reply-to: <199509141227.IAA01106@monty> (message from Guido van Rossum on Thu, 14 Sep 1995 08:27:04 -0400) Subject: Re: [PYTHON MATRIX-SIG] Why I won't add complex numbers to the base language Sender: owner-matrix-sig@python.org Precedence: bulk 1. Mathematically, there is at least one difference: complex numbers can't be compared (e.g. z1 < z2 is undefined when either has a nonzero imaginary party). Python will have to define this operation anyhow, So that would lead to an exception. Nothing exceptional ;-), since there are other operations that produce exceptions (like division by zero). -- but it leads me to believe that there should be plenty of algorithms out there that wouldn't work if some of their parameters were complex numbers. (At least if the designer of the code didn't think about the possibility of complex numbers, the correctness of the code can't be guaranteed in general.) As long as real and complex are True, but the same problem occurs with complex numbers defined in packages. Since Python's variables and function parameters are not types, nothing prevents me from passing complex numbers to an algorithm not designed for them. 2. The available of GNU or other pseudo-free packages for transcendental functions on complex numbers is still a big restriction for Python's portability. Everything that is part of the *core* language has to be portable to every new platform. Since some of That should not be a problem. All codes for complex numbers that I am aware of handle complex-number arithmetic in terms of (portable) real arithmetic. An implementation of complex numbers in portable C is just as possible as in portable Python. another problem with GNU code that's only relevant for a certain group but which I care about anyway: people want to be able to embed Python (at least it's core) in commercial applications, and if there's a GNU licence attached to any part of that core, this is a problem. (Don't There are non-GNU complex libraries, and even writing a new one is only a small task. 3. The implementation of complex numbers as the only floating-point type in Python would add considerable complexity (no pun intended :-). I take that this should mean that if x is a real number (e.g. 0.0) and z is a complex number (e.g. sqrt(-1.0)), type(x) == type(z). This Indeed. means that the representation in memory of Python complex number objects either has to contain the imaginary part at all times, or it has to contain a flag telling whether there's an imaginary part or Maybe. I don't know much about the internals of Python, I am just a simple user... But I know that most APL implementation use different internal types, but don't make this distinction visible to the user. In fact, they don't even distinguish between integers and reals. APL has only two types: character and numeric, although numeric has four internal representations (bits, integers, reals, and complex numbers). Consider the alternative: write an extension module ("complex") that defines arithmetic on complex numbers (there are already examples of how to do this in Python; a C version is totally plausible) and another extensions module ("cmath") that defines transcendental functions on complex number. Now the price is only paid by those who That would indeed be a good solution (I come to like Python's module system more and more every day). I'll explore that when I find some time... I don't see any real problem, everything necessary for a convenient implementation should be there (i.e. coercion from realfloat to complex, such that real constants can be entered conveniently). It would be nice to have a more convenient notation for complex constants, but that is not really essential. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Sep 14 13:37:51 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA15610 for matrix-sig-people; Thu, 14 Sep 1995 13:37:51 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id NAA15605 for ; Thu, 14 Sep 1995 13:37:46 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id NAA16541 (8.6.11/IDA-1.6); Thu, 14 Sep 1995 13:37:31 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA24105; Thu, 14 Sep 1995 13:37:30 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA09446; Thu, 14 Sep 1995 13:37:44 -0400 Date: Thu, 14 Sep 1995 13:37:44 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509141737.NAA09446@cyclone.ERE.UMontreal.CA> To: guido@CNRI.Reston.VA.US CC: matrix-sig@python.org In-reply-to: <199509141244.IAA01172@monty> (message from Guido van Rossum on Thu, 14 Sep 1995 08:44:20 -0400) Subject: Re: [PYTHON MATRIX-SIG] Forget adding new operators to Python Sender: owner-matrix-sig@python.org Precedence: bulk The only thing I regret is not having added "**" as an exponentiation operator -- this may happen someday (contributions accepted!). Oh, and there's also agreement that operators like "+=", "*=" should eventually be added; though not "++" and "--". Great. I am waiting ;-) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Sep 14 14:21:15 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA15992 for matrix-sig-people; Thu, 14 Sep 1995 14:21:15 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id OAA15988 for ; Thu, 14 Sep 1995 14:21:11 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id OAA18142 (8.6.11/IDA-1.6); Thu, 14 Sep 1995 14:20:49 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA03723; Thu, 14 Sep 1995 14:20:48 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA11773; Thu, 14 Sep 1995 14:21:02 -0400 Date: Thu, 14 Sep 1995 14:21:02 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509141821.OAA11773@cyclone.ERE.UMontreal.CA> To: guido@CNRI.Reston.VA.US CC: matrix-sig@python.org In-reply-to: <199509141320.JAA01256@monty> (message from Guido van Rossum on Thu, 14 Sep 1995 09:20:28 -0400) Subject: Re: [PYTHON MATRIX-SIG] A problem with slicing Sender: owner-matrix-sig@python.org Precedence: bulk One way out of this is to adopt the syntax M[i, j] for simple indexing. This would require only a minor tweaking of the grammar I believe. This could be extended to support M[i1:i2, j] M[i1:i2, j1:j2] M[i, j1:j2] (and of course higher-dimensional equivalents). How about allowing each of the index expressions to be an array (of integers) itself? That gives an enormous flexibility (consider that the indexing array could be a very complicated expression!). Of course adopting such a change would completely ruin any possbility of using things like M[3, 4, 7] = [1, 10, 100] as roughly equivalent to M[3] = 1 M[4] = 10 M[7] = 100 but then again I'm not too fond of that anyway (as a matter of fact, I'd oppose it strongly). According to my proposal, that could be done as M[[3,4,7]] = [1, 10, 100] - Should M[i][j] still be equivalent to M[i, j]? Basically the question is whether M[i] should be allowed (and with what meaning) if M has more than one dimension. Maybe it is better not to allow it at all, as it creates some confusion due to the imperfect analogy of arrays with (nested) sequences. - Now we have multidimensional sequence types, should be have a multidimensional equivalent of len()? Some ideas: - len(a, i) would return a's length in dimension i; len(a, i) == len(a) - dim(a) (or rank(a)?) would return the number of dimensions - shape(a) would return a tuple giving a's dimensions, e.g. for a 3x4 matrix it would return (3, 4), and for a one-dimensional sequence such as a string or list, it would return a singleton tuple: (len(a),). How about having len() return the total number of elements? shape() would be necessary anyway, and your proposed len(a,i) would just be equivalent to shape(a)[i]. Similarly, dim(a) would be len(shape(a)). APL actually has only shape, which returns a one-dimensional array of the lengths along each axis, i.e. a zero-length vector in case of a scalar. Then dim(a) is just shape(shape(a)). - How about multidimensional map(), filter(), reduce()? map() is no problem, as you pointed out. Actually, if one adopts a J-style rank concept, it wouldn't even be necessary. filter() and reduce() should operate along a specified axis. filter() would thereby return an array of equal dimension, but shorter along one axis, and reduce() would return an array with the dimension reduced by one. - Multidimensional for loops? Or should for iterate over the first dimension only? Again, iteration over an arbitrary axis would be useful. Iteration over all elements could be handled by map(). ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Sep 14 14:35:00 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA16145 for matrix-sig-people; Thu, 14 Sep 1995 14:35:00 -0400 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id OAA16141 for ; Thu, 14 Sep 1995 14:34:57 -0400 Received: from nineveh.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA09007; Thu, 14 Sep 95 14:35:03 EDT Received: by nineveh.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA04493; Thu, 14 Sep 95 14:41:40 -0400 Message-Id: <9509141841.AA04493@nineveh.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Thu, 14 Sep 95 14:41:39 -0400 To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] A problem with slicing References: <199509141513.PAA18008@servrcolkr.cr.usgs.gov> Sender: owner-matrix-sig@python.org Precedence: bulk I'm eager to see what Guido has to add in his "(More later.)", because at the moment I'm slightly confused by the trend in this discussion. If the point is that the proposed slicing semantics are a bad idea because they'd require basic changes to the language, then I'd answer that no changes are in fact necessary. If instead he is offering to open up this portion of the language to change in the hopes of creating a more "consistent design", then I think there are some interesting possibilities. In my opinion, the slicing semantics for matrices as currently proposed seem reasonable and can be implemented with zero changes to the core python language by using getitem and setitem for mapping objects (I know this because I've implemented them within the matrix module that way). However, there is a good argument to be made saying that - a[:3, :4] is a lot more consistent with the current implementation of lists than - a[(range(3), range(4))] In addition, something like - a[3:, 4:] is a lot clearer than - a[(range(3,shape(a[0]), range(4, shape(a)[1]))] However, one thing that I really like about the proposed indexing scheme is that I can say a[((1,3,5), (2,4,6))] and get back the desired non-contiguous chunk of the array. This is occasionally very useful from an efficiency point of view, and unfortunately, efficiency is something that needs to be kept in mind for large numeric arrays. If I had my wish, I would change the syntax of python so that start:step:stop, or start:stop was a shorthand for creating a range object (with some reasonable way of specifying start or stop as defaults). This would not need to change the semantics of basic sequence indexing, as these could be handled as a special case. I think that ":" should obviously be of higher precedence than ",". There are no cases that I can think of where it would be a reasonable thing to have a tuple as one element in a range. With this change, and the I feel completely reasonable proposal that "," be allowed for tuple creation within an index, then multidimensional arrays could be made to behave in a manner completely consistent with lists. And this would require minimal changes to the run-time architecture of slicing and indexing. On a slightly different track: I've been playing with another technique for indexing a matrix, borrowed from matlab. I've implemented indexing matrices with a matrix of booleans (integers) that is the same size as the matrix being indexed (this only makes sense for a setvalue). This is trivially done using the mapping semantics, and combined with some matrix comparision operators I've found this quite useful. ie. a.gt(x) is a matrix less than operator. (I'd prefer to use "a < x", but the point of this is to explore what can be done without changing the core python language that all of us love so much). a = matrix([1,2,3,4,5,4,3,2,1]) a[a.gt(3)] = 3 print a --> [1,2,3,3,3,3,3,2,1] I don't think that this is a substitute for any of the indexing methods currently being discussed, but I want to make sure that all candidate indexing methods are brought up as early on in the discussion as possible in order to ultimately create a coherent (rather than haphazard) indexing system. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Sep 14 18:47:09 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA17414 for matrix-sig-people; Thu, 14 Sep 1995 18:47:09 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id SAA17410 for ; Thu, 14 Sep 1995 18:47:06 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id SAA26236 (8.6.11/IDA-1.6); Thu, 14 Sep 1995 18:46:43 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id SAA06103; Thu, 14 Sep 1995 18:46:42 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id SAA26506; Thu, 14 Sep 1995 18:46:58 -0400 Date: Thu, 14 Sep 1995 18:46:58 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509142246.SAA26506@cyclone.ERE.UMontreal.CA> To: jjh@mama-bear.lcs.mit.edu CC: matrix-sig@python.org In-reply-to: <9509141841.AA04493@nineveh.LCS.MIT.EDU> (message from James Hugunin on Thu, 14 Sep 95 14:41:39 -0400) Subject: Re: [PYTHON MATRIX-SIG] A problem with slicing Sender: owner-matrix-sig@python.org Precedence: bulk I've been playing with another technique for indexing a matrix, borrowed from matlab. I've implemented indexing matrices with a matrix of booleans (integers) that is the same size as the matrix being indexed (this only makes sense for a setvalue). This is trivially done using the mapping semantics, and combined with some matrix comparision operators I've found this quite useful. Indeed. But how does it work for higher-dimensional arrays? APL provides a similar functionality using an operator called "compress". Its boolean argument is always one-dimensional and it a works along a specified axis. If used on the right-hand side of an expression, it is simply filter(). In APL2 it may also be used on the left hand side of an assignment, as btw can all expressions that evaluate to a subset of elements of an array. This is an extremely powerful feature, but so complicated to implement that to my knowledge only IBM's mainframe version does it without restrictions. In summary, I like your proposal, adding that it ought to work along one (arbitrary) axis for higher-dimensional arrays. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Sep 15 11:29:23 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA19907 for matrix-sig-people; Fri, 15 Sep 1995 11:29:23 -0400 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id LAA19903 for ; Fri, 15 Sep 1995 11:29:18 -0400 Received: from nineveh.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA03305; Fri, 15 Sep 95 11:29:08 EDT Received: by nineveh.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA01177; Fri, 15 Sep 95 11:29:10 -0400 Message-Id: <9509151529.AA01177@nineveh.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Fri, 15 Sep 95 11:29:09 -0400 To: hinsenk@ere.umontreal.ca (Hinsen Konrad) Subject: Re: [PYTHON MATRIX-SIG] A problem with slicing Cc: matrix-sig@python.org References: <199509142246.SAA26506@cyclone.ERE.UMontreal.CA> Sender: owner-matrix-sig@python.org Precedence: bulk > I've been playing with another technique for indexing a matrix, borrowed > from matlab. I've implemented indexing matrices with a matrix of booleans > (integers) that is the same size as the matrix being indexed (this only > makes sense for a setvalue). This is trivially done using the mapping > semantics, and combined with some matrix comparision operators I've found > this quite useful. > > Indeed. But how does it work for higher-dimensional arrays? > > APL provides a similar functionality using an operator called > "compress". Its boolean argument is always one-dimensional and it a > works along a specified axis. If used on the right-hand side of an > expression, it is simply filter(). In APL2 it may also be used on > the left hand side of an assignment, as btw can all expressions > that evaluate to a subset of elements of an array. This is an > extremely powerful feature, but so complicated to implement that > to my knowledge only IBM's mainframe version does it without > restrictions. My initial idea for higher-dimensional arrays was to use a matrix of booleans that was the size of the entire higher-d array being indexed, this fits naturally with the outputs from matrix comparision operators. ie. a = [[1,2,3], [4,5,6]] --> a.gt(2) == [[0,0,1], [1,1,1]] a[a.gt(2)] = 99 --> a == [[1,2,99], [99,99,99]] However, I agree that it should be possible to rationally apply this to a subset of the matrix. Something like: a = [[1,2,3], [4,5,6]] --> sum(a).gt(7) == [0,1] a[sum(a).gt(7), :] = 99 --> a == [[1,2,3], [99,99,99]] This example suggests that it should be possible to mix these boolean indices with range indices, and with single integer indices. While this all sounds reasonable (and extremely powerful) to me, I have to confess that a few of my feeping creaturism warning lights go off when I sit back and look at it. I'd love to hear more impressions on this, pro and con. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Sep 15 12:29:35 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA20234 for matrix-sig-people; Fri, 15 Sep 1995 12:29:35 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id MAA20230 for ; Fri, 15 Sep 1995 12:29:24 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id MAA15149 (8.6.11/IDA-1.6); Fri, 15 Sep 1995 12:28:23 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA05616; Fri, 15 Sep 1995 12:28:22 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA13937; Fri, 15 Sep 1995 12:28:42 -0400 Date: Fri, 15 Sep 1995 12:28:42 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509151628.MAA13937@cyclone.ERE.UMontreal.CA> To: jjh@mama-bear.lcs.mit.edu CC: matrix-sig@python.org In-reply-to: <9509151529.AA01177@nineveh.LCS.MIT.EDU> (message from James Hugunin on Fri, 15 Sep 95 11:29:09 -0400) Subject: Re: [PYTHON MATRIX-SIG] A problem with slicing Sender: owner-matrix-sig@python.org Precedence: bulk My initial idea for higher-dimensional arrays was to use a matrix of booleans that was the size of the entire higher-d array being indexed, this fits naturally with the outputs from matrix comparision operators. ie. a = [[1,2,3], [4,5,6]] --> a.gt(2) == [[0,0,1], [1,1,1]] a[a.gt(2)] = 99 --> a == [[1,2,99], [99,99,99]] That's nice on the left hand side of an assignment, but what is the value of a[a.gt(2)] in an expression? It can't be an array! If all you want is some form of selective assignment, that can be done with mapping, although you have to type a bit more. You could also achieve the above result (admittedly less efficiently) with a = (not a > 2)*a + (a > 2)*99. Therefore I am not so sure whether your proposed feature is that important, except if it were very common. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Sep 15 13:26:07 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA20422 for matrix-sig-people; Fri, 15 Sep 1995 13:26:07 -0400 Received: from retro.jhuapl.edu (retro.jhuapl.edu [128.244.146.239]) by python.org (8.6.12/8.6.12) with ESMTP id NAA20418 for ; Fri, 15 Sep 1995 13:25:59 -0400 Message-Id: <199509151725.NAA20418@python.org> Received: by retro.jhuapl.edu (1.37.109.16/16.2) id AA255275977; Fri, 15 Sep 1995 13:26:17 -0400 Date: Fri, 15 Sep 1995 13:26:17 -0400 From: Chris Chase S1A Cc: matrix-sig@python.org To: hinsenk@ere.umontreal.ca (Hinsen Konrad) Subject: Re: [PYTHON MATRIX-SIG] A problem with slicing In-Reply-To: <199509151628.MAA13937@cyclone.ERE.UMontreal.CA> References: <9509151529.AA01177@nineveh.LCS.MIT.EDU> <199509151628.MAA13937@cyclone.ERE.UMontreal.CA> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk Hinsen> a = [[1,2,3], [4,5,6]] --> a.gt(2) == [[0,0,1], [1,1,1]] Hinsen> a[a.gt(2)] = 99 --> a == [[1,2,99], [99,99,99]] Hinsen> That's nice on the left hand side of an assignment, but what Hinsen> is the value of a[a.gt(2)] in an expression? It can't be an Hinsen> array! It be an array if you define a selection type indexing, e.g., a.select(a.gt(2)) where select is a method that whose argument has the same dimensions as "a" and returns elements of "a" that correspond to "true" elements of a.gt(2). The language Octave allows for this type of selection indexing. There are several types of indexing schemes that I have seen for multi-demensional arrays. Without additions to the syntax, only one could be used with "[...]" and the others would have to be explicit method calls. I have a number of possible suggestions about different types of array indexing which I will post later. Chris Chase ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Sep 15 14:25:58 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA20703 for matrix-sig-people; Fri, 15 Sep 1995 14:25:58 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id OAA20699 for ; Fri, 15 Sep 1995 14:25:51 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id OAA19270 (8.6.11/IDA-1.6); Fri, 15 Sep 1995 14:24:55 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA29081; Fri, 15 Sep 1995 14:24:53 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA21099; Fri, 15 Sep 1995 14:25:10 -0400 Date: Fri, 15 Sep 1995 14:25:10 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509151825.OAA21099@cyclone.ERE.UMontreal.CA> To: chris.chase@jhuapl.edu CC: matrix-sig@python.org In-reply-to: <199509151725.NAA20418@python.org> (message from Chris Chase S1A on Fri, 15 Sep 1995 13:26:17 -0400) Subject: Re: [PYTHON MATRIX-SIG] A problem with slicing Sender: owner-matrix-sig@python.org Precedence: bulk Hinsen> That's nice on the left hand side of an assignment, but what Hinsen> is the value of a[a.gt(2)] in an expression? It can't be an Hinsen> array! It be an array if you define a selection type indexing, e.g., a.select(a.gt(2)) where select is a method that whose argument has the same dimensions as "a" and returns elements of "a" that correspond to "true" elements of a.gt(2). But *how* does it return the elements that correspong to "true"? Specifically, what is the shape of the array returned? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Sep 19 20:29:06 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id UAA08091 for matrix-sig-people; Tue, 19 Sep 1995 20:29:06 -0400 Received: from big.fishnet.net (root@big.fishnet.net [205.216.133.3]) by python.org (8.6.12/8.6.12) with ESMTP id UAA08087 for ; Tue, 19 Sep 1995 20:29:02 -0400 Received: from port50.fishnet.net (port50.fishnet.net [205.216.133.250]) by big.fishnet.net (8.6.12/8.6.9) with SMTP id RAA19637 for ; Tue, 19 Sep 1995 17:30:03 GMT Message-Id: <199509191730.RAA19637@big.fishnet.net> X-Sender: graham@fishnet.net X-Mailer: Windows Eudora Pro Version 2.1.2 Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Tue, 19 Sep 1995 17:31:31 -0700 To: matrix-sig@python.org From: Graham Hughes Subject: [PYTHON MATRIX-SIG] Assorted remarks Sender: owner-matrix-sig@python.org Precedence: bulk Another compendium of responses to various messages. --- Guido van Rossum --- >- Allowing M[i, j] for (multidimensional) sequence types would also >meaning that D[i, j] would be equivalent to D[(i, j)] for >dictionaries. Sounds good to me. Alternatively, we could interpret D[i,j] as D[i][j]... After all, basically that's what M[i,j] is. Might warrant consideration. >- Should M[i][j] still be equivalent to M[i, j]? Probably. While slicing would be difficult as your earlier remarks point out, it seems that allowing this will allow the use of sequences of sequences. Actually, I'm not entirely convinced that M[i:j][k] is *not* impossible; give me a week or so to decide on this, and I may come up with something that doesn't require any changing of the core language... >- Now we have multidimensional sequence types, should be have a >multidimensional equivalent of len()? Some ideas: Actually, I feel that for a multidimensional sequence, len() should behave the same way it always had. Allowing for greater dimensions will give greater flexibility, yes, but it may also break `older' functions that assume that what's being passed to them is a 2D array. This is mainly because len() is defined to return an integer, sadly, and not one of our magic sequences; if it did return the magic sequence, we would be able to totally ignore how many dimensions we have. > - len(a, i) would return a's length in dimension i; len(a, i) == len(a) As to extending len(), I don't think it's even necessary. Just do len(a[1]). Short, simple, requires nothing out of the ordinary, *and* will even return a 1 or something like that *if* [] returns an array of the proper rank (a 3D will return a rank 2 array, 2 will return 1, etc.). This will not in fact cause problems, because all operations on arrays should be defined to be 'elementwise' by default. This means we can pass a 3D array to something expecting a matrix, and it won't make a difference. APL/J ranking does this automatically, and given a little time I can probably work up an example matrix that will do that too. > - dim(a) (or rank(a)?) would return the number of dimensions If we have shape(), that's not terribly useful either. Just do len(shape(a)). > - shape(a) would return a tuple giving a's dimensions, e.g. for a > 3x4 matrix it would return (3, 4), and for a one-dimensional > sequence such as a string or list, it would return a singleton > tuple: (len(a),). Why a tuple, and not an array of rank 1? The array can be manipulated more effectively than the tuple... >- How about multidimensional map(), filter(), reduce()? Not needed, really; see following remarks. > - map() seems easy (except there seems to be no easy way to specify > the rank of the operator): it returns a similarly shaped > multidimensional object whose elements are the function results for > the corresponding elements of the input matrix If we use APL/J ranks, this is mostly taken care of automagically. The function will request what it wants from the matrix, and the matrix can do whatever it needs to with that. Ah, but let's suppose you want the dimensions in a different order; let's suppose you've got [ [ [1,2], [2,3] ] , [ [3,4], [4,5] ] ] (2x2x2 array). Normally, map would invert over [ [1,2], [2,3] ] and [ [3,4], [4,5] ]. But suppose you want it to behave differently; you want to 'rotate' the matrix. I argue that this is an important enough operation that it should be divorced from map, reduce, and filter, and brought into its own (notably unlike APL, BTW; APL had a special modifier on *every* sequence operation to do this. That is inefficient :). > - filter() is problematic since the columns won't be of the same > length filter() should work just like map(); i.e. just like it already does. The function that filter calls should know exactly what it wants out of the array passed; there's no reason for filter() to have this information. > - reduce()??? -- someone who knows APL tell me what it should mean Oh, ok; this follows right along the lines of the above stuff; it should work like it always did. Reducing a 2d array will simply have a grouping of 1d arrays being passed to the function. Since the function knows what it wants (sense a common theme here?), the overall effect will be transparent. I really *need* to get that sample matrix written up to illustrate this... >- Multidimensional for loops? Or should for iterate over the first >dimension only? Same as above, i.e. first dimension only. If you want a different dimension, rotate() the matrix. BTW, notice that if (very strong if) I can get slicing to work properly, *none* of this would require altering the language. Even if I can't, they shouldn't require any alterations beyond the original [i,j] change. Secret bonus; we can use [i,j] to signify non-continguous slices. While you said you're not fond of this earlier (not quoted), and I agree that the example given was a bit strange, this can give a great deal of additional power to the matrix class. More on this later. --- forrest@rose.rsoc.rockwell.com (Dave Forrest) --- >This is a strong reason to provide two different interfaces - matrix >and array. You can implement them both with the same code, but just >provide a simple interface to matrix that restricts itself to two >dimensions and does matrix multiplication with operator *. Array then >acts "element-wise" and generalizes to any dimension,etc. Not really. I'll reiterate what I've said before, that you don't need it because all your functions will work fine with whatever dimension thingy we pass it, with a bumper; *if* we do two different things, and you discover that your original code needs to be modified somehow, or someone gets confused and passes mixed arguments (matrix to array, or vice versa), all of a sudden everything gets thoroughly confused. This is not a problem in standard Python, since you can't add different types, but it might crop up here, particularly if we write most of the interface in Python (which is entirely possible, and possibly desirable). Basically, my main argument against splitting it up is that it adds unneeded complexity for a relatively small bonus (just write a matrix_multiply() function that expects rank 2 arrays, and everything works peachy), and that it would segregate the two entirely too much. For example, I would have no way of taking my array and doing parallel matrix multiplications on it (i.e. multiplying three sets of matrices at the same time), because while I can do the parallelism in 'array' class and the matrix multiplication in the 'matrix' class, I would have no way to fold them together. As a result, I would likely write a matrix_multiply() for the general matrix, and _entirely_ sidestep the submatrix class. If you ever need to multiply several matricies together (and you probably do) this parallelism is much more efficient (because you spend more time in C code multiplying than in Python figuring out which one to do next) than swapping matrices around. Examine your code; you probably do something like (dropping into C for a while) matrix m[4]; int i; for (i = 0; i < 4; i++) load_matrix(m[i]); matrix_multiply(m[0],m[1]); matrix_multiply(m[2],m[3]); Notice that you're doing two multiplications. In C, there is no reason to do parallelism (for efficiency sake) because you're not kicking back up into an interpreter every time you multiply something. The parallel matrix would work exactly the same way (would look a little different, I think, but not a real problem), but would only kick in the interpreter *once*. This, for 5, 10, 200 matrices is a *real* timesaver. This point is why APL is so fast, even though it is interpreted; it spends most of its time in C, not an interpreter. --- Graham Graham Hughes Home page http://www.fishnet.net/~graham/ ``I think it would be a good idea.'' -- Mahatma Ghandi, when asked what he thought of Western civilization finger for PGP public key. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 20 01:16:15 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id BAA08901 for matrix-sig-people; Wed, 20 Sep 1995 01:16:15 -0400 Received: from big.fishnet.net (root@big.fishnet.net [205.216.133.3]) by python.org (8.6.12/8.6.12) with ESMTP id BAA08893 for ; Wed, 20 Sep 1995 01:16:09 -0400 Received: from port27.fishnet.net (port27.fishnet.net [205.216.133.227]) by big.fishnet.net (8.6.12/8.6.9) with SMTP id WAA29411 for ; Tue, 19 Sep 1995 22:17:13 GMT Message-Id: <199509192217.WAA29411@big.fishnet.net> X-Sender: graham@fishnet.net X-Mailer: Windows Eudora Pro Version 2.1.2 Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Tue, 19 Sep 1995 22:18:43 -0700 To: matrix-sig@python.org From: Graham Hughes Subject: [PYTHON MATRIX-SIG] Slicing Sender: owner-matrix-sig@python.org Precedence: bulk I think I figured out how to get the slicing without any modification of the existing Python base to work. Earlier it was said that m[i:j][k] wouldn't work because of the precedence rules. I think one way of avoiding these problems is to look at the slicing this way: Assume for the moment that sequences of sequences are stored by row, i.e. like C. To get the slicing to work properly, we have to slice by *columns*. As an example, suppose we have [[1,2,3],[2,3,4]], or 1 2 3 2 3 4 if we accept the original premise. The matrix class will store it this way internally. However, every interaction with the user *must* make the matrix look like this: 1 2 2 3 3 4 Given this, slicing is relatively easy, and [:] will return a transpose of the internal storage. So m[0:1] will return in original form [[1],[2],[3]] or 1 2 3 This works great for slices. However, assigning to individual elements is a tad tricker... Note that a special case for single dimension arrays will simply do standard slicing, as the effect is the same. Graham Graham Hughes Home page http://www.fishnet.net/~graham/ ``I think it would be a good idea.'' -- Mahatma Ghandi, when asked what he thought of Western civilization finger for PGP public key. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 20 09:18:49 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA09918 for matrix-sig-people; Wed, 20 Sep 1995 09:18:49 -0400 Received: from servrcolkr.cr.usgs.gov (servrcolkr.cr.usgs.gov [136.177.112.5]) by python.org (8.6.12/8.6.12) with ESMTP id JAA09914 for ; Wed, 20 Sep 1995 09:18:46 -0400 Received: from localhost (dsjfqvarsa.er.usgs.gov [130.11.51.73]) by servrcolkr.cr.usgs.gov (EMAIL 1.2.1) with ESMTP id NAA29865; Wed, 20 Sep 1995 13:18:11 GMT Message-Id: <199509201318.NAA29865@servrcolkr.cr.usgs.gov> To: Graham Hughes cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Slicing In-reply-to: <199509192217.WAA29411@big.fishnet.net> Date: Wed, 20 Sep 1995 09:18:08 -0400 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk On Tue, 19 Sep 1995 22:18:43 -0700 Graham Hughes said: > I think I figured out how to get the slicing without any modification of the > existing Python base to work. The original proposal *already* says how to get slicing without any modifications to the core language. > Earlier it was said that m[i:j][k] wouldn't work because of the precedence > rules. I think one way of avoiding these problems is to look at the slicing > this way: > > Assume for the moment that sequences of sequences are stored by row, i.e. > like C. See my earlier post. Matrices are stored by sub-matrix. Storage by "rows" or by "columns" is a question of interpretation. Under my system of interpretation, matrices are stored by column (in Fortran, C, and the proposed Python extension). > To get the slicing to work properly, we have to slice by *columns*. > As an example, suppose we have [[1,2,3],[2,3,4]], or > > 1 2 3 > > 2 3 4 What does this mean? > if we accept the original premise. The matrix class will store it this way > internally. However, every interaction with the user *must* make the matrix > look like this: > > 1 2 > > 2 3 > > 3 4 Do you mean like: [[1,2],[2,3],[3,4]]? > Given this, slicing is relatively easy, How? > and [:] will return a transpose of > the internal storage. Why? This would be inconsistent with other sequence types. > So m[0:1] will return in original form [[1],[2],[3]] or > > 1 > > 2 > > 3 What has this gained? (Perhaps an example with greater than two rows and columns would be better?) Let's be clear about the goal. Goven a matrix that looks (by rows or by columns, take your pick) like this: 0 10 20 30 1 11 21 31 2 12 22 32 3 13 23 33 one wants to be able to access a matrix that looks like this: 11 21 12 22 Some even want to be able to access a matrix that looks like this: 1 21 3 23 or even this: 0 30 3 33 and so on. And, of course, this needs to generalize to higher dimensions. Also, modification to this access should be reflected in the matrix being accessed. I don't see how switching indexes solves this problem. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Sep 26 21:40:32 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id VAA06839 for matrix-sig-people; Tue, 26 Sep 1995 21:40:32 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id VAA06835 for ; Tue, 26 Sep 1995 21:40:27 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id VAA14168 (8.6.11/IDA-1.6 for ); Tue, 26 Sep 1995 21:37:37 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id VAA03450; Tue, 26 Sep 1995 21:37:37 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id VAA15833; Tue, 26 Sep 1995 21:38:49 -0400 Date: Tue, 26 Sep 1995 21:38:49 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509270138.VAA15833@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] J-style arrays in Python Sender: owner-matrix-sig@python.org Precedence: bulk To bring back some life into this discussion group, I'll distribute a Python implementation of J-like arrays, to give those unfamiliar with J a chance to become familiar with its array and rank system. And of course it is also usable, as long as the arrays don't become too big. This is a first test version; many important functions are still missing, others (like output formatting) need substantial improvement, and many could probably be made faster with some thought. To make (my) life simpler, I tried to stick to J's conventions as much as possible (but hopefully without violating Python traditions). I am not claiming that this is semantically the best way to implement arrays, but it is a start. Some general remarks: - Since my implementation uses nested lists to represent arrays, the elements can be arbitrary objects. - Like arrays in J, my arrays are immutable, i.e. there is no provision for changing individual elements. The reason for making arrays immutable in J was that J is half-way to being a functional language (it is not a pure functional language, but many substantial problems can easily be solved in a functional way). I have never missed element assignment, but probably there are some good applications... - All functions are implemented as external functions, not as methods. The main reason is that at first I could not think of a way to implement methods with variable rank, although later I figured out how to do this (in the same way as I implemented reduction). I'll send two Python files; the first is the Array module itself, and the second a kind of simple tutorial. Let me know if anything is unclear. And let me know what you think of the whole implementation! ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Sep 26 21:40:56 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id VAA06845 for matrix-sig-people; Tue, 26 Sep 1995 21:40:56 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id VAA06841 for ; Tue, 26 Sep 1995 21:40:53 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id VAA14175 (8.6.11/IDA-1.6 for ); Tue, 26 Sep 1995 21:37:59 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id VAA03454; Tue, 26 Sep 1995 21:37:58 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id VAA15839; Tue, 26 Sep 1995 21:39:11 -0400 Date: Tue, 26 Sep 1995 21:39:11 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509270139.VAA15839@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Array.py Sender: owner-matrix-sig@python.org Precedence: bulk # J-style array class # Arrays are represented by a scalar or a list, possibly containing # other lists in case of higher-rank arrays. Array rank is limited # only by the user's patience. # Send comments to Konrad Hinsen import math, string ###################################################################### # Error type ArrayError = 'ArrayError' ###################################################################### # Various functions that do the real work. Classes follow. # Construct string representation of array def _toString(data, dimension): s = '' if dimension == 0: s = s + `data` elif dimension == 1: for e in data: s = s + `e` + ' ' else: separator = (dimension-1)*'\n' for e in data: s = s + _toString(e,dimension-1) + separator return string.strip(s) # Find the shape of an array and check for consistency def _shape(data): if type(data) == type([]): if data and type(data[0]) == type([]): shapes = map(lambda x:_shape(x),data) for i in range(1,len(shapes)): if shapes[i] != shapes[0]: raise ArrayError, 'Inconsistent shapes' shape = [len(data)] shape = shape + shapes[0] return shape else: return [len(data)] else: return [] # Construct a one-dimensional list of all array elements def __ravel(data): if type(data) == type([]): if type(data[0]) == type([]): return reduce(lambda a,b: a+b, map(lambda x: __ravel(x), data)) else: return data else: return [data] def _ravel(array): return Array(__ravel(array._data), reduce(lambda a,b: a*b, array._shape, 1)) # Reshape an array def _reshape(array, shape): array = ravel(array) if len(shape._data) == 0: return take(array,0) else: array = array._data shape = shape._data n = reduce(lambda a,b: a*b, shape) if n > len(array): nr = (n+len(array)-1)/len(array) array = (nr*array)[:n] elif n < len(array): array = array[:n] for i in range(len(shape)-1, 0, -1): d = shape[i] n = n/d for j in range(n): array[j:j+d] = [array[j:j+d]] return Array(array,shape) # Map a function on the first dimensions of an array def _extract(a, index, dimension): if len(a[1]) < dimension: return a else: return (a[0][index], a[1][1:], a[2]) def _map(function, arglist, max_frame, scalar_flag): result = [] if len(max_frame) == 0: if scalar_flag: result = apply(function,tuple(map(lambda a: a[0], arglist))) else: result = apply(function,tuple(map(lambda a: Array(a[0],a[2]), arglist)))._data else: for index in range(max_frame[0]): result.append(_map(function, map(lambda a,i=index,d=len(max_frame): _extract(a,i,d), arglist), max_frame[1:], scalar_flag)) return result # Reduce an array with a given binary function def _reduce(function, array): function = function[0] array = array[0] if len(array._shape) == 0: return array elif array._shape[0] == 0: return reshape(function._neutral, array._shape[1:]) else: result = Array(array._data[0], array._shape[1:]) for i in range(1,array._shape[0]): result = function(result, Array(array._data[i], array._shape[1:])) return result # Find the higher of two ranks def _maxrank(a,b): if a == None or b == None: return None else: return max(a,b) ###################################################################### # Array class definition class Array: def __init__(self, scalar_or_list, shape = None): self._data = scalar_or_list if shape == None: self._shape = _shape(self._data) else: self._shape = shape def __str__(self): return _toString(self._data,len(self._shape)) __repr__ = __str__ def __len__(self): if type(self._data) == type([]): return len(self._data) else: return 1 def __getitem__(self, index): return take(self, index) def __getslice__(self, i, j): return take(self, range(i,j)) def __add__(self, other): return sum(self, other) __radd__ = __add__ def __sub__(self, other): return difference(self, other) def __rsub__(self, other): return difference(other, self) def __mul__(self, other): return product(self, other) __rmul__ = __mul__ def __div__(self, other): return quotient(self, other) def __rdiv__(self, other): return quotient(other, self) def __pow__(self,other): return power(self, other) def __rpow__(self,other): return power(other, self) def __neg__(self): return 0-self # Check for arrayness def isArray(x): return hasattr(x,'_shape') ###################################################################### # Array function class class ArrayFunction: def __init__(self, function, ranks, intrinsic_ranks=None): self._function = function if isArray(ranks): self._ranks = ranks._data elif type(ranks) == type([]): self._ranks = ranks else: self._ranks = [ranks] if intrinsic_ranks == None: self._intrinsic_ranks = self._ranks else: self._intrinsic_ranks = intrinsic_ranks if len(self._ranks) == 1: self._ranks = len(self._intrinsic_ranks)*self._ranks def __call__(self, *args): if len(self._ranks) != len(args): raise ArrayError, 'Wrong number of arguments for an array function' arglist = [] framelist = [] shapelist = [] for i in range(len(args)): if isArray(args[i]): arglist.append(args[i]) else: arglist.append(Array(args[i])) shape = arglist[i]._shape rank = self._ranks[i] intrinsic_rank = self._intrinsic_ranks[i] if rank == None: cell = 0 elif rank < 0: cell = min(-rank,len(shape)) else: cell = max(0,len(shape)-rank) if intrinsic_rank != None: cell = max(cell,len(shape)-intrinsic_rank) framelist.append(shape[:cell]) shapelist.append(shape[cell:]) max_frame = [] for frame in framelist: if len(frame) > len(max_frame): max_frame = frame for i in range(len(framelist)): if framelist[i] != max_frame[len(max_frame)-len(framelist[i]):]: raise ArrayError, 'Incompatible arguments' scalar_function = reduce(lambda a,b:_maxrank(a,b), self._intrinsic_ranks) == 0 return Array(_map(self._function, map(lambda a,b,c: (a._data,b,c), arglist, framelist, shapelist), max_frame, scalar_function)) def __getitem__(self, ranks): return ArrayFunction(self._function,ranks,self._intrinsic_ranks) class BinaryArrayFunction(ArrayFunction): def __init__(self, function, neutral_element, ranks, intrinsic_ranks=None): ArrayFunction.__init__(self, function, ranks, intrinsic_ranks) self._neutral = neutral_element self.over = ArrayFunction(ArrayOperator(_reduce, [self]), [None]) def __getitem__(self, ranks): return BinaryArrayFunction(self._function, self._neutral, ranks, self._intrinsic_ranks) class ArrayOperator: def __init__(self, operator, function_list): self._operator = operator self._functions = function_list def __call__(self, *args): return apply(self._operator, (self._functions, args)) ###################################################################### # Array functions # Structural functions shape = ArrayFunction(lambda a: Array(a._shape), [None]) reshape = ArrayFunction(_reshape, [None, 1]) ravel = ArrayFunction(_ravel, [None]) take = ArrayFunction(lambda a,i: Array(a._data[i._data], a._shape[1:]), [None, 0]) # Elementwise binary functions _sum = ArrayFunction(lambda a,b: a+b, [0, 0]) _difference = ArrayFunction(lambda a,b: a-b, [0, 0]) _product = ArrayFunction(lambda a,b: a*b, [0, 0]) _quotient = ArrayFunction(lambda a,b: a/b, [0, 0]) _power = ArrayFunction(pow, [0, 0]) sum = BinaryArrayFunction(_sum, 0, [None, None]) difference = BinaryArrayFunction(_difference, 0, [None, None]) product = BinaryArrayFunction(_product, 1, [None, None]) quotient = BinaryArrayFunction(_quotient, 1, [None, None]) power = BinaryArrayFunction(_power, 1, [None, None]) # Scalar functions of one variable sqrt = ArrayFunction(math.sqrt, [0]) exp = ArrayFunction(math.exp, [0]) log = ArrayFunction(math.log, [0]) log10 = ArrayFunction(math.log10, [0]) sin = ArrayFunction(math.sin, [0]) cos = ArrayFunction(math.cos, [0]) tan = ArrayFunction(math.tan, [0]) asin = ArrayFunction(math.asin, [0]) acos = ArrayFunction(math.acos, [0]) atan = ArrayFunction(math.atan, [0]) sinh = ArrayFunction(math.sinh, [0]) cosh = ArrayFunction(math.cosh, [0]) tanh = ArrayFunction(math.tanh, [0]) floor = ArrayFunction(math.floor, [0]) ceil = ArrayFunction(math.ceil, [0]) ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Sep 26 21:41:19 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id VAA06862 for matrix-sig-people; Tue, 26 Sep 1995 21:41:19 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id VAA06857 for ; Tue, 26 Sep 1995 21:41:14 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id VAA14191 (8.6.11/IDA-1.6 for ); Tue, 26 Sep 1995 21:38:23 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id VAA03460; Tue, 26 Sep 1995 21:38:22 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id VAA15853; Tue, 26 Sep 1995 21:39:35 -0400 Date: Tue, 26 Sep 1995 21:39:35 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509270139.VAA15853@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] ArrayTutorial.py Sender: owner-matrix-sig@python.org Precedence: bulk # This file illustrates the use of the the Array class. # # Send comments to Konrad Hinsen # from Array import * ###################################################################### # Arrays are constructed from (nested) lists: a = Array(range(10)) b = Array([ [2,3,7], [9,8,2] ]) c = Array([ [ [4,5,6], [0,4,5] ], [ [1,6,5], [8,5,2] ] ]) # Scalars make arrays of rank 0: s = Array(42) # Array elements can be anything: text_array = Array(['Hello', 'world']) # Arrays can be printed: print 'a:\n', a print 'b:\n', b print 'c:\n', c print 's:\n', s # shape() returns an array containing the dimensions of another array: print 'shape(a):\n', shape(a) print 'shape(b):\n', shape(b) print 'shape(c):\n', shape(c) print 'shape(s):\n', shape(s) # Scalar functions act on each element of an array: print 'sqrt(b):\n',sqrt(b) # Binary operators likewise work elementwise: print 'c+c:\n',c+c # But you can also add a scalar: print 'c+s:\n',c+s # To understand the general rule for combining arrays of different # shapes in a function, we need some more technical terms: # The length of the shape vector of an array is called its rank. # The elements of an array along the first axis are called items. # The items of an array of rank N have rank N-1. More generally, # the shape vector is divided into frames and cells. Frames and # cells are not properties of an array, but describe ways of looking # at an array. For example, a rank-3 array can be regarded as # as just that - a single rank-3 array. It can also be regarded # as a rank-1 frame of rank-2 cells, or as a rank-2 frame of # rank-1 cells. Or even as a rank-3 array of rank-0 cells, i.e. # scalar cells. # # When two arrays are to be added (or multiplied, or...), their # shapes need not equal, but the lower-rank array must match # an equal-rank cell of the higher-rank array. The lower-rank # array will then be combined with each corresponding cell, and # the result will have the shape of the higher-rank array. print 'b+c:\n',b+c # The addition of a scalar is just a special case: it has rank 0 # and therefore matches the rank-0 cells (scalar elements) of any array! print 'b+s:\n',b+s # All operators are also available as normal binary function, # e.g. addition can be written as print 'sum(a,s):\n',sum(a,s) # You'll need this form to perform reductions, e.g. a sum # of all items of an array: print 'sum.over(a):\n',sum.over(a) # Let's do it with a higher-rank array: print 'product.over(b):\n',product.over(b) # But how do you get the product along the second axis # of b? Easy: print 'product.over[1](b):\n',product.over[1](b) # The [1] after the function name product.over modifies # the functions rank. Function ranks are related to array # ranks, in that a function of rank N operates on the # N-cells of its argument. If the argument has a higher # rank, the function is applied to each N-cell and the # result is combined with the frame of the argument. # In the last example, product.over will be # called for each of the 1-cells of b, returning a # rank-0 result for each, and the results will be # collected in the 1-frame of b, producing as a net # result an array of rank 1. # # All functions have ranks; if no rank is explicitly # given, the default rank will be used. The default # rank of all reductions is 'unbounded', which means # that the function will operate on the highest-level # cells possible. Many functions have unbounded rank, # for example shape(): print 'shape(c):\n',shape(c) # But of course you can modify the rank of shape(): print 'shape[1](c):\n',shape[1](c) print 'shape[2](c):\n',shape[2](c) # Functions with more than one argument can have a different # rank for each. The rank is applied to each argument, # defining its frames and cells for this purpose. The frames # must match in the same way as indicated above for # addition of two arrays. The function is then applied # to the cells, and if appropriate, the same matching # procedure is applied once again. This may seem confusing # at first, but it is really just the application of a # single principle everywhere! # # For example, let's take a (our rank-1 array) and add # b (our rank-2 array) to each of a's 0-cells: print 'sum[[0,None]](a,b):\n',sum[[0,None]](a,b) # 'None' stands for 'unbounded'. Since the rank of sum is # 0 for its first argument, a is divided into 1-frames # and 0-cells. For b the rank is unbounded, so it is # treated as a 0-frame with 2-cells. b's 0-frame matches # a's 1-frame (a 0-frame matches everything!), and # the result gets the 1-frame. The cells of the result # are sums of a rank-0 array (element of a) and a rank-2 # array (b), i.e. rank-2 arrays by the matching rules # given above. So the net total is a rank-3 array, # as you have seen. # From now on we will specify the default rank of each function. # It should be noted that specifying a higher rank than the # default rank has no effect on the function's behaviour. Only # lower ranks make a difference. # # All the scalar mathematical functions (sqrt, sin, ...) have # rank 0. The binary arithmetic functions (sum, product, ...) # have unbounded rank for both argument. For structural functions # (i.e. those that modify an array's shape rather than its # elements), the rank varies. As we have seen, shape() is # unbounded. The other structural functions are yet to be # introduced: # ravel() produces a rank-1 array containing all elements # of its argument. It has unbounded rank: print 'ravel(c):\n',ravel(c) # reshape() allows you to change the shape of an array. # It first obtains a rank-1 list of elements of its first # argument (like ravel()) and then reassembles the # elements into an array with the shape given by the # second argument: print 'reshape(a,[2,2]):\n',reshape(a,[2,2]) print 'reshape(b,[10]):\n',reshape(b,[10]) # As you have seen in the second case, reshape() reuses # the elements of its arguments all over if they get # exhausted. # You may have noticed that in some examples, a nested list # has been used instead of an array as a function argument. # In general, all array functions will convert its arguments # to arrays if necessary. # Now we need a way to select parts of an array. You can # use standard indexing and slicing to obtain items # (i.e. N-1 cells for a rank-N array): print 'c[1]:\n',c[1] print 'a[3:7]:\n',a[3:7] # You can also specify an array as the index and obtain an # array of the corresponding items: print 'a[[2,6,1]]:\n',a[[2,6,1]] # The function take() does exactly the same: print 'take(c,1):\n',take(c,1) # You will have to use take() if you want to modify its # ranks, which are [None,0] by default. There is little # point in changing the second rank (it wouldn't make # any difference), but changing the first rank lets you # select along other dimensions than the first: print 'take[1](c,0):\n',take[1](c,0) print 'take[2](c,1):\n',take[2](c,1) # Isn't there something wrong here? take() takes # two arguments and therefore needs two ranks. But for # convenience, only one rank must be given if all ranks # are to be the same. # And that's the end of this introduction. Stay tuned for # updates of the array package that will provide some # important still missing in this version. In the meantime, # play round with what there is to get a feeling for # how things work. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 27 05:39:14 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id FAA08092 for matrix-sig-people; Wed, 27 Sep 1995 05:39:14 -0400 Received: from qvarsx.er.usgs.GOV (qvarsx.er.usgs.gov [130.11.51.82]) by python.org (8.6.12/8.6.12) with ESMTP id FAA08088 for ; Wed, 27 Sep 1995 05:39:09 -0400 Received: from localhost (dsjfqvarsa [130.11.51.73]) by qvarsx.er.usgs.GOV (EMAIL 1.2.1) with ESMTP id JAA29752; Wed, 27 Sep 1995 09:31:59 GMT Message-Id: <199509270931.JAA29752@qvarsx.er.usgs.GOV> To: hinsenk@ere.umontreal.ca (Hinsen Konrad) cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] J-style arrays in Python In-reply-to: <199509270138.VAA15833@cyclone.ERE.UMontreal.CA> Date: Wed, 27 Sep 1995 05:37:12 -0400 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk On Tue, 26 Sep 1995 21:38:49 -0400 Hinsen Konrad said: > To bring back some life into this discussion group, I'll distribute a > Python implementation of J-like arrays, to give those unfamiliar with > J a chance to become familiar with its array and rank system. Cool. This will help me become more familiar with some of the generic processing operators that have been discussed. I for one have not had much to say lately because I'm waiting for a chance to study some of the suggestions for map-ish operators. > And of > course it is also usable, as long as the arrays don't become too big. This is perfectly appropriate for a prototype. A real implementation will need to handle large arrays well. > This is a first test version; many important functions are still > missing, others (like output formatting) need substantial improvement, > and many could probably be made faster with some thought. > > To make (my) life simpler, I tried to stick to J's conventions as much > as possible (but hopefully without violating Python traditions). I am > not claiming that this is semantically the best way to implement arrays, > but it is a start. Please keep in mind that: - If there is a standard matrix class, it will be implemented in C, for performance reasons, - There is already a base implementation, currently being worked on by James Hugunin. - Much of the base implementation was dictated by the *stated* goal that the implementation should hold the data in an homogenous and contingous block of data suitable for passing directly to existing Fortran and C libraries. > Some general remarks: > > - Since my implementation uses nested lists to represent arrays, > the elements can be arbitrary objects. Which violates one of the basic goals of this effort. I realize that you may not agree with the goal, but this was clearly stated in the announcement for *this* SIG. > - Like arrays in J, my arrays are immutable, i.e. there is no > provision for changing individual elements. The reason for > making arrays immutable in J was that J is half-way to being > a functional language (it is not a pure functional language, > but many substantial problems can easily be solved in a > functional way). I have never missed element assignment, but > probably there are some good applications... Gee. I would miss element assignment. > - All functions are implemented as external functions, not as > methods. The main reason is that at first I could not think of > a way to implement methods with variable rank, although later > I figured out how to do this (in the same way as I implemented > reduction). This brings up a good point. I think that whatever we come up with should adhere to the KISS (Keep It Simple Stupid) rule as much as possible. I'm in favor of a fairly lean matrix module with auxilary modules to provide support for specific application areas. > I'll send two Python files; the first is the Array module itself, > and the second a kind of simple tutorial. Let me know if > anything is unclear. And let me know what you think of the > whole implementation! I look forward to studying what you sent. (When I have a chance. :) Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 27 09:37:22 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA08793 for matrix-sig-people; Wed, 27 Sep 1995 09:37:22 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id JAA08789 for ; Wed, 27 Sep 1995 09:37:16 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id JAA23343 (8.6.11/IDA-1.6); Wed, 27 Sep 1995 09:34:18 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA07132; Wed, 27 Sep 1995 09:34:17 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA06038; Wed, 27 Sep 1995 09:35:32 -0400 Date: Wed, 27 Sep 1995 09:35:32 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509271335.JAA06038@cyclone.ERE.UMontreal.CA> To: jfulton@usgs.gov CC: matrix-sig@python.org In-reply-to: <199509270931.JAA29752@qvarsx.er.usgs.GOV> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] J-style arrays in Python Sender: owner-matrix-sig@python.org Precedence: bulk Please keep in mind that: - If there is a standard matrix class, it will be implemented in C, for performance reasons, I hope so! - Much of the base implementation was dictated by the *stated* goal that the implementation should hold the data in an homogenous and contingous block of data suitable for passing directly to existing Fortran and C libraries. Actually I can't think of any reason not to keep the data in a continous block... > Some general remarks: > > - Since my implementation uses nested lists to represent arrays, > the elements can be arbitrary objects. Which violates one of the basic goals of this effort. I realize that you may not agree with the goal, but this was clearly stated in the announcement for *this* SIG. I do not disagree with that goal at all; in fact I seriously considered adding type checking (or rather consistency checking) to my Python implementation. But it would have slowed down everything without producing much of an advantage (I assume no one will produce mixed arrays by accident), so I left it out. Again, I do not claim in the least that any future array module should resemble my implementation in any respect. On the contrary, I expect that both could be used in parallel for different applications. I started writing this because I had a need for flexible (but small) arrays, and then polished it up a bit to make it usable as a demonstration for people in this SIG. Gee. I would miss element assignment. Really? I realize that element assignment is necessary to implement many of the standard linear algebra algorithms, but these would be implemented in C/Fortran/whatever anyway. I have never missed element assignment in J; in fact, I only noticed its absence while working on my Python implementation! This brings up a good point. I think that whatever we come up with should adhere to the KISS (Keep It Simple Stupid) rule as much as possible. I'm in favor of a fairly lean matrix module with auxilary modules to provide support for specific application areas. So am I. But we must make sure that the auxiliary modules can be used together conveniently. For example, the function names should be distinct, to make it possible to import them all into a single namespace (important for calculator-style use). ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 27 10:15:58 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA09012 for matrix-sig-people; Wed, 27 Sep 1995 10:15:58 -0400 Received: from qvarsx.er.usgs.GOV (qvarsx.er.usgs.gov [130.11.51.82]) by python.org (8.6.12/8.6.12) with ESMTP id KAA09008 for ; Wed, 27 Sep 1995 10:15:56 -0400 Received: from localhost (dsjfqvarsa [130.11.51.73]) by qvarsx.er.usgs.GOV (EMAIL 1.2.1) with ESMTP id OAA04669; Wed, 27 Sep 1995 14:08:38 GMT Message-Id: <199509271408.OAA04669@qvarsx.er.usgs.GOV> To: hinsenk@ere.umontreal.ca (Hinsen Konrad) cc: "James L Fulton, Hydrologist, Reston, VA " cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] J-style arrays in Python In-reply-to: <199509271335.JAA06038@cyclone.ERE.UMontreal.CA> Date: Wed, 27 Sep 1995 10:13:50 -0400 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk On Wed, 27 Sep 1995 09:35:32 -0400 Hinsen Konrad said: > > Please keep in mind that: > > - If there is a standard matrix class, it will be implemented in C, > for performance reasons, > > I hope so! > > - Much of the base implementation was dictated by the *stated* goal > that the implementation should hold the data in an homogenous and > contingous block of data suitable for passing directly to existing > Fortran and C libraries. > > Actually I can't think of any reason not to keep the data in a > continous block... > > > Some general remarks: > > > > - Since my implementation uses nested lists to represent arrays, > > the elements can be arbitrary objects. > > Which violates one of the basic goals of this effort. I realize that > you may not agree with the goal, but this was clearly stated in the > announcement for *this* SIG. > > I do not disagree with that goal at all; in fact I seriously > considered adding type checking (or rather consistency checking) to my > Python implementation. But it would have slowed down everything > without producing much of an advantage (I assume no one will produce > mixed arrays by accident), so I left it out. > > Again, I do not claim in the least that any future array module > should resemble my implementation in any respect. On the contrary, > I expect that both could be used in parallel for different > applications. I started writing this because I had a need for > flexible (but small) arrays, and then polished it up a bit to > make it usable as a demonstration for people in this SIG. Sounds like we're on the same wave-length then. > Gee. I would miss element assignment. > > Really? I realize that element assignment is necessary to implement > many of the standard linear algebra algorithms, but these would be > implemented in C/Fortran/whatever anyway. I have never missed element > assignment in J; in fact, I only noticed its absence while working on > my Python implementation! > > This brings up a good point. I think that whatever we come up with > should adhere to the KISS (Keep It Simple Stupid) rule as much as > possible. I'm in favor of a fairly lean matrix module with auxilary > modules to provide support for specific application areas. > > So am I. But we must make sure that the auxiliary modules can > be used together conveniently. For example, the function names > should be distinct, to make it possible to import them all into > a single namespace I don't agree with this. In general, I'm not a fan of "from spam import *". > (important for calculator-style use). Why? Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Sep 27 11:03:33 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA09219 for matrix-sig-people; Wed, 27 Sep 1995 11:03:33 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id LAA09215 for ; Wed, 27 Sep 1995 11:03:30 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id LAA26916 (8.6.11/IDA-1.6); Wed, 27 Sep 1995 11:00:30 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id LAA24095; Wed, 27 Sep 1995 11:00:29 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id LAA09849; Wed, 27 Sep 1995 11:01:45 -0400 Date: Wed, 27 Sep 1995 11:01:45 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509271501.LAA09849@cyclone.ERE.UMontreal.CA> To: jfulton@usgs.gov, matrix-sig@python.org In-reply-to: <199509271408.OAA04669@qvarsx.er.usgs.GOV> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] J-style arrays in Python Sender: owner-matrix-sig@python.org Precedence: bulk I don't agree with this. In general, I'm not a fan of "from spam import *". Neither am I, in general. But... > (important for calculator-style use). Why? To avoid having to type the module name all over again, of course. I am using my array module mostly to work (interactively) on simulation results read in from files (I didn't include the I/O in the version I sent around, because it still works only for a few special cases). So I am typing things like sqrt(Array("file1"))*Array("file2) which I prefer a lot to Array.sqrt(Array.Array("file1"))*Array.Array("file2) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Sep 29 08:36:05 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id IAA20328 for matrix-sig-people; Fri, 29 Sep 1995 08:36:05 -0400 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id IAA20324 for ; Fri, 29 Sep 1995 08:36:02 -0400 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa08365; 29 Sep 95 8:17 EDT Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) for id IAA19851; Fri, 29 Sep 1995 08:21:01 -0400 Message-Id: <199509291221.IAA19851@monty> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Time for recap? From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Fri, 29 Sep 1995 08:21:01 -0400 Sender: owner-matrix-sig@python.org Precedence: bulk I'll try to be short. I like the general idea of going with APL/J style multidimensional objects. I liked Konrad's sample implementation even though I agree an actual implementation would have to have a contiguous representation. I'm not sure about Konrad's particular notation for changing the rank of an operator (though it is very cool that that is possible in Python!). More thought needs to go in details like this. I think the best way to proceed is to use the built-in operators for APL/J style elementwise operations and to have the specific 2D linear algebra operations as methods. Likewise, instead of my earlier wild ideas about multidimensional slices, I propose to use a slice() method that can produce arbitrary slices; e.g. a[k][i:j] would be equivalent to a.slice(k, (i,j)). I'm not against dropping the parentheses in a[(i,j)] but I'm not sure if I can give it the proper thought and testing before October 13, the scheduled release of Python 1.3 (at 13:13:13 hours :-). We can work on a standard patch shortly after that date though. - - - I have one idea I would like to float by this group. How about separating out the representation and the structure? I believe I've seen C/Fortran matrix packages that made quite good use of this. The representation would be a simple 1-dimensional sequence. You'd normally not see or use this, but it would be there if you needed access to it (e.g. for passing to C/Fortran code). There's a simple way to map an index in an N-dim array into an index in the 1-dim representation array (Fortran compilers use it all the time :-). To make efficient use of this, I propose that slicing and indexing, as long as they return an object of rank >= 1, return an object that points into the same representation sequence. I propose one additional feature (which I believe I saw in Algol-68; some matrix packages may also use it): add a "stride" to each dimension (including the last). This makes it possible to have slices reference discontiguous parts of the underlying representation, and even to represent a transposed version! If we do things just right, it may be possible to pass in the sequence to be used as the representation -- it could be a Python list, tuple or array (from the array module). Again, all this can be prototyped in Python! I would give an example, but I have to run. --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Sep 29 09:47:32 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA20640 for matrix-sig-people; Fri, 29 Sep 1995 09:47:32 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id JAA20636 for ; Fri, 29 Sep 1995 09:47:27 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id JAA26509 (8.6.11/IDA-1.6); Fri, 29 Sep 1995 09:43:43 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA22407; Fri, 29 Sep 1995 09:43:42 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA27953; Fri, 29 Sep 1995 09:45:05 -0400 Date: Fri, 29 Sep 1995 09:45:05 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199509291345.JAA27953@cyclone.ERE.UMontreal.CA> To: guido@CNRI.Reston.VA.US CC: matrix-sig@python.org In-reply-to: <199509291221.IAA19851@monty> (message from Guido van Rossum on Fri, 29 Sep 1995 08:21:01 -0400) Subject: Re: [PYTHON MATRIX-SIG] Time for recap? Sender: owner-matrix-sig@python.org Precedence: bulk I liked Konrad's sample implementation even though I agree an actual implementation would have to have a contiguous representation. The two main reasons for choosing the nested list implementation (remember, that was before I even knew about the Matrix SIG) were: 1) the possibility of using nested lists as a notation for array constants; this means I rarely have to type Array(). 2) many of the array operations can be written concisely as recursive functions. The second reason is less important for our purposes, but we do have to think about a convenient notation for array constants. We might actually use nested lists and provide a coercion function that transforms them into a contiguous representation. I'm not sure about Konrad's particular notation for changing the rank of an operator (though it is very cool that that is possible in Python!). More thought needs to go in details like this. Definitely. I am not too pleased with it myself, mostly because the specification of ranks for functions with more than one argument is cumbersome, requiring two sets of square brackets. But I couldn't find another way to specify rank in Python! I think the best way to proceed is to use the built-in operators for APL/J style elementwise operations and to have the specific 2D linear algebra operations as methods. Why such a division? There is no clear borderline between "elementwise" and "linear algebra". If we should distinguish at all between methods and functions, I'd propose to use methods for procedures that change an array in place and functions for those that return a new array. I have one idea I would like to float by this group. How about separating out the representation and the structure? I believe I've seen C/Fortran matrix packages that made quite good use of this. The representation would be a simple 1-dimensional sequence. You'd normally not see or use this, but it would be there if you needed access to it (e.g. for passing to C/Fortran code). I have also seen C++ packages that make the representation available in this way, and it makes sense. To make efficient use of this, I propose that slicing and indexing, as long as they return an object of rank >= 1, return an object that points into the same representation sequence. Again, there are C++ classes that do this, and of course this makes slicing very efficient. On the other hand, it makes element assignment more complicated, since the array may have to be copied first. I propose one additional feature (which I believe I saw in Algol-68; some matrix packages may also use it): add a "stride" to each dimension (including the last). This makes it possible to have slices reference discontiguous parts of the underlying representation, and even to represent a transposed version! Again, this is used in C++ classes... There's a book that I have recommended before (Scientific end Engineering C++, by John J. Barton and Lee R. Nackman) that describes a matrix class with all the implementation features you just proposed. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Sep 29 15:20:56 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA22181 for matrix-sig-people; Fri, 29 Sep 1995 15:20:56 -0400 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id PAA22177 for ; Fri, 29 Sep 1995 15:20:51 -0400 Received: from nineveh.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA25738; Fri, 29 Sep 95 15:17:47 EDT Received: by nineveh.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA07305; Fri, 29 Sep 95 15:18:03 -0400 Message-Id: <9509291918.AA07305@nineveh.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Fri, 29 Sep 95 15:18:02 -0400 To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Time for recap? References: <199509291345.JAA27953@cyclone.ERE.UMontreal.CA> Sender: owner-matrix-sig@python.org Precedence: bulk > I like the general idea of going with APL/J style multidimensional > objects. > > I liked Konrad's sample implementation even though I agree an actual > implementation would have to have a contiguous representation. > > I'm not sure about Konrad's particular notation for changing the rank > of an operator (though it is very cool that that is possible in > Python!). More thought needs to go in details like this. I found Konrad's implementation enlightening regarding the ways of APL. I have a different notation for many of the things he does. Some of this notation should be discussed ASAP, but I'll save that for a later message. > I think the best way to proceed is to use the built-in operators for > APL/J style elementwise operations and to have the specific 2D linear > algebra operations as methods. I agree completely! > Likewise, instead of my earlier wild ideas about multidimensional > slices, I propose to use a slice() method that can produce arbitrary > slices; e.g. a[k][i:j] would be equivalent to a.slice(k, (i,j)). I disagree slightly here. I still prefer a[(k, range(i,j))] for Guido's example. This makes a[(k, [x,y,z])] possible. > I have one idea I would like to float by this group. How about > separating out the representation and the structure? > > I believe I've seen C/Fortran matrix packages that made quite good use > of this. The representation would be a simple 1-dimensional sequence. > You'd normally not see or use this, but it would be there if you > needed access to it (e.g. for passing to C/Fortran code). > > There's a simple way to map an index in an N-dim array into an index > in the 1-dim representation array (Fortran compilers use it all the > time :-). > > To make efficient use of this, I propose that slicing and indexing, as > long as they return an object of rank >= 1, return an object that > points into the same representation sequence. > > I propose one additional feature (which I believe I saw in Algol-68; > some matrix packages may also use it): add a "stride" to each > dimension (including the last). This makes it possible to have slices > reference discontiguous parts of the underlying representation, and > even to represent a transposed version! I agree almost completely with this specification. In fact, in many ways this is exactly what I have been implementing on top of Jim Fulton's original matrix object. (btw - the real or the imaginary part of a complex matrix are also particularly easy to represent using this style). The one thing that I've been doing differently is that slices (ie. a[1:5]) are returned by value rather than by reference. This was Jim Fulton's original implementation and I kept it because it was similar to the notion of slicing a list. Conceptually, I have no problems with treating slices the same as any other discontinuous index (ie. a[(range(1,5), range(4,6))]) and returning them by reference. Actually, I like the simplicity of being able to think of every index into a matrix returning by reference. I would be interested in hearing other opinions on this issue. > If we do things just right, it may be possible to pass in the sequence > to be used as the representation -- it could be a Python list, tuple > or array (from the array module). This is an interesting notion, but I can't see what it would gain over using a 1d C-array of basic types as the representation. It wouldn't be too hard to expand the definition of basic type to include (PyObject *)'s if you'd like to have the possibility of a matrix of "real" python objects. I should have time to finalize some of these things this weekend so that I can make an alpha version of a matrix object written in C available to interested members of this group on Monday. It is surprisingly similar to Konrad's sample implementation in python (though a lot bigger, uglier and yet faster owing to it's C-implementation). -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Sep 30 14:15:42 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA25684 for matrix-sig-people; Sat, 30 Sep 1995 14:15:42 -0400 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id OAA25680 for ; Sat, 30 Sep 1995 14:15:39 -0400 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa12822; 30 Sep 95 14:03 EDT Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id OAA22282; Sat, 30 Sep 1995 14:07:40 -0400 Message-Id: <199509301807.OAA22282@monty> To: James Hugunin cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Time for recap? In-reply-to: Your message of "Fri, 29 Sep 1995 15:18:02 EDT." <9509291918.AA07305@nineveh.LCS.MIT.EDU> References: <199509291345.JAA27953@cyclone.ERE.UMontreal.CA> <9509291918.AA07305@nineveh.LCS.MIT.EDU> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Sat, 30 Sep 1995 14:07:39 -0400 Sender: owner-matrix-sig@python.org Precedence: bulk > > Likewise, instead of my earlier wild ideas about multidimensional > > slices, I propose to use a slice() method that can produce arbitrary > > slices; e.g. a[k][i:j] would be equivalent to a.slice(k, (i,j)). > > I disagree slightly here. I still prefer a[(k, range(i,j))] for Guido's > example. This makes a[(k, [x,y,z])] possible. Ah. I was writing from memory, and forgot this feature. I don't like it. How many times do you have to select a more or less arbitrary group of rows/columns from a matrix? It makes the slice representation bulkier -- contiguous slices can be stored (in C) as 4 ints per dimension, while random selections will require a variable-length list of indices per dimension. (It is also wasteful to have to generate the full range of numbers when what you mean is a contiguous slice.) Perhaps a more important reason why I don't like this is that I see problems with various notations. In the current version of the language, you can't write a[2,3] -- you have to write a[(2,3)] and it will be used as a mapping index (not as a sequence index!). The proposal is to make this mean a[2][3]. That's fine with me. There are certain ambiguities with the use of parentheses in Python that I can't completely get rid of without breaking the type structure: (1) means the same as 1 -- a scalar, while (1,2) is a tuple of length 2. To write a tuple of length one, you can't write (1) -- you have to write (1,). (A tuple of length 0 is () -- easy.) For consistency, I think I'll have to make a[(1,)] mean the same as a[1], if a[(1,2)] is the same as a[1,2]. Finally, notice that to the object being indexed, there is no way to tell a[(1,2)] from x=(1,2); a[x]. A tuple is just one example of a sequence in Python. Lists are another example. In many situations, any sequence is acceptable and the results are the same (e.g. for loops). (And in situations where only lists or only tuples are accepted by the current version of the language, Steve Majewski has often made the point that there is no inherent reason why only one type should be accepted and that this should be fixed. I agree in most cases.) If we extend this rule to N-dimensional indexing, a[sequence], for some sequence of integers whose elements are i, j, k, ..., should be equivalent to a[i][j][k]..., and we can't make a[(1,2,3)] mean a[1][2][3] while at the same time interpreting a[[1,2,3]] as a[1:4]. (Sooner or later, you'll be passing a vector to the index. Then the question will arise, should this have the same meaning as a tuple or as a list. It's better if they all three mean the same.) Instead of supporting a[[2,3,5]] to select elements 2, 3 and 5 from a, I would propose to use filter() or a multi-dimensional extension thereof if you want to access selected subarrays. Or perhaps just a method a.select() where each argument is either an index (meaning a reduction of dimensionality in this dimension by picking just the sub-array with this index) or a sequence of indices (meaning selecting the set of sub-arrays with the indices in the sequence). How about it? Is this acceptable? > The one thing that I've been doing differently is that slices (ie. a[1:5]) > are returned by value rather than by reference. This was Jim Fulton's > original implementation and I kept it because it was similar to the notion > of slicing a list. Conceptually, I have no problems with treating slices > the same as any other discontinuous index (ie. a[(range(1,5), range(4,6))]) > and returning them by reference. Actually, I like the simplicity of being > able to think of every index into a matrix returning by reference. It's somewhat odd that slices are returned by reference (since they return a new value for lists), but not against the rules or silent assumptions of the language, and I think it is necessary to make the most of the flexibility that you get by using the notion of separating the indexing and the representation object. > I would be interested in hearing other opinions on this issue. Me too. > > If we do things just right, it may be possible to pass in the sequence > > to be used as the representation -- it could be a Python list, tuple > > or array (from the array module). > > This is an interesting notion, but I can't see what it would gain > over using a 1d C-array of basic types as the representation. I like this idea because the list or array containing the representation may already be available in memory -- so why copy it? Also by using an immutable underlying sequence (e.g. a tuple) it is easy to create immutable N-dimensional arrays without the need for a read-only flag. Finally it makes it possible to use a representation where the actual values are stored in disk and only fetched into memory when needed, using a cache -- this way you can implement your own virtual memory system, persistent matrices, etc. There could still be a "default" underlying representation that is highly optimized and that the indexing object knows about, for speedier access. > It wouldn't be too hard to expand the definition of basic type to > include (PyObject *)'s if you'd like to have the possibility of a > matrix of "real" python objects. Yes, the latter is definitely something that should be possible even if my idea doesn't find the acceptance I hope it will get. > I should have time to finalize some of these things this weekend so that I > can make an alpha version of a matrix object written in C available to > interested members of this group on Monday. It is surprisingly similar to > Konrad's sample implementation in python (though a lot bigger, uglier and > yet faster owing to it's C-implementation). Cool! --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Sep 30 14:36:36 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA25744 for matrix-sig-people; Sat, 30 Sep 1995 14:36:36 -0400 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id OAA25740 for ; Sat, 30 Sep 1995 14:36:33 -0400 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa13152; 30 Sep 95 14:26 EDT Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id OAA22332; Sat, 30 Sep 1995 14:30:25 -0400 Message-Id: <199509301830.OAA22332@monty> To: Hinsen Konrad cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Time for recap? In-reply-to: Your message of "Fri, 29 Sep 1995 09:45:05 EDT." <199509291345.JAA27953@cyclone.ERE.UMontreal.CA> References: <199509291345.JAA27953@cyclone.ERE.UMontreal.CA> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Sat, 30 Sep 1995 14:30:24 -0400 Sender: owner-matrix-sig@python.org Precedence: bulk (Me:) > I think the best way to proceed is to use the built-in operators for > APL/J style elementwise operations and to have the specific 2D linear > algebra operations as methods. (Konrad:) > Why such a division? There is no clear borderline between "elementwise" > and "linear algebra". If we should distinguish at all between methods > and functions, I'd propose to use methods for procedures that change > an array in place and functions for those that return a new array. I just meant to end the debate about whether a*b should mean matrix multiplication in the LA sense or elementwise multiplication like APL/J. This only an issue for * and /. For / most people agree that "matrix division" is numerically ill-defined and you should have specified the particular decomposition you need instead. This leaves *, and, in spite of the elegance of linear algebra matrix multiplication, I vote for the consistency of elementwise multiplication. This means that we need to define a method that implements LA matrix multiply, e.g. a.mul(b). Functions are out of the question (since they would have to live in the namespace reserved for built-in functions.) (Me:) > To make efficient use of this, I propose that slicing and indexing, as > long as they return an object of rank >= 1, return an object that > points into the same representation sequence. (Konrad:) > Again, there are C++ classes that do this, and of course this makes > slicing very efficient. On the other hand, it makes element assignment > more complicated, since the array may have to be copied first. I don't see much of a problem with that. Functions/methods that take an array and return a like-shaped array should always copy their argument before modifying it. Methods that are supposed to modify an array in place should not also return a reference to the array. Functions/methods that wish to modify the array only as a means of calculating some property of the array should come in to flavors: a "low-level" version whose name ends in "_inplace", which works on the array in place, and a "high-level" version which copies the array and then invokes the low-level version. The user can then decide to use the low-level version if performance so dictates, in which case the consequences of course must be considerd first. (I'd hate to see optional "in-place" flag arguments -- in general, I don't like to see flag arguments where the calls will always contain a constant argument value.) --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Sep 30 14:57:12 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA25851 for matrix-sig-people; Sat, 30 Sep 1995 14:57:12 -0400 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id OAA25847 for ; Sat, 30 Sep 1995 14:57:09 -0400 Received: from nineveh.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA01106; Sat, 30 Sep 95 14:53:53 EDT Received: by nineveh.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA00377; Sat, 30 Sep 95 14:53:50 -0400 Message-Id: <9509301853.AA00377@nineveh.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Sat, 30 Sep 95 14:53:50 -0400 To: Guido van Rossum Subject: Re: [PYTHON MATRIX-SIG] Time for recap? Cc: matrix-sig@python.org References: <199509291345.JAA27953@cyclone.ERE.UMontreal.CA> <9509291918.AA07305@nineveh.LCS.MIT.EDU> <199509301807.OAA22282@monty> Sender: owner-matrix-sig@python.org Precedence: bulk > > > Likewise, instead of my earlier wild ideas about multidimensional > > > slices, I propose to use a slice() method that can produce arbitrary > > > slices; e.g. a[k][i:j] would be equivalent to a.slice(k, (i,j)). > > > > I disagree slightly here. I still prefer a[(k, range(i,j))] for Guido's > > example. This makes a[(k, [x,y,z])] possible. > > Ah. I was writing from memory, and forgot this feature. I don't like > it. How many times do you have to select a more or less arbitrary > group of rows/columns from a matrix? It makes the slice representation > bulkier -- contiguous slices can be stored (in C) as 4 ints per > dimension, while random selections will require a variable-length > list of indices per dimension. (It is also wasteful to have to > generate the full range of numbers when what you mean is a contiguous > slice.) As far as implementation efficiency is concerned, I have implemented things so that each dimension requires 2 ints and an int pointer. If the dimension corresponds to a quasi-contiguous slice then the pointer is set to NULL, and the ints give the number of indices and a stride. If the dimension corresponds to a random slice, then the pointer points to an array of indices. So, very little efficiency is lost for quasi-contiguous slices, and yet it is still possible to represent random slices. My current favorite use for arbitrary indexing is for image zooming. ie: img = [1,2,3,4] a = img[((1,1,2,2,2,3,3,3,4,4),)] a = [1,1,2,2,2,3,3,3,4,4] This is a reasonably efficient approach for coarse-zooming in to even large 2d images, and it will work for on any arbitrary matrix (unlike C-code which I must write a special case for each matrix). My other reason for really liking this feature is that matlab supports it, and I would really like to see this object support as large a subset of matlab code as possible. > To write a tuple of length one, you can't write (1) -- > you have to write (1,). (A tuple of length 0 is () -- easy.) For > consistency, I think I'll have to make a[(1,)] mean the same as a[1], > if a[(1,2)] is the same as a[1,2]. Finally, notice that to the object > being indexed, there is no way to tell a[(1,2)] from x=(1,2); a[x]. I disagree completely with this. The proposal to allow tuplizing inside of []'s I feel should be completely compatible with "tupleizing" outside of brackets. Just as you can right in python "a = 1," and a will be a 1d tuple, you should be able to write a[1,] and have the index be a 1d tuple. Of course, this will ultimately be your decision. > If we extend this rule to N-dimensional indexing, a[sequence], for > some sequence of integers whose elements are i, j, k, ..., should be > equivalent to a[i][j][k]..., and we can't make a[(1,2,3)] mean > a[1][2][3] while at the same time interpreting a[[1,2,3]] as a[1:4]. > (Sooner or later, you'll be passing a vector to the index. Then the > question will arise, should this have the same meaning as a tuple or > as a list. It's better if they all three mean the same.) I actually agree with you completely on this. I have used Jim Fulton's generic sequence indexing functions in order to be sure that the index can happily be either a tuple or a list or a matrix (which actually should be special cased and optimized at some point). Either I missed a point somewhere, or you have a different interpretation of my indexing scheme than I do. As some examples: a[1,2,3] == a[(1,2,3)] == a[[1,2,3]] == a[1][2][3] ^ if this becomes allowed a[(1,2,3),] == a[((1,2,3),)] == [a[1], a[2], a[3]] However, if there is sufficient disapproval of this method of indexing, a.select() is a reasonable alternative. Just my two cents worth. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Oct 2 12:31:41 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA00550 for matrix-sig-people; Mon, 2 Oct 1995 12:31:41 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id MAA00546 for ; Mon, 2 Oct 1995 12:31:35 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id MAA18714 (8.6.11/IDA-1.6); Mon, 2 Oct 1995 12:27:24 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA19570; Mon, 2 Oct 1995 12:27:24 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA14222; Mon, 2 Oct 1995 12:27:22 -0400 Date: Mon, 2 Oct 1995 12:27:22 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199510021627.MAA14222@cyclone.ERE.UMontreal.CA> To: guido@CNRI.Reston.VA.US CC: matrix-sig@python.org In-reply-to: <199509301807.OAA22282@monty> (message from Guido van Rossum on Sat, 30 Sep 1995 14:07:39 -0400) Subject: Re: [PYTHON MATRIX-SIG] Time for recap? Sender: owner-matrix-sig@python.org Precedence: bulk Ah. I was writing from memory, and forgot this feature. I don't like it. How many times do you have to select a more or less arbitrary group of rows/columns from a matrix? It makes the slice representation Quite often, if you use matrices as tables of numbers. bulkier -- contiguous slices can be stored (in C) as 4 ints per dimension, while random selections will require a variable-length list of indices per dimension. (It is also wasteful to have to generate the full range of numbers when what you mean is a contiguous slice.) Maybe there should be two internal representations, one for contigous blocks (i.e. submatrices) and one for arbitrary slices. A tuple is just one example of a sequence in Python. Lists are another example. In many situations, any sequence is acceptable and the results are the same (e.g. for loops). (And in situations where only lists or only tuples are accepted by the current version of the language, Steve Majewski has often made the point that there is no inherent reason why only one type should be accepted and that this should be fixed. I agree in most cases.) Actually, I have never really understood why Python needs both tuples and lists. Anything you can with tuples you can do with lists, so why have tuples? Instead of supporting a[[2,3,5]] to select elements 2, 3 and 5 from a, I would propose to use filter() or a multi-dimensional extension thereof if you want to access selected subarrays. Or perhaps just a You mean actually having to iterate over the whole array just to pick some element? That sounds a bit wasteful. One problem I am seeing in this discussion is that indexing is being treated separately from other array operations, although it is really only one structural function among many (reshaping, transposing, etc.). We should rather discuss the complete set of structural functions together; they should behave consistently and together allow all array manipulations that might occur, no matter which can be done by "indexing" and which by something with another name. If you look at my Array implementation, you will see that there indexing is just syntactic sugar for a structural function "take" that selects an arbitrary set of items from its argument. It has the useful property that the shape of the result is the combination of the shape of the "index" argument and the shape of the items of the "data" argument. That gives great flexibility in selecting subarrays, and provides an easy-to-understand behaviour even in complicated cases. In my implementation, indexing itself is limited, since you can't specify rank, but I don't really care, since "take" lets me do whatever I want. > It wouldn't be too hard to expand the definition of basic type to > include (PyObject *)'s if you'd like to have the possibility of a > matrix of "real" python objects. Yes, the latter is definitely something that should be possible even if my idea doesn't find the acceptance I hope it will get. You can add support from me ;-) General arrays would be a useful feature without causing much effort. The main problem I see is how to specify whether e.g. a constant array of integers is supposed to be an array of integers or a general array whose initial values happen to be integers. But this can be overcome with explicit conversion, if necessary. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Oct 2 13:12:44 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA00732 for matrix-sig-people; Mon, 2 Oct 1995 13:12:44 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id NAA00728 for ; Mon, 2 Oct 1995 13:12:41 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id NAA19702 (8.6.11/IDA-1.6); Mon, 2 Oct 1995 13:08:13 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA26008; Mon, 2 Oct 1995 13:08:12 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA15986; Mon, 2 Oct 1995 13:08:11 -0400 Date: Mon, 2 Oct 1995 13:08:11 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199510021708.NAA15986@cyclone.ERE.UMontreal.CA> To: guido@CNRI.Reston.VA.US CC: matrix-sig@python.org In-reply-to: <199509301830.OAA22332@monty> (message from Guido van Rossum on Sat, 30 Sep 1995 14:30:24 -0400) Subject: Re: [PYTHON MATRIX-SIG] Time for recap? Sender: owner-matrix-sig@python.org Precedence: bulk I just meant to end the debate about whether a*b should mean matrix multiplication in the LA sense or elementwise multiplication like APL/J. This only an issue for * and /. For / most people agree that ... OK, on that I agree, of course. I thought you were referring to functions like matrix inversion. I don't see much of a problem with that. Functions/methods that take an array and return a like-shaped array should always copy their argument before modifying it. Methods that are supposed to modify an array in place should not also return a reference to the array. That seems a good way to make the distinction clear. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Oct 3 12:45:12 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA06254 for matrix-sig-people; Tue, 3 Oct 1995 12:45:12 -0400 Received: from alfven.jhuapl.edu (alfven.jhuapl.edu [128.244.146.147]) by python.org (8.6.12/8.6.12) with SMTP id MAA06249 for ; Tue, 3 Oct 1995 12:45:05 -0400 Message-Id: <199510031645.MAA06249@python.org> Received: by alfven.jhuapl.edu (1.38.193.4/16.2) id AA12935; Tue, 3 Oct 1995 12:47:47 -0400 Date: Tue, 3 Oct 1995 12:47:47 -0400 From: Chris Chase S1A To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Mutli-dimensional indexing and other comments Reply-To: chris.chase@jhuapl.edu Sender: owner-matrix-sig@python.org Precedence: bulk This is a long note. I have tried to go back and read through the various matrix/array proposals from the start of this list. I have some comments on various proposals and opinions. I also would like to make some suggestions about multi-dimensional indexing. ---- Hinsen Konrad's proposal: - Like arrays in J, my arrays are immutable, i.e. there is no provision for changing individual elements. I find that changinng individual elements a necessity in almost all the computations that I do. For example, this kind of operation is pervasive in imaging applications (e.g. segmentation and region-of-interest manipulations) and block-matrix linear algebra computations. For me, setting elements is a must. ----- Array functions: Hinsen Konrad supports calculator sytle array usage, i.e. interactive computation: I am typing things like sqrt(Array("file1"))*Array("file2) which I prefer a lot to Array.sqrt(Array.Array("file1"))*Array.Array("file2) I definitely prefer the first sytle for interactive computations. I also like Konrad's array function prototype that allows rank specification, e.g. product.over[1](b). Guido did not like the rank specification via an index argument. Could the rank be overriden using a keyword argument instead of an index argument, e.g. product.over(b,rank=1)? Since keywords are commonly used to override default arguments this would seem a natural possibility. ------ Heirarchical (list-like) indexing: Jim Fulton's proposal thinks of multi-dimensional arrays as being heirarchical so that a[i] returns a N-1 dimensioned array. This allows him to use list style indexing, e.g. a[i][j]. This approach would seem to rule out flatten indexing (one-dimensional indexing) and multi-dimensional slicing, e.g. would a[2:9][1:5] just end up being equivalent to a[3:7]? Perhaps that is what is wanted. If the heirarchical approach is used then a different mechanism would be necessary for more general multi-dimensional indexing. I could see that heirarchical indexing like list indexing, would be useful. My opinion is that arbitrary multi-dimensional slicing is more useful for the homogeneous arrays we are discussing (they are not lists). Perhaps both could be supported. ---- I am confused about proposals for array slices to be references. In Jim's proposal, assignment is by copy and access is by reference. Then am I correct in understanding that for b=a[i] (a is two-dimensional) changing elements of b will not change a? ----- General Multi-dimensional indexing: In general, I have not been able to keep straight the various proposals. It seems that most issues have been concerned with implementation issues or with limiting extensions so they will not break the current language. Instead of diving into the implementation questions I would like to present my views on multi-dimensional indexing for arrays. Then I would like to know how the current proposals fit into this view. I have worked with or investigated a number of interactive array-oriented computational languages: IDL, Matlab, Tela, scilab, Octave, rlab, Mathematica, APL. I have seen the following indexing concepts all of which are very useful and completely generalize array access. Like Konrad, I will use the term "rank" to refer to the number of dimensions of an array and "shape" to refer to the ordered list of dimension sizes of an array. The arrays are homogeneous and can be viewed as one-dimensional (as in C or Fortran) or multi-dimensional. index vector: a scalar, slice (with optional stride), or array. An array index could be our new array (matrix) object, tuple or list. Let A be a multi-dimensional array (rank is 3 in the examples) to be indexed and let a1, a2, a3 be arbitrary index vectors. one-dimensional indexing: a) flattened indexing. Takes a single index vector and the array is viewed like contiguously-stored arrays in Fortran or C (one or the other, but consistent in that always the first or last dimension changes most rapidly.) The shape of the result is the same as the shape of the index vector. b (not really 1-D?) heirachical indexing (Jim Fulton's list-like indexing): Mentioned above. The index vector is used only for the first dimension. I have never used this type of indexing for homogeneous numeric arrays (because it is a special case of the product indexing), but I see that it could be useful when viewing the array as a list. For a scalar index the result has rank one less than A. If a non-scalar is used as an index vector then the index vector would be treated in flattened form and the result would have have the same rank as A. Multi-dimensional indexing: The number of index vectors is equal to the rank A with one index vector for each dimension. There are two types of indexing that I have seen: a) product indexing: You might call this arbitrary slice indexing. I sometimes call this Cartesian product indexing because all combinations of the index vector elements are used for indexing. All index vectors are used in flattened form. The result has the same rank but the size of each dimension is equal to the length of the corresponding index vector. The elements of the result are taken from all possible ordered combinations of the index vector elements, e.g. the result element at index i,j,k is taken from A at index a1(i),a2(j),a3(k). b) mapped indexing (as in Tela): The index vectors all have the same shape and are used in flattened form. Indexes into array A are generated from ordered elementwise grouping, i.e. the result at 1-D index i is taken from A at index a1(i),a2(i),a3(i). The shape of the result is the same as the shape of the index vectors. Selection indexing (as in Octave or APL): A type of 1-D indexing where the index vector is a {0,1} vector that has the same number of elements as A. The result contains only those elements of A where the corresponding index vector is non-zero. Rather than support this directly, many languages, e.g. IDL and Matlab, support this indirectly with a "where" or "find" function. where(A) returns a rank 1 array containing in increasing order the one-dimensional indexes where A is nonzero. The result of where(A) can then be used for 1-D indexing. This is more powerful then supporting selection directly. Insertion indexing (as in IDL): Used in setting blocks of items. Takes a 1-D or multi-dimensional scalar index that specifies the starting position for the object insertion, overwriting existing items. For a 1-D scalar index the object inserted is used in flattened form. For a multi-dim index the object inserted must have the same rank as A. When the inserted object would extend beyond the dimension bounds of A there are several possible behaviors: signal error, index wrap-around, or truncation of A to fit. In most of the languages that I have used, the syntax A[] is used for both 1-D and multi-dimensional product/slicing indexing. Sometimes, 1-D indexing is used when only a single index vector is given. Konrad suggests that this is just syntactic sugar in place of function calls like "take" and "ravel". But it is extremely useful for writing clear, readable code (even APL acknowledges this by supporting "[]" syntax for indexing). A natural solution syntactically would be: 1. "[]" subscripting to support 1-D indexing and product indexing depending on the number of index vectors given. 2. allowing ":" slice notation for index vectors. Mapped indexing, heirarchical indexing, insertion and where/selection could be access methods. It does not seem possible without major internal Python changes to implement both 1) and 2) because of the ingrained and non-extensible 1-D indexing built-in to Python. Someone (?) suggested that arbitrary (product) indexing did not seem useful and slicing was sufficient. For my personal interests, product indexing is necessary for a huge number of applications. I use mapped indexing less often, but it is similarly necessary in many applications. Of course all indexing can be converted manually into 1-D indexing, but this would make an array extension almost unuseable for interactive use. ------ Question: Do any of the proposals completely support these types of indexing in some form? The closest I have seen to supporting product indexing is James Hugunin proposal of using a mapping type: M[(range(1,3),range(2,4))] -> [[13,14],[23,24]] The idea suggests wrapping the index arguments of [] into a tuple. However, this would not allow both 1-D and multi-dimensional indexing for the same array without possible ambiguity. To avoid the extra parentheses Guido suggests: - Allowing M[i, j] for (multidimensional) sequence types would also mean that D[i, j] would be equivalent to D[(i, j)] for dictionaries. But Jim Fulton: I see no reason to support M[i,j] for arbitrary sequence types. I'd say that if a type wants to support multiple arguments to [], then it should provide mapping behavior and have the mapping implementation sniff for either an integer or a tuple argument and do the right thing. I am *very much* against a language change to support this. One complex solution that preserves the current Python language: Treat M[i,j] as multiple dimension indexing and M[(i,j)] as 1-D indexing. Think of M[i,j] as a function call with 2 arguments and M[(i,j)] as a function call with a single argument. As specified by the current language, M[(i,j)] is a mapping type, but (i,j) is treated as a one-dimensional index vector selecting items i and j. On the other hand, when multiple index arguments are used, e.g. M[i,j], a different multi-dimension index method taking a variable length argument list is called, e.g. __msetitem__ or __mgetitem__. This allows both 1-D and multi-dim indexing for the same object that _preserves_ existing language behavior. This also overcomes the discussion of about bundling the index arguments into a tuple, e.g. a[1,], which I think could be a source of errors and confusion. Of course, this solution requires large changes to Python internals. ------------- Multi-dimensional slicing: If possible I would prefer slice notation over range(). Using range() is certainly not as clean as ":". ":" is more than syntaic sugar since it will use dimension length information for the object that is not available to range(). With range() how do I specify the entire dimension or dimension length for the upper bound? For large array expressions the repetitive range() can generate a lot of clutter. For example: a[1:3,:,5:] == a[range(1:3),range(0,shape(a)[1])),range(5,shape(a)[2])] The left side is much more readable at a glance. Then imagine if this was part of a larger expression. Of course this could have been made a little simpler by storing the shape in an auxillary variable: s = shape(a) then: a[1:3,:,5:] == a[range(1:3),range(0,s[1])),range(5,len(s[2]))] To support slice ":" for multi-dimensions Guido suggests an expression like (2:3). But Jim Fulton replies: Can't be, (2:3) is not a valid expression, so it can't yield a valid element of the tuple. Additionally, allowing (2:3) expressions in place of range() expressions would break old slice usage. A possible complex solution to allow ":" multi-dimensional expressions with backwards compatibility: To make slice expressions valid there would have to be some kind of built-in slice type. For example, it could just be a tuple subclass with the only difference being a type name of "slice". When used for multi-dim indexing the indexing function would have to support scalars, arrays or slices for the indexes by checking the type of the index. For backwards compatibility, when using one-dimensional indexing M[2:3] would call __getslice__(2,3) by unpacking the slice tuple. When using slices for multi-dimensional indexes, supporting negative upper bounds would have to be a built-in dimension length function so that the dimension upper bound would be available, e.g. when computing M[2,3:-1]. If negative upper bounds could be forfeited for multi-dim slices then the dimension length function would not be necessary ("None" could be use for the upper bound in "2:"). If such additions to Python are just too much work, then I think Jim's range() workaround is the next best thing. ------ The solutions above to allow ":" and 1-D plus multi-dimensional indexing are not simple to implement. They would require major internal changes to the compiler in addition to other areas. But they would _preserve_ backwards compatibility with the current language. They are complex because the original language was not designed with extensibility to completely general multi-dimensional indexing in mind. General opinion about possible Python array/matrix extensions: Trying to implement multi-dimensional arrays without any changes to Python internals could end up being cumbersome, inelegant workarounds that may be confusing and discouraging to end users (especially those coming from numeric array languages like APL, Matlab, IDL, Tela, etc). The extensions would most likely not be completely general. Why should we limit the hamstring the capabilities of an array extension just to avoid internal Python changes? Additionally, while they might be useable in scripts, they will be inefficient for extensive interactive use. Chris Chase ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Oct 3 22:08:50 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id WAA09719 for matrix-sig-people; Tue, 3 Oct 1995 22:08:50 -0400 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id WAA09714 for ; Tue, 3 Oct 1995 22:08:45 -0400 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa24939; 3 Oct 95 21:54 EDT Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id VAA05077; Tue, 3 Oct 1995 21:58:07 -0400 Message-Id: <199510040158.VAA05077@monty> To: chris.chase@jhuapl.edu cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Mutli-dimensional indexing and other comments In-reply-to: Your message of "Tue, 03 Oct 1995 12:47:47 EDT." <199510031645.MAA06249@python.org> References: <199510031645.MAA06249@python.org> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Tue, 03 Oct 1995 21:58:07 -0400 Sender: owner-matrix-sig@python.org Precedence: bulk > I also like Konrad's array function prototype that allows rank > specification, e.g. product.over[1](b). Guido did not like the rank > specification via an index argument. Could the rank be overriden > using a keyword argument instead of an index argument, > e.g. product.over(b,rank=1)? Since keywords are commonly used to > override default arguments this would seem a natural possibility. Sounds good to me, though I'm still struggling with the concent of the rank of an operator... > Heirarchical (list-like) indexing: > > Jim Fulton's proposal thinks of multi-dimensional arrays as being > heirarchical so that a[i] returns a N-1 dimensioned array. This > allows him to use list style indexing, e.g. a[i][j]. This approach > would seem to rule out flatten indexing (one-dimensional indexing) and > multi-dimensional slicing, e.g. would a[2:9][1:5] just end up being > equivalent to a[3:7]? Yes. > Perhaps that is what is wanted. Not necessarily, but it's the only thing that's possible given the various constraints and the current language implementation. While a[i,j] could easily be added (meaning a[i][j]), a[i:j,k:l] would require major re-engineering of a complicated piece of code. I proposed a.slice((i,j), (k,l)) to express this. > I am confused about proposals for array slices to be references. In > Jim's proposal, assignment is by copy and access is by reference. > Then am I correct in understanding that for b=a[i] (a is > two-dimensional) changing elements of b will not change a? No. Assignment to a simple name (e.g. "b") will always be by reference, this is fundamental in the language. However, *slice* assignment has to copy (e.g. b[i:-1j] = a[i-1:j]), and I have proposed that slice references (a[i:j] used in an expression) of arrays return a reference to the original elements of a. E.g. after b = a[i:j], assignment to elements of b will change the corresponding elements of a. This is different from slice assignments for Python lists, but seems to be the most useful rslice semantics for arrays. > In general, I have not been able to keep straight the various > proposals. It seems that most issues have been concerned with > implementation issues or with limiting extensions so they will not > break the current language. Indeed, such are the practicalities of extending a language that has been used and developed extensively by/for thousands of users over the past five years. > Instead of diving into the implementation questions I would like to > present my views on multi-dimensional indexing for arrays. Then I > would like to know how the current proposals fit into this view. An approach that I like, at least in theory. [What followed was too long for me to digest completely.] > Why should we limit the hamstring the capabilities of an array > extension just to avoid internal Python changes? Because nobody wants to do the work. > Additionally, while they might be useable in scripts, they will be > inefficient for extensive interactive use. Surely you mean "too much to type" and not "inefficient to execute". Even though Python is quite usable interactively I don't think its its real strengths lie there, and I don't think that interactive use should be used as an argument in favor or against certain solutions. --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Oct 4 15:05:18 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA01483 for matrix-sig-people; Wed, 4 Oct 1995 15:05:18 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id PAA01479 for ; Wed, 4 Oct 1995 15:05:12 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id PAA21349 (8.6.11/IDA-1.6); Wed, 4 Oct 1995 15:03:43 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA02184; Wed, 4 Oct 1995 15:03:42 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA18613; Wed, 4 Oct 1995 15:03:40 -0400 Date: Wed, 4 Oct 1995 15:03:40 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199510041903.PAA18613@cyclone.ERE.UMontreal.CA> To: chris.chase@jhuapl.edu CC: matrix-sig@python.org In-reply-to: <199510031645.MAA06249@python.org> (message from Chris Chase S1A on Tue, 3 Oct 1995 12:47:47 -0400) Subject: Re: [PYTHON MATRIX-SIG] Mutli-dimensional indexing and other comments Sender: owner-matrix-sig@python.org Precedence: bulk specification via an index argument. Could the rank be overriden using a keyword argument instead of an index argument, e.g. product.over(b,rank=1)? Since keywords are commonly used to override default arguments this would seem a natural possibility. I have never used keyword arguments, but that's no reason not to use them ;-) Basically the idea is not bad; however, it lacks one useful feature of my implementation: it is not possible to create function objects with modified ranks if the rank is only supplied in the function call. Whenever I need a particular function/rank combination often, I just define a new function, e.g. reshape_items = reshape[[-1,1]] reshape_items(range(5),[2,2]) gives the result 0 0 0 0 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 This is useful not just to save typing, but also to give meaningful names to nonobvious array functions, which helps to make the code more readable. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Oct 12 19:28:39 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id TAA15707 for matrix-sig-people; Thu, 12 Oct 1995 19:28:39 -0400 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id TAA15703; Thu, 12 Oct 1995 19:28:33 -0400 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA00469; Thu, 12 Oct 95 16:26:44 PDT Received: by kristen.llnl.gov (5.0/SMI-SVR4) id AA13225; Thu, 12 Oct 1995 16:26:42 +0800 Date: Thu, 12 Oct 1995 16:26:42 +0800 Message-Id: <9510122326.AA13225@kristen.llnl.gov> From: "P. Dubois" To: matrix-sig@python.org Cc: guido@python.org Subject: [PYTHON MATRIX-SIG] Hello from LLNL Content-Length: 1881 Sender: owner-matrix-sig@python.org Precedence: bulk Greetings, I head the computer science effort at Lawrence Livermore National Laboratory in X-Division, where we do plasma physics and the design of targets for laser fusion experiments. We have many large numerical applications in Fortran and are starting newer codes in C++ and Eiffel. For the last ten years we have used a programmable applications shell for Fortran which I wrote. This is reaching the end of its useful lifetime as we move further and further away from the world for which it was designed, the monolithic (one CPU, one process, one language = Fortran) large code. I had been trying to design a replacement system for us but I found Python already is very close to what I was designing (except it is better done than I would have managed). We have decided that building on Python is the way to go. So, we'd like to know what is going on in the Matrix SIG, and other items of interest to those of us who are numerically and graphically intensive. We bring a big pile of experienced and eager manpower to the table and hope we can give a hand to this effort. I designed the EiffelMath numerical library for the Eiffel language, so I have considerable experience in both numerical mathematics and object-oriented technology. And of course having run my own extension language for 10 years I and the members of my team can do things of a compiler sort. We have some special expertise in MPI and things parallel/ vector, too. So: what do you have already? and how can we help? A bunch of us will be there in December to get trained properly... (My Fortran system is at http://www-phys.llnl.gov/X_Div/htdocs/basis.html) Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com Editor, Scientific Programming Department Computers in Physics ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Oct 12 22:08:29 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id WAA16504 for matrix-sig-people; Thu, 12 Oct 1995 22:08:29 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id WAA16500 for ; Thu, 12 Oct 1995 22:08:25 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id WAA13096 (8.6.11/IDA-1.6 for ); Thu, 12 Oct 1995 22:05:02 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id WAA20076; Thu, 12 Oct 1995 22:05:00 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id WAA14506; Thu, 12 Oct 1995 22:04:59 -0400 Date: Thu, 12 Oct 1995 22:04:59 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199510130204.WAA14506@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] J-style arrays in Python, second edition Sender: owner-matrix-sig@python.org Precedence: bulk In the meantime I have debugged and extended the Python implementation of J-style arrays that I distributed earlier, so it is time for an update. The main new features: file I/O, better output formatting, more functions (cumulative reduction, comparison, and more mathematical functions). And hopefully fewer bugs; reshape() was actually quite destructive in the first version... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Oct 12 22:08:46 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id WAA16516 for matrix-sig-people; Thu, 12 Oct 1995 22:08:46 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id WAA16512 for ; Thu, 12 Oct 1995 22:08:39 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id WAA13112 (8.6.11/IDA-1.6 for ); Thu, 12 Oct 1995 22:05:23 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id WAA20234; Thu, 12 Oct 1995 22:05:22 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id WAA14547; Thu, 12 Oct 1995 22:05:22 -0400 Date: Thu, 12 Oct 1995 22:05:22 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199510130205.WAA14547@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Array.py Sender: owner-matrix-sig@python.org Precedence: bulk # J-style array class # Arrays are represented by a scalar or a list, possibly containing # other lists in case of higher-rank arrays. Array rank is limited # only by the user's patience. # Send comments to Konrad Hinsen import types, math, string, regexp, copy ###################################################################### # Error type ArrayError = 'ArrayError' ###################################################################### # Various functions that do the real work. Classes follow. # Construct string representation of array def _output(data, dimension, maxlen): s = '' if dimension == 0: s = s + string.rjust(data,maxlen) elif dimension == 1: for e in data: s = s + string.rjust(e,maxlen) + ' ' else: separator = (dimension-1)*'\n' for e in data: s = s + _output(e,dimension-1,maxlen) + separator i = len(s)-1 while i > 0 and s[i] == '\n': i = i-1 return s[:i+1] # Find the shape of an array and check for consistency def _shape(data): if type(data) == types.ListType: if data and type(data[0]) == types.ListType: shapes = map(lambda x:_shape(x),data) for i in range(1,len(shapes)): if shapes[i] != shapes[0]: raise ArrayError, 'Inconsistent shapes' shape = [len(data)] shape = shape + shapes[0] return shape else: return [len(data)] else: return [] # Copy the data structure of an array def _copy(data, dimension): if (dimension == 0): return data else: c = copy.copy(data) for i in range(len(c)): c[i] = _copy(c[i], dimension-1) return c # Construct a one-dimensional list of all array elements def __ravel(data): if type(data) == types.ListType: if len(data) and type(data[0]) == types.ListType: return reduce(lambda a,b: a+b, map(lambda x: __ravel(x), data)) else: return data else: return [data] def _ravel(array): return Array(__ravel(array._data), [reduce(lambda a,b: a*b, array._shape, 1)]) # Reshape an array def _reshape(array, shape): array = _ravel(array) if len(shape._data) == 0: return take(array,0) else: array = _copy(array._data, len(array._shape)) shape = shape._data n = reduce(lambda a,b: a*b, shape) if n > len(array): nr = (n+len(array)-1)/len(array) array = (nr*array)[:n] elif n < len(array): array = array[:n] for i in range(len(shape)-1, 0, -1): d = shape[i] n = n/d for j in range(n): array[j:j+d] = [array[j:j+d]] return Array(array,shape) # Map a function on the first dimensions of an array def _extract(a, index, dimension): if len(a[1]) < dimension: return a else: return (a[0][index], a[1][1:], a[2]) def _map(function, arglist, max_frame, scalar_flag): result = [] if len(max_frame) == 0: if scalar_flag: result = apply(function,tuple(map(lambda a: a[0], arglist))) else: result = apply(function,tuple(map(lambda a: Array(a[0],a[2]), arglist)))._data else: for index in range(max_frame[0]): result.append(_map(function, map(lambda a,i=index,d=len(max_frame): _extract(a,i,d), arglist), max_frame[1:], scalar_flag)) return result # Reduce an array with a given binary function def _reduce(function, array): function = function[0] array = array[0] if len(array._shape) == 0: return array elif array._shape[0] == 0: return reshape(function._neutral, array._shape[1:]) else: result = Array(array._data[0], array._shape[1:]) for i in range(1,array._shape[0]): result = function(result, Array(array._data[i], array._shape[1:])) return result def _cumulative(function, array): function = function[0] array = array[0] if len(array._shape) == 0: return array elif array._shape[0] == 0: return array else: shape = array._shape last_result = array._data[0] result = [last_result] for i in range(1,array._shape[0]): last_result = function(last_result, Array(array._data[i], array._shape[1:]))._data result.append(last_result) return Array(result, shape) # Find the higher of two ranks def _maxrank(a,b): if a == None or b == None: return None else: return max(a,b) ###################################################################### # Array class definition class Array: def __init__(self, scalar_or_list, shape = None): self._data = scalar_or_list if shape == None: self._shape = _shape(self._data) else: self._shape = shape def __copy__(self): return Array(_copy(self._data, len(self._shape)), copy.copy(self._shape)) def __str__(self): s = tostr(self) maxstrlen = maximum.over(_strlen(_ravel(s)))._data return _output(s._data,len(s._shape),maxstrlen) __repr__ = __str__ def __len__(self): if type(self._data) == types.ListType: return len(self._data) else: return 1 def __getitem__(self, index): return take(self, index) def __getslice__(self, i, j): return take(self, range(i,j)) def __add__(self, other): return sum(self, other) __radd__ = __add__ def __sub__(self, other): return difference(self, other) def __rsub__(self, other): return difference(other, self) def __mul__(self, other): return product(self, other) __rmul__ = __mul__ def __div__(self, other): return quotient(self, other) def __rdiv__(self, other): return quotient(other, self) def __pow__(self,other): return power(self, other) def __rpow__(self,other): return power(other, self) def __neg__(self): return 0-self def writeToFile(self, filename): file = open(filename, 'w') file.write(str(self)+'\n') file.close # Check for arrayness def isArray(x): return hasattr(x,'_shape') # Read array from file _int_pattern = regexp.compile('-?[0-9]+') _float_pattern = regexp.compile('-?[0-9]*\\.[0-9]*([eE][+-]?[0-9]+)*') def _match(pattern,string): r = pattern.match(string) for i in r: if i == (0, len(string)): return 1 return 0 def _convertEntry(s): if _match(_int_pattern,s): return string.atoi(s) elif _match(_float_pattern,s): return string.atof(s) else: return s def readArray(filename): list = a = [] stack = [] blanks = 0 file = open(filename) line = file.readline() while line: if line[0] != '#': elements = map(_convertEntry, string.split(line)) if len(elements): if blanks: while blanks > len(stack): a = [a] stack.append(a) list = copy.copy([]) stack[blanks-1].append(list) for i in range(blanks-2,-1,-1): list.append(copy.copy([])) stack[i] = list list = list[0] list.append(elements) blanks = 0 else: blanks = blanks + 1 line = file.readline() file.close() while type(a) == types.ListType and len(a) == 1: a = a[0] return Array(a) ###################################################################### # Array function class class ArrayFunction: def __init__(self, function, ranks, intrinsic_ranks=None): self._function = function if isArray(ranks): self._ranks = ranks._data elif type(ranks) == types.ListType: self._ranks = ranks else: self._ranks = [ranks] if intrinsic_ranks == None: self._intrinsic_ranks = self._ranks else: self._intrinsic_ranks = intrinsic_ranks if len(self._ranks) == 1: self._ranks = len(self._intrinsic_ranks)*self._ranks def __call__(self, *args): if len(self._ranks) != len(args): raise ArrayError, 'Wrong number of arguments for an array function' arglist = [] framelist = [] shapelist = [] for i in range(len(args)): if isArray(args[i]): arglist.append(args[i]) else: arglist.append(Array(args[i])) shape = arglist[i]._shape rank = self._ranks[i] intrinsic_rank = self._intrinsic_ranks[i] if rank == None: cell = 0 elif rank < 0: cell = min(-rank,len(shape)) else: cell = max(0,len(shape)-rank) if intrinsic_rank != None: cell = max(cell,len(shape)-intrinsic_rank) framelist.append(shape[:cell]) shapelist.append(shape[cell:]) max_frame = [] for frame in framelist: if len(frame) > len(max_frame): max_frame = frame for i in range(len(framelist)): if framelist[i] != max_frame[len(max_frame)-len(framelist[i]):]: raise ArrayError, 'Incompatible arguments' scalar_function = reduce(lambda a,b:_maxrank(a,b), self._intrinsic_ranks) == 0 return Array(_map(self._function, map(lambda a,b,c: (a._data,b,c), arglist, framelist, shapelist), max_frame, scalar_function)) def __getitem__(self, ranks): return ArrayFunction(self._function,ranks,self._intrinsic_ranks) class BinaryArrayFunction(ArrayFunction): def __init__(self, function, neutral_element, ranks, intrinsic_ranks=None): ArrayFunction.__init__(self, function, ranks, intrinsic_ranks) self._neutral = neutral_element self.over = ArrayFunction(ArrayOperator(_reduce, [self]), [None]) self.cumulative = ArrayFunction(ArrayOperator(_cumulative, [self]), [None]) def __getitem__(self, ranks): return BinaryArrayFunction(self._function, self._neutral, ranks, self._intrinsic_ranks) class ArrayOperator: def __init__(self, operator, function_list): self._operator = operator self._functions = function_list def __call__(self, *args): return apply(self._operator, (self._functions, args)) ###################################################################### # Array functions # Functions for internal use _strlen = ArrayFunction(len, [0]) # Structural functions shape = ArrayFunction(lambda a: Array(a._shape,[len(a._shape)]), [None]) reshape = ArrayFunction(_reshape, [None, 1]) ravel = ArrayFunction(_ravel, [None]) take = ArrayFunction(lambda a,i: Array(a._data[i._data], a._shape[1:]), [None, 0]) # Elementwise binary functions _sum = ArrayFunction(lambda a,b: a+b, [0, 0]) _difference = ArrayFunction(lambda a,b: a-b, [0, 0]) _product = ArrayFunction(lambda a,b: a*b, [0, 0]) _quotient = ArrayFunction(lambda a,b: a/b, [0, 0]) _power = ArrayFunction(pow, [0, 0]) _max = ArrayFunction(max, [0, 0]) _min = ArrayFunction(min, [0, 0]) _smaller = ArrayFunction(lambda a,b: ab, [0, 0]) _equal = ArrayFunction(lambda a,b: a==b, [0, 0]) sum = BinaryArrayFunction(_sum, 0, [None, None]) difference = BinaryArrayFunction(_difference, 0, [None, None]) product = BinaryArrayFunction(_product, 1, [None, None]) quotient = BinaryArrayFunction(_quotient, 1, [None, None]) power = BinaryArrayFunction(_power, 1, [None, None]) maximum = BinaryArrayFunction(_max, 0, [None, None]) minimum = BinaryArrayFunction(_min, 0, [None, None]) smaller = BinaryArrayFunction(_smaller, 1, [None, None]) greater = BinaryArrayFunction(_greater, 1, [None, None]) equal = BinaryArrayFunction(_equal, 1, [None, None]) # Scalar functions of one variable tostr = ArrayFunction(str, [0]) sqrt = ArrayFunction(math.sqrt, [0]) exp = ArrayFunction(math.exp, [0]) log = ArrayFunction(math.log, [0]) log10 = ArrayFunction(math.log10, [0]) sin = ArrayFunction(math.sin, [0]) cos = ArrayFunction(math.cos, [0]) tan = ArrayFunction(math.tan, [0]) asin = ArrayFunction(math.asin, [0]) acos = ArrayFunction(math.acos, [0]) atan = ArrayFunction(math.atan, [0]) sinh = ArrayFunction(math.sinh, [0]) cosh = ArrayFunction(math.cosh, [0]) tanh = ArrayFunction(math.tanh, [0]) floor = ArrayFunction(math.floor, [0]) ceil = ArrayFunction(math.ceil, [0]) # Nasty hack to make fix max and min safe to use. # Without this, they would return an array as most users # would expect, but it would not be the correct answer. # I know I shouldn't do this, but it seems the lesser of # two evils. builtin_max = max builtin_min = min def max(*args): return apply(builtin_max,args) def min(*args): return apply(builtin_min,args) # test data x = Array(range(10)) ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Oct 12 22:09:04 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id WAA16530 for matrix-sig-people; Thu, 12 Oct 1995 22:09:04 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id WAA16525 for ; Thu, 12 Oct 1995 22:09:01 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id WAA13120 (8.6.11/IDA-1.6 for ); Thu, 12 Oct 1995 22:05:49 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id WAA20350; Thu, 12 Oct 1995 22:05:48 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id WAA14571; Thu, 12 Oct 1995 22:05:48 -0400 Date: Thu, 12 Oct 1995 22:05:48 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199510130205.WAA14571@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] ArrayExamples.py Sender: owner-matrix-sig@python.org Precedence: bulk # This file illustrates the use of the the Array class. # # Send comments to Konrad Hinsen # from Array import * ###################################################################### # Arrays are constructed from (nested) lists: a = Array(range(10)) b = Array([ [2,3,7], [9,8,2] ]) c = Array([ [ [4,5,6], [0,4,5] ], [ [1,6,5], [8,5,2] ] ]) # Scalars make arrays of rank 0: s = Array(42) # Array elements can be anything: text_array = Array(['Hello', 'world']) # Arrays can be printed: print 'a:\n', a print 'b:\n', b print 'c:\n', c print 's:\n', s # shape() returns an array containing the dimensions of another array: print 'shape(a):\n', shape(a) print 'shape(b):\n', shape(b) print 'shape(c):\n', shape(c) print 'shape(s):\n', shape(s) # Scalar functions act on each element of an array: print 'sqrt(b):\n',sqrt(b) # Binary operators likewise work elementwise: print 'c+c:\n',c+c # But you can also add a scalar: print 'c+s:\n',c+s # To understand the general rule for combining arrays of different # shapes in a function, we need some more technical terms: # The length of the shape vector of an array is called its rank. # The elements of an array along the first axis are called items. # The items of an array of rank N have rank N-1. More generally, # the shape vector is divided into frames and cells. Frames and # cells are not properties of an array, but describe ways of looking # at an array. For example, a rank-3 array can be regarded as # as just that - a single rank-3 array. It can also be regarded # as a rank-1 frame of rank-2 cells, or as a rank-2 frame of # rank-1 cells. Or even as a rank-3 array of rank-0 cells, i.e. # scalar cells. # # When two arrays are to be added (or multiplied, or...), their # shapes need not equal, but the lower-rank array must match # an equal-rank cell of the higher-rank array. The lower-rank # array will then be combined with each corresponding cell, and # the result will have the shape of the higher-rank array. print 'b+c:\n',b+c # The addition of a scalar is just a special case: it has rank 0 # and therefore matches the rank-0 cells (scalar elements) of any array! print 'b+s:\n',b+s # All operators are also available as normal binary function, # e.g. addition can be written as print 'sum(a,s):\n',sum(a,s) # You'll need this form to perform reductions, e.g. a sum # of all items of an array: print 'sum.over(a):\n',sum.over(a) # Let's do it with a higher-rank array: print 'product.over(b):\n',product.over(b) # But how do you get the product along the second axis # of b? Easy: print 'product.over[1](b):\n',product.over[1](b) # The [1] after the function name product.over modifies # the functions rank. Function ranks are related to array # ranks, in that a function of rank N operates on the # N-cells of its argument. If the argument has a higher # rank, the function is applied to each N-cell and the # result is combined with the frame of the argument. # In the last example, product.over will be # called for each of the 1-cells of b, returning a # rank-0 result for each, and the results will be # collected in the 1-frame of b, producing as a net # result an array of rank 1. # # All functions have ranks; if no rank is explicitly # given, the default rank will be used. The default # rank of all reductions is 'unbounded', which means # that the function will operate on the highest-level # cells possible. Many functions have unbounded rank, # for example shape(): print 'shape(c):\n',shape(c) # But of course you can modify the rank of shape(): print 'shape[1](c):\n',shape[1](c) print 'shape[2](c):\n',shape[2](c) # Functions with more than one argument can have a different # rank for each. The rank is applied to each argument, # defining its frames and cells for this purpose. The frames # must match in the same way as indicated above for # addition of two arrays. The function is then applied # to the cells, and if appropriate, the same matching # procedure is applied once again. This may seem confusing # at first, but it is really just the application of a # single principle everywhere! # # For example, let's take a (our rank-1 array) and add # b (our rank-2 array) to each of a's 0-cells: print 'sum[[0,None]](a,b):\n',sum[[0,None]](a,b) # 'None' stands for 'unbounded'. Since the rank of sum is # 0 for its first argument, a is divided into 1-frames # and 0-cells. For b the rank is unbounded, so it is # treated as a 0-frame with 2-cells. b's 0-frame matches # a's 1-frame (a 0-frame matches everything!), and # the result gets the 1-frame. The cells of the result # are sums of a rank-0 array (element of a) and a rank-2 # array (b), i.e. rank-2 arrays by the matching rules # given above. So the net total is a rank-3 array, # as you have seen. # From now on we will specify the default rank of each function. # It should be noted that specifying a higher rank than the # default rank has no effect on the function's behaviour. Only # lower ranks make a difference. # # All the scalar mathematical functions (sqrt, sin, ...) have # rank 0. The binary arithmetic functions (sum, product, ...) # have unbounded rank for both argument. For structural functions # (i.e. those that modify an array's shape rather than its # elements), the rank varies. As we have seen, shape() is # unbounded. The other structural functions are yet to be # introduced: # ravel() produces a rank-1 array containing all elements # of its argument. It has unbounded rank: print 'ravel(c):\n',ravel(c) # reshape() allows you to change the shape of an array. # It first obtains a rank-1 list of elements of its first # argument (like ravel()) and then reassembles the # elements into an array with the shape given by the # second argument: print 'reshape(a,[2,2]):\n',reshape(a,[2,2]) print 'reshape(b,[10]):\n',reshape(b,[10]) # As you have seen in the second case, reshape() reuses # the elements of its arguments all over if they get # exhausted. # You may have noticed that in some examples, a nested list # has been used instead of an array as a function argument. # In general, all array functions will convert its arguments # to arrays if necessary. # Now we need a way to select parts of an array. You can # use standard indexing and slicing to obtain items # (i.e. N-1 cells for a rank-N array): print 'c[1]:\n',c[1] print 'a[3:7]:\n',a[3:7] # You can also specify an array as the index and obtain an # array of the corresponding items: print 'a[[2,6,1]]:\n',a[[2,6,1]] # The function take() does exactly the same: print 'take(c,1):\n',take(c,1) # You will have to use take() if you want to modify its # ranks, which are [None,0] by default. There is little # point in changing the second rank (it wouldn't make # any difference), but changing the first rank lets you # select along other dimensions than the first: print 'take[1](c,0):\n',take[1](c,0) print 'take[2](c,1):\n',take[2](c,1) # Isn't there something wrong here? take() takes # two arguments and therefore needs two ranks. But for # convenience, only one rank must be given if all ranks # are to be the same. # Now the hard part is over. You are supposed to know # how data and function ranks work together. What # remains to be done is to show some more array functions. # First of all, unary functions that act on each element # of a matrix. These are: tostr, sqrt, exp, log, log10, # sin, cos, tan, asin, acos, atan, sinh, cosh, tanh, # floor, and ceil. # # Of course you expect the standard arithmetic operations, # + - * / and pow(). The first four also exist as named functions # which allow you to change rank; the names are sum, difference, # product, and quotient. They work as expected. But there are # some more binary functions that work in the same way. # maximum() and minimum select the larger/smaller one of # their arguments. With .over they find the maximum/minimum # of an arbitrary list: print 'maximum.over(a):\n',maximum.over(a) # Then there are the comparison operations: smaller, greater, # and equal. They exist only as functions of that name, not in # the form of the familiar operators. It is simply not possible # with the current Python implementation to assign such a meaning # to the comparison operators, so you'll have to live with that # for a while. # Sometimes you want not just the sum over some list # of items, but all the intermediate partial sums along # the way. There is a special function for that: print 'sum.cumulative(a):\n',sum.cumulative(a) # Finally, you might want to read arrays from disk files, # or write an array to a file. The second problem is handled # by a method, so you write something like c.writeToFile('just_a_test') # You can read this back using c_from_file = readArray('just_a_test') # and then you should check whether the two are indeed equal: print 'equal(c,c_from_file):\n',equal(c,c_from_file) # And that's the end of this introduction. Stay tuned for # updates of the array package that will provide some # important still missing in this version. In the meantime, # play round with what there is to get a feeling for # how things work. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Oct 19 13:38:31 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA06948 for matrix-sig-people; Thu, 19 Oct 1995 13:38:31 -0400 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id NAA06943 for ; Thu, 19 Oct 1995 13:38:26 -0400 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA18689; Thu, 19 Oct 95 10:38:11 PDT Received: by kristen.llnl.gov (5.0/SMI-SVR4) id AA25244; Thu, 19 Oct 1995 10:38:09 +0800 Date: Thu, 19 Oct 1995 10:38:09 +0800 Message-Id: <9510191738.AA25244@kristen.llnl.gov> From: "P. Dubois" To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] a few thoughts Content-Length: 5327 Sender: owner-matrix-sig@python.org Precedence: bulk I have only begun to peruse the archive but I did see a couple of discussions worth some comments which I would like to make. For ten years I have been in charge of a Fortran extension language called Basis, which has now been used in perhaps 150-200 codes. I designed this language to look like an array Fortran. It has been extremely popular with users. So what follows is a synthesis of a whole lot of experience. Let me begin with a short synopsis of Basis' array rules: Arrays can be of any of the standard Fortran types, including complex. They consist of a single block of storage with auxillary "shape" information. They can be up to seven dimensional (the Fortran limit) but in practice five is the highest I have seen in use. Had I been implementing them in a modern language of course no limit would have been necessary. Operations are element-wise but we have a separate operator for matrix multiply and matrix divide (the latter means, a /! b is that x such that b *! x = a), where /! and *! are the matrix operators). All functions such as sin(x) operate element wise and do the right thing depending on the type of x. It is important to be able to pass the address of the storage area off to compiled code. Scalars broadcast but we have an explicit function for "dup'ing" something to make it match something higher dimensional, rather than do this automagically Now some comments: 1. Usage of the matrix operators is perhaps 1% or less of the usage of the element-wise numbers. This is because when two-dimensional matrices do arise, they usually represent the spatial or other type of discretization, far more often than they represent operators. If I had it to do over again, I would not have special operators, just a function call, since the light usage is not worth the trouble. 2. Speed is crucial. The basic operations must take place in compiled loops without function calls, with as little overhead as possible. In other words, x + y must be lead to a compiled loop doing the corresponding operation on the blocks of storage. Yes, maybe one has broadcast/type coercion/shape checks or operations first, but if both x and y are really double arrays of length n we want to be into a C loop doing xa[i] + ya[i] where xa and ya point to the storage areas. This is because when a code is written as a programmable application with an interpreter over compiled routines, the codes tend to evolve to having more and more parts of the code in the interpreter. Also, the interpreter is used to calculate information that is derivable from the compiled state variables rather than add new compiled code, but sometimes these calculations are quite intensive. 3. I think it is mistaken to try to reduce the implementor's job by doing many types in one like the "array" built-in object does. Having basic double/integer/complex stuff work fast should be the primary consideration, even if it means some tedious and not terribly elegant coding. A completely Python class like the J-array posting is beautiful and suitable for cases where uniformity of representation is of value, but it is a completely different question than having something fast that can talk to C or Fortran. I timed this class doing x+x where x was 100,000 elements long, and it was about 1000 times slower than a simple C extension to Python and even ten times slower than doing a for loop in Python (but boy, I learned a lot about Python from it!). 4. In Basis, sqrt(-1.) is an error not a complex; we actually went through a time when it was complex and found it to be a disadvantage. One must balance the need to avoid/find errors against the need for easy expression. 5. One might want to consider having a very fast, very raw vector class on which to base higher level classes that have concepts like shape, etc. 6. Banging malloc too hard can be a problem in these beasts. You probably couldn't afford to represent a big matrix with independently malloc'ed pointers for each row. Some historical notes: a. I noticed some discussion of representing things always as complex numbers. The original Matlab (when Cleve Moler wrote it to teach students linear algebra) represented everything as a complex number. There was a limit of I think 5000 numbers in a system, period, because it used a fixed array. When I ported it to a Cray I replaced that part of it with a memory manager so that you could have a lot of complex numbers. But of course having all the numbers be secretly complex is completely crazy from an efficiency point of view, not to mention storage. b. Yes, I implemented an extension language in Fortran. It was 1984 and it was the only language available on a Cray. The computer center manager said that I didn't need C, you can do everything in Fortran. c. The documentation for the Basis language is available on line at http://www-phys.llnl.gov/X_Div/htdocs/basis.html. I have decided to base my next generation system on Python. d. Reference for some philosophy: Dubois, P.F., "Making Applications Programmable", Computers in Physics Jan/Feb 1994. Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com Editor, Scientific Programming Department, Computers in Physics ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Oct 19 14:39:57 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA07408 for matrix-sig-people; Thu, 19 Oct 1995 14:39:57 -0400 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id OAA07404 for ; Thu, 19 Oct 1995 14:39:50 -0400 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA05846; Thu, 19 Oct 95 14:27:49 -0400 Received: by maigret (950215.SGI.8.6.10/940406.SGI) id OAA09461; Thu, 19 Oct 1995 14:43:29 -0400 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199510191843.OAA09461@maigret> Subject: Re: [PYTHON MATRIX-SIG] a few thoughts To: dubois@kristen.llnl.gov (P. Dubois) Date: Thu, 19 Oct 1995 14:43:29 -0400 (EDT) Cc: matrix-sig@python.org In-Reply-To: <9510191738.AA25244@kristen.llnl.gov> from "P. Dubois" at Oct 19, 95 10:38:09 am X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 659 Sender: owner-matrix-sig@python.org Precedence: bulk Paul Dubois' comments struck a chord with me. While I appreciate the cleanliness of python's generic containers, I am painfully aware of the slowness of python's operations on, say, sound files with thousands of elements, to take a mid-size example. 1. How fervently are people opposed to splitting the array/tensors in terms of element type? One array for any python object, and one for numbers, all of the same type (one type per array). The two array types could have the same interface, but one could use the uniformity of its elements' types for optimization. 2. Am I right that this dichotomy would allow massive optimization? -david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Oct 19 15:24:09 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA07697 for matrix-sig-people; Thu, 19 Oct 1995 15:24:09 -0400 Received: from qvarsx.er.usgs.GOV (qvarsx.er.usgs.gov [130.11.51.82]) by python.org (8.6.12/8.6.12) with ESMTP id PAA07693 for ; Thu, 19 Oct 1995 15:24:03 -0400 Received: from localhost (dsjfqvarsa [130.11.51.73]) by qvarsx.er.usgs.GOV (EMAIL 1.2.1) with ESMTP id TAA00071; Thu, 19 Oct 1995 19:18:07 GMT Message-Id: <199510191918.TAA00071@qvarsx.er.usgs.GOV> To: da@maigret.cog.brown.edu (David Ascher) cc: dubois@kristen.llnl.gov (P. Dubois) cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] a few thoughts In-reply-to: <199510191843.OAA09461@maigret> Date: Thu, 19 Oct 1995 15:23:40 -0400 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk On Thu, 19 Oct 1995 14:43:29 -0400 (EDT) David Ascher said: > Paul Dubois' comments struck a chord with me. While I appreciate the > cleanliness of python's generic containers, I am painfully aware of the > slowness of python's operations on, say, sound files with thousands of > elements, to take a mid-size example. > > 1. How fervently are people opposed to splitting the array/tensors in > terms of element type? One array for any python object, and one for > numbers, all of the same type (one type per array). The two array > types could have the same interface, but one could use the uniformity > of its elements' types for optimization. > > 2. Am I right that this dichotomy would allow massive optimization? The current proposal is for homogenous matrices. So also supporting heterogenous matrices would not allow any additional optimizaton. Note also that the current proposal also allows homogenous matrices of characters, to aid in interfacing to Fortran routines with character arguments. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Oct 19 18:07:50 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA08692 for matrix-sig-people; Thu, 19 Oct 1995 18:07:50 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id SAA08688 for ; Thu, 19 Oct 1995 18:07:46 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id SAA24708 (8.6.11/IDA-1.6); Thu, 19 Oct 1995 18:05:39 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id SAA04266; Thu, 19 Oct 1995 18:05:38 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id SAA12015; Thu, 19 Oct 1995 18:05:37 -0400 Date: Thu, 19 Oct 1995 18:05:37 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199510192205.SAA12015@cyclone.ERE.UMontreal.CA> To: dubois@kristen.llnl.gov CC: matrix-sig@python.org In-reply-to: <9510191738.AA25244@kristen.llnl.gov> (dubois@kristen.llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] a few thoughts Sender: owner-matrix-sig@python.org Precedence: bulk 1. Usage of the matrix operators is perhaps 1% or less of the usage of the element-wise numbers. This is because when two-dimensional That raises the question of what kinds of applications were typically involved. I can think of several that would make heavy use of arrays as operators (e.g. all kinds of quantum mechanics). Maybe they were just underrepresented among your users. fast that can talk to C or Fortran. I timed this class doing x+x where x was 100,000 elements long, and it was about 1000 times slower than a simple C extension to Python and even ten times slower than doing a for loop in Python (but boy, I learned a lot about Python from it!). I agree that speed is a real problem. In fact, I openly admit that I keep a "stripped-down" version that does only one-dimensional lists for the many cases where I need long one-dimensional arrays. But this is purely an implementation problem, not a problem in principle. Much of the efficiency problems probably comes from the decision to use nested lists as an internal representation, and this decision was mostly based on laziness; it let me use simple recursive functions to handle higher-rank arrays. But of course having all the numbers be secretly complex is completely crazy from an efficiency point of view, not to mention storage. When I made this suggestion, I referred to modern implementations of APL (including J), which in fact have many internal representations for numbers, for efficiency reasons. A typical APL implementation has 1) Bits 2) small integers (i.e. bytes) 3) long integers (4 bytes) 4) real numbers 5) complex numbers But to the user all this looks like a single number type, since all conversions happen automatically. The price to pay is not in efficiency (internal APL operations tend to outperform Fortran), but in a rather complex implementation, which has to decide the optimal data type based on various criteria. For example, the high cost of unpacking bit arrays means that they will be used only for very large objects and/or when memory runs down. The advantage of this is that numbers behave like you would expect from mathematics, e.g. 1/3 equals 1./3., not 0. This prevents many errors. There are actually more user-friendly features like this in APL, e.g. non-zero comparison tolerance. Actually, my dream language would also handle symbolic operations in the style of Mathematica or Maple... b. Yes, I implemented an extension language in Fortran. It was 1984 and it was the only language available on a Cray. The computer center manager said that I didn't need C, you can do everything in Fortran. Isn't it great to have someone who always knows your real needs? ;-) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Oct 19 18:11:04 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA08738 for matrix-sig-people; Thu, 19 Oct 1995 18:11:04 -0400 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id SAA08734 for ; Thu, 19 Oct 1995 18:11:00 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id SAA24758 (8.6.11/IDA-1.6); Thu, 19 Oct 1995 18:08:34 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id SAA04817; Thu, 19 Oct 1995 18:08:32 -0400 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id SAA12155; Thu, 19 Oct 1995 18:08:31 -0400 Date: Thu, 19 Oct 1995 18:08:31 -0400 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199510192208.SAA12155@cyclone.ERE.UMontreal.CA> To: da@maigret.cog.brown.edu CC: matrix-sig@python.org In-reply-to: <199510191843.OAA09461@maigret> (da@maigret.cog.brown.edu) Subject: Re: [PYTHON MATRIX-SIG] a few thoughts Sender: owner-matrix-sig@python.org Precedence: bulk 1. How fervently are people opposed to splitting the array/tensors in terms of element type? One array for any python object, and one for numbers, all of the same type (one type per array). The two array types could have the same interface, but one could use the uniformity of its elements' types for optimization. Isn't that what Guido already proposed? I see nothing that speaks agains that. The implementation of "general" arrays wouldn't even be much different from the others; a general array would simply be an array of object pointers. Only the application of non-structural functions and operators has be handled differently. 2. Am I right that this dichotomy would allow massive optimization? Of what? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Oct 20 12:18:59 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA12457 for matrix-sig-people; Fri, 20 Oct 1995 12:18:59 -0400 Received: from eeel.nist.gov (sparky.eeel.nist.gov [129.6.64.163]) by python.org (8.6.12/8.6.12) with SMTP id MAA12453 for ; Fri, 20 Oct 1995 12:18:54 -0400 Received: from acdc.eeel.nist.gov by eeel.nist.gov (4.1/SMI-3.2-del.6) id AA29059; Fri, 20 Oct 95 12:18:43 EDT Received: by acdc.eeel.nist.gov (4.1/SMI-3.2-del.5) id AA21381; Fri, 20 Oct 95 12:15:01 EDT Date: Fri, 20 Oct 95 12:15:01 EDT Message-Id: <9510201615.AA21381@acdc.eeel.nist.gov> From: Michael McLay To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Comments on 'a few thoughts' In-Reply-To: <9510191738.AA25244@kristen.llnl.gov> References: <9510191738.AA25244@kristen.llnl.gov> Sender: owner-matrix-sig@python.org Precedence: bulk P. Dubois writes: > 1. Usage of the matrix operators is perhaps 1% or less of the usage of > the element-wise numbers. This is because when two-dimensional > matrices do arise, they usually represent the spatial or other type of > discretization, far more often than they represent operators. If I > had it to do over again, I would not have special operators, just a > function call, since the light usage is not worth the trouble. Wouldn't it be appropriate to make them methods instead of functions? I agree that a good design rule would be to not implement symbolic operators for rarely used operations. Unfortunately convincing someone who is creating an operator that it is truly rare may not be easy. The goal of this design rule is to ensure code is readable. Readability is dependent on the vocabulary of the audience. If code is only to be read by a small group who have agreed upon a standardized shorthand then overloading symbols is Ok. However, to reach the broadest audience the use of functions or methods is necessary in order for the code to be unambiguous to the human parsing the source. Granted, it will make the application source code a little wordy, but that is the price to be paid for clarity. Giving programmers the unrestricted ability to overload operators can be dangerous since it tends to lead to the development of obscure dialects in notation. While I'm on the subject of design rules... Perhaps the Mathematica convention of spelling out the full name of everything instead of choosing arbitrary abbreviations should also be adopted. This would be consistent with the Python tradition of making the source code readable to others. A simple example will illustrate. How would you interpret the following expression. speed = m/hr In scanning the text can you be sure if this is miles/hour or meters/hour? I'd also suggest that only SI units be used in setting up libraries. It is simple enough to do unit conversion at the user interface. > 3. I think it is mistaken to try to reduce the implementor's job by > doing many types in one like the "array" built-in object does. > Having basic double/integer/complex stuff work fast should be the > primary consideration, even if it means some tedious and not terribly > elegant coding. > 5. One might want to consider having a very fast, very raw vector class > on which to base higher level classes that have concepts like shape, etc. This proposal suggests adding many specialized numerical types to Python. Each type would be tuned for efficient performance in solving a specific problem. From an implantation viewpoint this should not be difficult to put in place. Each new numeric type would be implemented as a dynamically linked module. This solution may be inevitable. Each application domain will by necessity build an efficient set of types required for the application domain's calculations. This approach to implementation be a pragmatic necessity since at the implementation level the computational requirements will demand that all operations on large data sets run at the speed of compiled code. Dividing the problem into discrete, importable modules compartmentalizes the work. Each numeric type module can independently be implement. All the operations needed for a numeric type would be incorporated into the module. This proposal just solves the easy part of the problem. Hinsen Konrad writes: > When I made this suggestion, I referred to modern implementations of > APL (including J), which in fact have many internal representations > for numbers, for efficiency reasons. A typical APL implementation > has > 1) Bits > 2) small integers (i.e. bytes) > 3) long integers (4 bytes) > 4) real numbers > 5) complex numbers > But to the user all this looks like a single number type, since all > conversions happen automatically. The price to pay is not in efficiency > (internal APL operations tend to outperform Fortran), but in a > rather complex implementation, which has to decide the optimal > data type based on various criteria. For example, the high cost > of unpacking bit arrays means that they will be used only for > very large objects and/or when memory runs down. > > The advantage of this is that numbers behave like you would expect > from mathematics, e.g. 1/3 equals 1./3., not 0. This prevents many > errors. There are actually more user-friendly features like this in > APL, e.g. non-zero comparison tolerance. This is the hard part of the problem to solve. Providing automatic type conversion would be a great feature and would help reduce the number of bugs and the complexity of applications. The only hitch is that it may take a significant effort to create a working implementation. Assuming that the first solution is inevitable. That is, people will write point solutions to solve their problems. Is there something that would prevent the addition of automatic type conversion from being implemented as a layer on top of the numeric type modules that will be created independently? What rules need to be established be ensure that the initially independent numeric types can be integrated into the more elegant solution? > Actually, my dream language would also handle symbolic operations > in the style of Mathematica or Maple... Yes, and one of the features from Mathematica that is missing is the ability to tag objects with symbols that represent units of measure. In Mathematica you can do the following: In[1]: 12 meters The meters symbol tags the integer 12 as its unit of measure. Michael ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Oct 27 10:39:29 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA23280 for matrix-sig-people; Fri, 27 Oct 1995 10:39:29 -0400 Received: from rnet.rose.rsoc.rockwell.com (rose.rsoc.rockwell.com [161.40.39.100]) by python.org (8.6.12/8.6.12) with SMTP id KAA23276 for ; Fri, 27 Oct 1995 10:39:24 -0400 Received: by rnet.rose.rsoc.rockwell.com (4.1/SMI-4.1) id AA16105; Fri, 27 Oct 95 09:38:33 CDT Received: from sunrise(161.40.50.11) by rnet via smap (V1.3) id sma016097; Fri Oct 27 09:38:09 1995 Received: from feynman.rsoc.rockwell.com by rose.rsoc.rockwell.com (5.0/SMI-SVR4) id AA24224; Fri, 27 Oct 1995 09:38:08 -0500 Received: by feynman.rsoc.rockwell.com (5.0/SMI-SVR4) id AA17917; Fri, 27 Oct 1995 09:38:07 -0500 Date: Fri, 27 Oct 1995 09:38:07 -0500 From: forrest@rose.rsoc.rockwell.com (Dave Forrest) Message-Id: <9510271438.AA17917@feynman.rsoc.rockwell.com> To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Comments on 'a few thoughts' X-Sun-Charset: US-ASCII Content-Length: 1608 Sender: owner-matrix-sig@python.org Precedence: bulk > From owner-matrix-sig@python.org Fri Oct 20 11:20 CDT 1995 [snip] > P. Dubois writes: > > > 1. Usage of the matrix operators is perhaps 1% or less of the usage of > > the element-wise numbers. This is because when two-dimensional Not in my office. Here, when we implemented a Matrix class in C++, we didn't even supply elementwise multiplication or _any_ division operator because _we_never_use_those_operations (well, we did implement division by a scalar - but that's it! :-). I made this plea once before but I'm going to make it again - please consider providing two interfaces into the matrix implementation - one that acts like a two-dimensional, linear-systems-type matrix and one that acts like a tensor. Even in C you can have the same code underlying it (reuse is good). I wouldn't try to deny you tensor-types the elementwise behaviour that you obviously need, but we need linear algebra - it's at the core of all our math models - please take this need seriously. > > matrices do arise, they usually represent the spatial or other type of > > discretization, far more often than they represent operators. If I > > had it to do over again, I would not have special operators, just a > > function call, since the light usage is not worth the trouble. > > Wouldn't it be appropriate to make them methods instead of functions? Absolutely - failing making them operators (as above) - methods are definitely the way to go. Actually, I think that that is what everyone meant, it's just a matter of OO semantics. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Oct 30 12:28:02 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA01490 for matrix-sig-people; Mon, 30 Oct 1995 12:28:02 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id MAA01485 for ; Mon, 30 Oct 1995 12:27:54 -0500 Received: from nineveh.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA02555; Mon, 30 Oct 95 12:26:42 EST Received: by nineveh.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA09927; Mon, 30 Oct 95 12:28:39 -0500 Message-Id: <9510301728.AA09927@nineveh.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Mon, 30 Oct 95 12:28:37 -0500 To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Alpha Matrix Object in C available - Guinea Pigs wanted Cc: jjh@mama-bear.lcs.mit.edu Sender: owner-matrix-sig@python.org Precedence: bulk I finally have an alpha version of a working, reasonably efficient matrix object implemented in C. It compiles sucessfully on a Sparc10 running SUNOS, and on a P5(and P6) running NeXTStep. I think that this object should satisfy many of the features requested by members of this group. 1) Extremely fast basic arithmetic functions To me, this is sine qua non of a useful matrix object. If I can't do arithmetic at close to the speed of hand coded C, then I have no use for the object. As a sanity check that things are reasonably efficient, I performed a very rough comparision of the basic speeds of matlab, octave, python, and C for vector arithmetic. The operation was to multiply a 10000 length vector of doubles with itself 1000 times (10 M floating point multiplies). The tests were all run on an unloaded Sun Sparc 10. tool speed(in MFlops) Array.py in python 0.002 for loop in python 0.03 octave 0.2 matlab 1.6 matrixobject in python w/-O 2.1 C w/-O 2.4 C w/-O4 2.6 So, the python object is within 10% of the speed of the hand-coded C. This is good enough for me. 2) Fancy arithmetic operations borrowing concepts from J. Every operator can be given a rank, and can be used to perform direct, outer, and inner products as well as reductions and accumulations. Much of the style of this part of the code is borrowed from Konrad's Array.py object. Thanks Konrad for the introduction to J! 3) Very general multidimensional slicing operation. This is based on Jim Fulton's proposal for multidimensional slicing, which those of you who have been following this list have seen. This is a superset of most other slicing approaches that I've seen. (Though I think that I need to go over Chris Chase's post on slices a little bit more carefully). 4) reshaping, general transposition, byteswaping, interface to/from strings, ... What's obviously missing: 1) Complex numbers don't work right yet. My basic problem is that I need a good complex number class in C to interface with, and I just haven't felt like writing this myself yet. 2) outer, inner, reduce, and accumulate style arithmetic functions could be optimized by a factor of 2-4 for simple cases. 3) inner product specfication is not completely clear to me when applied to operators with non-zero rank. 4) Documentation and comments in the code are minimal. 5) I'm sure that there must be a couple of memory leaks left hanging around. What remains to be argued about (in my opinion at least) 1) Matrix-style multiplication as an option. I really hate to leave the linear-algebra folks out in the cold. On the other hand, I don't like the idea of setting a flag to determine how the "*" operator will act on any given object. I'm open to suggestions. 2) Return by-value vs. by-reference. I finally made the decision that every matrix slicing function returns by-value, and that indexing by a single value returns by-reference. The by-value decision was based on a couple of bad experiences with the complexity of doing by-reference returns "right", and my observation that even in code that relied heavily on slicing a matrix I couldn't identify a performance difference that was ever greater than 20%. Having a single index return by-reference is necessary to allow typical python style "multi-dimensional" indexing (ie. a[0][0][0]) to remain efficient (this great hack is Jim Fulton's idea). 3) Type-coercion. At the moment, if I try to add a matrix of ints to a matrix of floats, I'll get an exception. I'm not at all sure that it wouldn't be better to just coerce the matrix of ints to floats and then do the add. 4) Default type of a matrix. At the moment, if I say "Matrix(1,2,3)", I'll get back a matrix of doubles. This should probably be setable with a matrix module function, so that if I'm working a lot with complex numbers, I can make this return a complex matrix by default instead, or whatever other type I like to work in. 5) Rank-system. I've really grown to like the J-style rank operators. One thing that they provide is that I can say Matrix(1,2,3)+Matrix([1,2,3],[11,12,13]) -> Matrix([2,4,6], [12,14,16]). There are those who might believe that it would be safer for this to return an error that the two matrices aren't the same size. 6) There are a couple of possible patches to "ceval.c" to modify python to deal a little bit better with an object that is a sequence type, but that doesn't implement the sequence copy method when multiplied by a scalar. These would general-purpose changes and should be completely compatible with all existing python code. At the moment, I've left this patch out. 7) Many other things that I'm sure I'm missing, which leads me to... I'd really like to have people play around with this object a bit and offer me as harsh feedback as you wish. This code is all based on Jim Fulton's implementation of a matrix object for using to interface to FORTRAN libraries, Unfortunately, his employer has a small restriction on the distribution of this code (until it becomes officially released). THIS SOFTWARE HAS *NOT* BEEN APPROVED FOR RELEASE TO THE PUBLIC. THIS SOFTWARE MAY ONLY BE PROVIDED TO INDIVIDUALS WHO ARE CO-DEVELOPERS, REVIEWERS, OR TESTERS OF THE SOFTWARE, OR TO PERSONNEL OF THE U.S. GEOLOGICAL SURVEY. So, anybody who wants to play around with this object, send me a message (at hugunin@mit.edu) saying that you agree to test and review it, I'll then point you to the appropriate FTP site. If I get some positive feedback from people saying that they like the overall design of the object, then I'll go in and write up documentation and comment the code better. 'til then I'm not sure that I wouldn't just be wasting my time. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Oct 31 12:52:48 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA00809 for matrix-sig-people; Tue, 31 Oct 1995 12:52:48 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id MAA00800 for ; Tue, 31 Oct 1995 12:52:43 -0500 Received: from nineveh.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA10495; Tue, 31 Oct 95 12:51:41 EST Received: by nineveh.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA11479; Tue, 31 Oct 95 12:53:44 -0500 Message-Id: <9510311753.AA11479@nineveh.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Tue, 31 Oct 95 12:53:43 -0500 To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Ranks of operators Sender: owner-matrix-sig@python.org Precedence: bulk Yesterday I announced an alpha version of my C-based matrix object, and I already have gotten significant feedback on it from a number of folks on the list. One topic was brought up by Konrad Hinsen that I think merits discussion by the full list. This is the issue of ranks of functions, and generalizations of functions to outer, inner, reduce, etc. And yes, I know that we've been over this before. Here's what I'm pretty sure of: 1) We need an object that can hold a basic mathematical function in a form that it can be efficiently (nearly as fast as hand-coded C) applied to a matrix of raw basic types (ints and floats). 2) Many of these are binary functions (add, subtract, etc.). These functions should be able to be applied as outer products, reductions and accumulations also efficiently (inner products as well, but these involve two binary functions, so I'll leave them out for now). 3) The basic notion of rank in J is a sound one. A n-dimensional matrix is well viewed as a k-dimensional frame of (n-k)-dimensional cells. Thus a "matrix" can be a 2d cell, or a 1d frame of 1d cells (vector of vectors) or a 2d array of 0d numbers. My new plan is the following. I want to throw this out to the list to see if there are any significant complaints about this approach before I spend a couple of days modifying my code to match this spec. add is a python object of type ofunc. This means that the add object knows how to add matrices together efficiently. add(a,b) <--> a+b Every basic object of type ofunc has the members "reduce", "outer", "accumulate", ("inner"?). These members are themselves ofuncs, however, they don't have the members given above. [This is in contrast to these being methods of the ofunc object, my initial implementation.] ie. add.reduce is an ofunc where reduce(add, m) <-> add.reduce(m) (except that the second form is a couple orders of magnitude faster). Note: add.reduce.reduce is an error Every object of type ofunc can have its rank specified using [] notation. [This is in contrast to using a keyword specifier for rank during the call.] ie. add.reduce[1] is an ofunc with rank 1. Note: add.reduce[1][0] is an ofunc with rank 0 Negative ranks are allowed (as in J). These are similar to negative indices in slices, and specify an offset from the rank of there argument. ie. add[-1]([1,2,3],[[1,2],[4,5]]) <--> add[(0,1)]([1,2,3],[[1,2],[4,5]]) I'm open to suggestions as to how the rank of inner product should be treated. Well, that's it for now. Let me know if this makes sense, or seems like a bad idea. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Oct 31 15:45:31 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA02206 for matrix-sig-people; Tue, 31 Oct 1995 15:45:31 -0500 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id PAA02202 for ; Tue, 31 Oct 1995 15:45:24 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id PAA12764 (8.6.11/IDA-1.6); Tue, 31 Oct 1995 15:43:11 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA21404; Tue, 31 Oct 1995 15:43:05 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA00284; Tue, 31 Oct 1995 15:43:04 -0500 Date: Tue, 31 Oct 1995 15:43:04 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199510312043.PAA00284@cyclone.ERE.UMontreal.CA> To: jjh@mama-bear.lcs.mit.edu CC: matrix-sig@python.org In-reply-to: <9510311753.AA11479@nineveh.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Tue, 31 Oct 95 12:53:43 -0500) Subject: Re: [PYTHON MATRIX-SIG] Ranks of operators Sender: owner-matrix-sig@python.org Precedence: bulk 1) We need an object that can hold a basic mathematical function in a form that it can be efficiently (nearly as fast as hand-coded C) applied to a matrix of raw basic types (ints and floats). Also to complex numbers... Every basic object of type ofunc has the members "reduce", "outer", "accumulate", ("inner"?). These members are themselves ofuncs, however, they don't have the members given above. OK. For "inner" I'd prefer a notation of the form add.inner.multiply(a,b) with the usual possibility of specifying ranks. It seems that this requires an explicit list of all possible combinations of binary functions somewhere, but for the built-in functions that list is not too long. Another possibility would be to make the second binary function an argument, i.e. add.inner(multiply,a,b), but I'd rather not have functions as arguments. I'm open to suggestions as to how the rank of inner product should be treated. No different from other operations. The inner product imposes a restriction on its arguments in that the dimension of the rank-1 cells of the first argument must be equal to the dimension of the rank-1 frames of the second argument, but this doesn't affect the general principle of applying ranks. If the arguments do not fit, an exception should be raised. Well, that's it for now. Let me know if this makes sense, or seems like a bad idea. It seems OK to me! ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Nov 1 13:56:41 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA04623 for matrix-sig-people; Wed, 1 Nov 1995 13:56:41 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id NAA04614 for ; Wed, 1 Nov 1995 13:56:35 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA12580; Wed, 1 Nov 95 10:56:21 PST Received: by kristen.llnl.gov (5.0/SMI-SVR4) id AA29129; Wed, 1 Nov 1995 10:56:19 +0800 Date: Wed, 1 Nov 1995 10:56:19 +0800 Message-Id: <9511011856.AA29129@kristen.llnl.gov> From: "P. Dubois" To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Some notes on the matrix class Content-Length: 3363 Sender: owner-matrix-sig@python.org Precedence: bulk This is a very nice facility already and has an amazing amount of functionality. What follows are some notes I made on it. Other members of the MATRIX-SIG have probably played enough before with the idea to realize what the examples don't show: the subscript notation for multiple dimensions works perfectly...I'm stunned. a. The already implemented list of functions/reduction operators is impressive. We have also found useful: 1. A function equal to max(m) - min(m), often used as a reduction operator in higher dimensions. We call it ptp (peak-to-peak). 2. ranf(m) fills m with uniform random numbers on (0.0, 1.0) 3. A function reshape(x,n,m,...) equal to a copy of x reshaped as (n,m,...) The decision to make reshape a status-altering method is fine, but often the need for a reshape is in the context of some temporary value needed in an expression. Some of the clever rank stuff mitigates this need but I bet this still comes in very useful. Actually, our function of this kind also duplicates copies of x as needed provided length(x) divides the new size. 4. The existing ones(n,m) will return a matrix of shape (n,m) filled with 1.0; equally useful would be one where the i,j element is 1.0 iff i==j, 0.0 otherwise. b. compress(condition, x) = those elements of x corresponding to those elements of condition that are "true". condition and x must have the same length (or x scalar), result is a one-dimensional vector with length equal to the number of true elements in condition. Naturally this is most useful if x < y returns a vector instead of a scalar. Most used form however is with a condition of the form x > 0. or some similar comparison to a scalar, which are expressible already in Python if we take "true" in this sense to be the normal Python meaning. c. My Basis users love: where(condition,x,y) is shaped like condition and has elements of x and y where condition is respectively true or false; we allow broadcast of scalars x or y, so some thought about ranks should reveal the equivalent notion here. Often used in the form where(z, x/z, 1.e99); in this case we rely on a convention that something/0.0 = Infinity, where Infinity is some variable the users can set to suit them, but anyway the idea is that the computation x/z goes at compiled speed and does not cause an exception. We haven't found this perfect and of course would prefer that the computation is simply not done at components where z is false. Given the genius you guys have already displayed ... (Yes, you can do this with a for loop, this is just a speed-freak need). d. One curiosity is the notation something[(a,b,c)]. This is required because something[a,b,c] is a syntax error. But perhaps Python could allow the latter interpreted as the former. For if you do x=[1.,2.,3.] then trying x[(0,1,2)] produces a perfectly sensible error message. And it would be in the spirit of tuples not always needing the parentheses. (That sound you hear is the shot of a large caliber weapon as Guido justifiably shoots a newbie for making a suggested language change before he can even use half the language yet; in fact, I'm tempted to shoot myself, but I'm enjoying Python too much to die yet.) ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Nov 1 14:13:08 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA04756 for matrix-sig-people; Wed, 1 Nov 1995 14:13:08 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id OAA04752 for ; Wed, 1 Nov 1995 14:13:05 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa14624; 1 Nov 95 14:10 EST Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id OAA04346; Wed, 1 Nov 1995 14:09:27 -0500 Message-Id: <199511011909.OAA04346@monty> To: "P. Dubois" cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Some notes on the matrix class In-reply-to: Your message of "Wed, 01 Nov 1995 10:56:19 +0800." <9511011856.AA29129@kristen.llnl.gov> References: <9511011856.AA29129@kristen.llnl.gov> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Wed, 01 Nov 1995 14:09:26 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk > d. One curiosity is the notation something[(a,b,c)]. This is required > because something[a,b,c] is a syntax error. But perhaps Python could > allow the latter interpreted as the former. For if you do x=[1.,2.,3.] > then trying x[(0,1,2)] produces a perfectly sensible error message. > And it would be in the spirit of tuples not always needing the parentheses. > (That sound you hear is the shot of a large caliber weapon as Guido > justifiably shoots a newbie for making a suggested language change before > he can even use half the language yet; in fact, I'm tempted to shoot > myself, but I'm enjoying Python too much to die yet.) Am I known to shoot newbies? Only in my dreams :-) This is actually a very reasonable request that has come up in this context before. It's a tad complicated to implement because there some interference with the syntax for x[i:j], but should be possible. If someone could contribute patches I'd be willing to review them and add them to the next release; otherwise it may happen slower. I expect that changes are needed only to Grammar and compile.c... --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Nov 1 15:28:20 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA05271 for matrix-sig-people; Wed, 1 Nov 1995 15:28:20 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id PAA05265 for ; Wed, 1 Nov 1995 15:28:15 -0500 Received: from nineveh.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA20086; Wed, 1 Nov 95 15:27:03 EST Received: by nineveh.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA00472; Wed, 1 Nov 95 15:28:59 -0500 Message-Id: <9511012028.AA00472@nineveh.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Wed, 1 Nov 95 15:28:58 -0500 To: "P. Dubois" Subject: Re: [PYTHON MATRIX-SIG] Some notes on the matrix class Cc: matrix-sig@python.org References: <9511011856.AA29129@kristen.llnl.gov> Sender: owner-matrix-sig@python.org Precedence: bulk > Date: Wed, 1 Nov 1995 10:56:19 +0800 > From: "P. Dubois" > > This is a very nice facility already and has an amazing amount of > functionality. Thanks for the compliment and the feedback. > 1. A function equal to max(m) - min(m), often used as a reduction operator > in higher dimensions. We call it ptp (peak-to-peak). I'll add the following to the standard functions def ptp(m): return max(m)-min(m) where within Matrix.py, max(m) <--> maximum.reduce(m) > 2. ranf(m) fills m with uniform random numbers on (0.0, 1.0) I just added a function for this, (I needed to initialize a weight matrix for a NN) it has the form, m = rand(d1,...,dn). where m will be a new matrix with the given dimensions. It uses whrandom.py for getting its random numbers so is slow. If somebody with a good random number generator in C would like to add this to matrixmodule.c as a new instantiation method in C, I'd use it. > 3. A function reshape(x,n,m,...) equal to a copy of x reshaped as (n,m,...) Already in my working copy of Matrix.py (requested by Konrad Hinsen). I use the syntax reshape(x, (d1,...dn)) so that reshape(a, b.shape) works. I don't like the idea of automatically making copies of the matrix in calls to reshape, but I have added the method m.copy(n=1) to make an arbitray number of copies of a matrix (this is much like the multiply operator for lists). (I also added a concat method noticing that I was missing this other functionality of lists). > 4. The existing ones(n,m) will return a matrix of shape (n,m) > filled with 1.0; equally useful would be one where the i,j element is > 1.0 iff i==j, 0.0 otherwise. This is now given by the eye(n,m,k=0) function (stolen from matlab), where the i,j element is 1.0 iff i-j==k, 0.0 otherwise. This is easy enough to contruct yourself out of outer products (which is what eye does). ie. identity = equal.outer(mrange(n), mrange(m)) Aren't outer products cool? > b. compress(condition, x) = those elements of x corresponding to those > elements of condition that are "true". condition and x must have > the same length (or x scalar), result is a one-dimensional vector with > length equal to the number of true elements in condition. Naturally this > is most useful if x < y returns a vector instead of a scalar. > Most used form however is with a condition of the form x > 0. or some > similar comparison to a scalar, which are expressible already in Python > if we take "true" in this sense to be the normal Python meaning. This can be expressed as reshape(x, None)[condition.nonzero()]. My personal take on this is that if condition is not a vector, then this function is meaningless. If it is a vector, it should be able to be applied to any dimension of x. I'll add the following to Matrix.py to support this: def compress(condition, m, dimension=-1): if dimension < 0: dimension = len(m.shape)+dimension if len(condition.shape)!=1 or len(condition)!=m.shape[dimension]: raise ValueError index = [All]*len(m.shape) index[dimension] = condition.nonzero() print index return m[index] BTW - while x < y returns a (meaningless) scalar, Matrix.less(x, y) returns a matrix of booleans. There has been a small amount of talk about the possibilty of having "boolean types" in addition to number, sequence, and mapping types that would support >, <, ==, etc. After writing a lot of code with matrices, I feel that many things could be expressed a lot more elegantly if this was allowed. Unfortunately, Guido appears to be skeptical of this proposal, so don't hold your breath. I've run into particular collisions with and, or, and not. These functions when applied to matrices are renamed to the ugly "Matrix.andBoolean", etc. The problem is that the former are reserved words in python, so you can't create functions with those names (even in the "safe" context of a module). > c. My Basis users love: > where(condition,x,y) is shaped like condition and has elements of x and > y where condition is respectively true or false; we allow broadcast of > scalars x or y, so some thought about ranks should reveal the equivalent > notion here. Often used in the form where(z, x/z, 1.e99); in this case > we rely on a convention that something/0.0 = Infinity, where Infinity is > some variable the users can set to suit them, but anyway the idea is that > the computation x/z goes at compiled speed and does not cause an exception. I had something like this in my original matrix proposal, but was advised that it could be expressed as without any extra C-code: def where(condition, x, y): c = notEqual(condition, 0) return c*x+(1-c)*y where the I am relying on the fact that all comparision operators return 1 for true. This is in fact about 1/5th the speed of writing a where function in C, so it should be dropped to compiled code at some time. I'll add this to the standard set of functions in Matrix.py. > We haven't found this perfect and of course would prefer that the > computation is simply not done at components where z is false. Given > the genius you guys have already displayed ... (Yes, you can do this > with a for loop, this is just a speed-freak need). I'm a speed freak too, if you ever have to use a for loop to iterate over the contents of a matrix, feel free to report it as a bug in the matrix package. -- You raise another interesting issue here that I've completely ignored in the matrix object so far. Numeric exceptions are not handled at all; in return, matrix math executes at the speed of compiled code. However, it wouldn't be too hard to add functions like safeDivide to the ofunc collection that would signal exceptions if someone considers this important. I don't even have the hack that x/0.0 = some user settable Infinity. x/0.0 is set to the IEEE value for Infinity. > d. One curiosity is the notation something[(a,b,c)]. This is required > because something[a,b,c] is a syntax error. But perhaps Python could > allow the latter interpreted as the former. Guido beat me to this one. I'd like to add that if anybody decides to go rummaging around in Grammer and compile.c, Guido also mentioned that he'd be willing to let a**b <--> pow(a, b) into the syntax. This also seems like an easy hack for someone with the right know-how, and would be a nice bonus for numerical code. There have been so many things added to the matrix object this week, that it seems I should get a new version out to those interested testers (so you won't need to ask me for features that are already there). I'll put out a new object on Friday (if all goes well). It will have all of the above additions (and many more), slightly different syntax for specifying the rank of an operator, working complex numbers, possibly working generic Python Objects as matrix elements and almost all of the memory leaks that I identified as clearly belonging to my code squashed (thanks to Purify). There are still a few memory leaks that appear to be in the python internals, and not due to my code (I think). I'll check those out when I have more time. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Nov 1 15:35:01 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA05369 for matrix-sig-people; Wed, 1 Nov 1995 15:35:01 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id PAA05365 for ; Wed, 1 Nov 1995 15:34:58 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa16169; 1 Nov 95 15:32 EST Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id PAA04571; Wed, 1 Nov 1995 15:31:19 -0500 Message-Id: <199511012031.PAA04571@monty> To: James Hugunin cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Some notes on the matrix class In-reply-to: Your message of "Wed, 01 Nov 1995 15:28:58 EST." <9511012028.AA00472@nineveh.lcs.mit.edu.LCS.MIT.EDU> References: <9511011856.AA29129@kristen.llnl.gov> <9511012028.AA00472@nineveh.lcs.mit.edu.LCS.MIT.EDU> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Wed, 01 Nov 1995 15:31:19 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk > There has been a small amount of talk about the possibilty of having > "boolean types" in addition to number, sequence, and mapping types that > would support >, <, ==, etc. After writing a lot of code with matrices, I > feel that many things could be expressed a lot more elegantly if this was > allowed. Unfortunately, Guido appears to be skeptical of this proposal, so > don't hold your breath. Couldn't this be done by implementing a "numeric" type (at the C level) that has a range of [0..1]? It could have two objects... Also, what exactly is the advantage of having a Boolean type? (Am I showing my age here? :-) Re a**b for pow(a, b): yes, that's also an acceptable addition. --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Nov 1 16:01:56 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA05599 for matrix-sig-people; Wed, 1 Nov 1995 16:01:56 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id QAA05595 for ; Wed, 1 Nov 1995 16:01:54 -0500 Received: from nineveh.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA20373; Wed, 1 Nov 95 16:00:42 EST Received: by nineveh.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA00485; Wed, 1 Nov 95 16:02:31 -0500 Message-Id: <9511012102.AA00485@nineveh.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Wed, 1 Nov 95 16:02:31 -0500 To: Guido van Rossum Subject: Re: [PYTHON MATRIX-SIG] Some notes on the matrix class Cc: matrix-sig@python.org References: <9511011856.AA29129@kristen.llnl.gov> <9511012028.AA00472@nineveh.lcs.mit.edu.LCS.MIT.EDU> <199511012031.PAA04571@monty> Sender: owner-matrix-sig@python.org Precedence: bulk > Couldn't this be done by implementing a "numeric" type (at the C > level) that has a range of [0..1]? It could have two objects... > > Also, what exactly is the advantage of having a Boolean type? (Am I > showing my age here? :-) What I'm asking for is to be able to create a new python object in C, and to include boolean methods for that object. So, just like I can implement my matrix_add function for the syntax "a+b" when a and b are matrices (modulo coercion), I'd really like to be able to implement a matrix_greater function for the syntax "a>b" when a and b are matrices (still modulo coercion). This seems completely in the original spirit of the language. The reason for this is that if a = [1,2,3], and b = [2,2,2], I'd like to define a>b = [0,0,1]. (where all of the above are matrices). This is useful in an astounding range of matrix computations, and there is no sensible definition of a>b for matrices that returns a scalar boolean value. ie. the traditional discrete delta function is elegantly expressed as def delta(i): return i==0 delta([-2,-1,0,1,2,3,4]) -> [0,0,1,0,0,0,0] It's also useful for "vectorizing" traditional operations like filter. If x is a vector, filter(lambda x: x<2 and x>0, x) <--> x.filter(x<2 and x>0) except the second is orders of magnitude faster for matrices. I understand that this is a major change to some of the more complicated parts of python, so there are good practical reasons that it won't be done anytime soon. Nevertheless, I would love though to have you agree that this would be a good feature if somebody ever has the time to add it. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Nov 1 16:15:46 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA05699 for matrix-sig-people; Wed, 1 Nov 1995 16:15:46 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id QAA05695 for ; Wed, 1 Nov 1995 16:15:44 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa16962; 1 Nov 95 16:09 EST Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id QAA06690; Wed, 1 Nov 1995 16:08:06 -0500 Message-Id: <199511012108.QAA06690@monty> To: James Hugunin cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Some notes on the matrix class In-reply-to: Your message of "Wed, 01 Nov 1995 16:02:31 EST." <9511012102.AA00485@nineveh.lcs.mit.edu.LCS.MIT.EDU> References: <9511011856.AA29129@kristen.llnl.gov> <9511012028.AA00472@nineveh.lcs.mit.edu.LCS.MIT.EDU> <199511012031.PAA04571@monty> <9511012102.AA00485@nineveh.lcs.mit.edu.LCS.MIT.EDU> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Wed, 01 Nov 1995 16:08:06 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk > The reason for this is that if a = [1,2,3], and b = [2,2,2], I'd > like to define a>b = [0,0,1]. (where all of the above are > matrices). This is useful in an astounding range of matrix > computations, and there is no sensible definition of a>b for > matrices that returns a scalar boolean value. I see that I misunderstood your question. I'm not sure I like subsuming the ">" operator (and its companions "<", "<=" etc.) for this purpose. In Python it's quite common (well, at least it's possible) to use comparisons between objects of unknown type. If I write def spam(a, b): if a < b: ...do something... else: ...do something else... and a and b can be anything. If for certain types ab is true). Currently, it is not possible for comparisons to raise exceptions. This may change in the future, however, and then matrices (like complex numbers) should raise some exception to indicate that comparing them doesn't make. There is also the comparison chaining (a URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Nov 1 16:55:27 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA06014 for matrix-sig-people; Wed, 1 Nov 1995 16:55:27 -0500 Received: from qvarsx.er.usgs.GOV (qvarsx.er.usgs.gov [130.11.51.82]) by python.org (8.6.12/8.6.12) with ESMTP id QAA06009 for ; Wed, 1 Nov 1995 16:55:22 -0500 Received: from localhost (dsjfqvarsa [130.11.51.73]) by qvarsx.er.usgs.GOV (EMAIL 1.2.1) with ESMTP id VAA04206; Wed, 1 Nov 1995 21:49:53 GMT Message-Id: <199511012149.VAA04206@qvarsx.er.usgs.GOV> To: James Hugunin cc: Guido van Rossum cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Some notes on the matrix class In-reply-to: <9511012102.AA00485@nineveh.lcs.mit.edu.LCS.MIT.EDU> Date: Wed, 01 Nov 1995 16:54:58 -0500 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk On Wed, 1 Nov 95 16:02:31 -0500 James Hugunin said: > > > Couldn't this be done by implementing a "numeric" type (at the C > > level) that has a range of [0..1]? It could have two objects... > > > > Also, what exactly is the advantage of having a Boolean type? (Am I > > showing my age here? :-) > > What I'm asking for is to be able to create a new python object in C, and > to include boolean methods for that object. So, just like I can implement > my matrix_add function for the syntax "a+b" when a and b are matrices > (modulo coercion), I'd really like to be able to implement a matrix_greater > function for the syntax "a>b" when a and b are matrices (still modulo > coercion). This seems completely in the original spirit of the language. > > The reason for this is that if a = [1,2,3], and b = [2,2,2], I'd like to > define a>b = [0,0,1]. (where all of the above are matrices). This is > useful in an astounding range of matrix computations, and there is no > sensible definition of a>b for matrices that returns a scalar boolean value. > > > ie. the traditional discrete delta function is elegantly expressed as > def delta(i): return i==0 > > delta([-2,-1,0,1,2,3,4]) -> [0,0,1,0,0,0,0] > > It's also useful for "vectorizing" traditional operations like filter. If > x is a vector, > > filter(lambda x: x<2 and x>0, x) <--> x.filter(x<2 and x>0) > > except the second is orders of magnitude faster for matrices. > > I understand that this is a major change to some of the more complicated > parts of python, so there are good practical reasons that it won't be done > anytime soon. Nevertheless, I would love though to have you agree that this > would be a good feature if somebody ever has the time to add it. I think what you want to do is best done with a separate function. I'm against having comparison operators return non-boolean values. I think comparison operators should always either return boolean, or raise an exception if an operation is not supported. With your proposal, would [1,2,3]==[0,2,3] return [0,1,1]? Then you would lose the ability to say: "if a == b:" I'd much rather be able to say a particular type supports == and != and does not support <, <=, >, and >=. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Nov 2 09:35:03 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA10552 for matrix-sig-people; Thu, 2 Nov 1995 09:35:03 -0500 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id JAA10543 for ; Thu, 2 Nov 1995 09:34:56 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id JAA00967 (8.6.11/IDA-1.6 for ); Thu, 2 Nov 1995 09:33:18 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA05977; Thu, 2 Nov 1995 09:33:17 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA13755; Thu, 2 Nov 1995 09:33:16 -0500 Date: Thu, 2 Nov 1995 09:33:16 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511021433.JAA13755@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Who wants to test my complex number module? Sender: owner-matrix-sig@python.org Precedence: bulk This does not directly relate to matrices, but I suppose that most of use are also interested in other numerical applications of Python. I have written a C module that implements complex numbers, including the standard arithmetic function, and another C module that is the analogue of math for complex numbers. Before releasing these to a larger public, I'd like some others to test them for a while. Any volunteers? The modules work with 1.2 as well as 1.3. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Nov 2 10:03:53 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA11060 for matrix-sig-people; Thu, 2 Nov 1995 10:03:53 -0500 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id KAA11056 for ; Thu, 2 Nov 1995 10:03:46 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id JAA01448 (8.6.11/IDA-1.6); Thu, 2 Nov 1995 09:46:59 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA09614; Thu, 2 Nov 1995 09:46:56 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA14200; Thu, 2 Nov 1995 09:46:55 -0500 Date: Thu, 2 Nov 1995 09:46:55 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511021446.JAA14200@cyclone.ERE.UMontreal.CA> To: dubois@kristen.llnl.gov CC: matrix-sig@python.org In-reply-to: <9511011856.AA29129@kristen.llnl.gov> (dubois@kristen.llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] Some notes on the matrix class Sender: owner-matrix-sig@python.org Precedence: bulk 2. ranf(m) fills m with uniform random numbers on (0.0, 1.0) I'd prefer a matrix constructor (similar to mrange) that creates a random matrix with a given shape. b. compress(condition, x) = those elements of x corresponding to those elements of condition that are "true". condition and x must have A useful generalization of this, especially since booleans are just integers in Python, would be a function that replicates each item of x n times, where n is the corresponding item in the "condition" argument. the same length (or x scalar), result is a one-dimensional vector with length equal to the number of true elements in condition. Naturally this Why should this be restricted to one-dimensional arguments? Of course the "condition" argument must be one-dimensional, but the compression (or copying, in the sense outlined above) can occur along any axis of a multidimensional array. is most useful if x < y returns a vector instead of a scalar. Indeed... c. My Basis users love: where(condition,x,y) is shaped like condition and has elements of x and y where condition is respectively true or false; we allow broadcast of If Python had a selection operator similar to C's ?:, this would be its generalization to array arguments. And in fact I'd love to have such an operator in Python in general... (That sound you hear is the shot of a large caliber weapon as Guido justifiably shoots a newbie for making a suggested language change before he can even use half the language yet; in fact, I'm tempted to shoot myself, but I'm enjoying Python too much to die yet.) It would be more in the spirit of Python to use poisonous snakes instead of guns ;-) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Nov 2 10:12:29 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA12341 for matrix-sig-people; Thu, 2 Nov 1995 10:12:29 -0500 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id KAA12337 for ; Thu, 2 Nov 1995 10:12:23 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id KAA02438 (8.6.11/IDA-1.6); Thu, 2 Nov 1995 10:10:12 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id KAA15180; Thu, 2 Nov 1995 10:10:10 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id KAA15138; Thu, 2 Nov 1995 10:10:08 -0500 Date: Thu, 2 Nov 1995 10:10:08 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511021510.KAA15138@cyclone.ERE.UMontreal.CA> To: jjh@mama-bear.lcs.mit.edu CC: matrix-sig@python.org In-reply-to: <9511012028.AA00472@nineveh.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Wed, 1 Nov 95 15:28:58 -0500) Subject: Re: [PYTHON MATRIX-SIG] Some notes on the matrix class Sender: owner-matrix-sig@python.org Precedence: bulk random numbers so is slow. If somebody with a good random number generator in C would like to add this to matrixmodule.c as a new instantiation method in C, I'd use it. There is no shortage of random number generators in C, although I can't find one right now on my disk... (I also added a concat method noticing that I was missing this other functionality of lists). Good. I was just about to ask for it... There has been a small amount of talk about the possibilty of having "boolean types" in addition to number, sequence, and mapping types that I don't see the advantage in a language like Python. In a type-checked compiled language, a boolean type makes sense, but in Python integers are just as good and even more useful since you can use the results of logical operations in arithmetic expressions. On the other hand, I agree it would be nice (*very* nice) to be able to define comparison operators that return matrix objects... You raise another interesting issue here that I've completely ignored in the matrix object so far. Numeric exceptions are not handled at all; in return, matrix math executes at the speed of compiled code. However, it I must say that I am not entirely happy with the fact tht Python uses exceptions for floating point operations at all. In terms of speed and versatility, I prefer to have IEEE-style operations that simply return NaN or one of the infinities as the result of an "impossible" operation. Of course there should be functions to check for these special results. Such a strategy would be easily generalizable to matrices, and on systems with IEEE implementations in hardware (practically all computers on the market now) it would not cause any speed penalty. rummaging around in Grammer and compile.c, Guido also mentioned that he'd be willing to let a**b <--> pow(a, b) into the syntax. This also seems like an easy hack for someone with the right know-how, and would be a nice bonus for numerical code. I'd love that. And once we are discussing syntax, I'd also love to have a concise input notation for complex numbers... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Nov 2 10:29:52 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA13347 for matrix-sig-people; Thu, 2 Nov 1995 10:29:52 -0500 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id KAA13338 for ; Thu, 2 Nov 1995 10:29:23 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id KAA03043 (8.6.11/IDA-1.6); Thu, 2 Nov 1995 10:27:15 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id KAA20107; Thu, 2 Nov 1995 10:27:11 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id KAA15740; Thu, 2 Nov 1995 10:27:10 -0500 Date: Thu, 2 Nov 1995 10:27:10 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511021527.KAA15740@cyclone.ERE.UMontreal.CA> To: guido@CNRI.Reston.VA.US CC: matrix-sig@python.org In-reply-to: <199511012108.QAA06690@monty> (message from Guido van Rossum on Wed, 01 Nov 1995 16:08:06 -0500) Subject: Re: [PYTHON MATRIX-SIG] Some notes on the matrix class Sender: owner-matrix-sig@python.org Precedence: bulk You can always define methods equal, greater and less, where a.greater(b) = [0,0,1] in your example. But more complicated logical expressions with such a syntax become impossible to read. In fact, I'd rather write a comparison using map() and a lambda form if it weren't so inefficient. The current state of comparisons in Python is extremely dangerous, because comparisons always succeed even if they don't make any sense. The worst consequence is that max() and min() always return a result of the expected type, but which is usually not what the user thinks it is. A perfect source for hard-to-find user errors. Of course that could be fixed by just disallowing comparisons of non-comparable objects, which however still does not solve the matrix comparison problem. Just an unfinished idea: would it be possible to construct a "matrix expression evaluator" similar in syntax to map(), but efficient? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Nov 2 11:15:22 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA14967 for matrix-sig-people; Thu, 2 Nov 1995 11:15:22 -0500 Received: from rnet.rose.rsoc.rockwell.com (rose.rsoc.rockwell.com [161.40.39.100]) by python.org (8.6.12/8.6.12) with SMTP id LAA14962 for ; Thu, 2 Nov 1995 11:15:15 -0500 Received: by rnet.rose.rsoc.rockwell.com (4.1/SMI-4.1) id AA01190; Thu, 2 Nov 95 10:14:50 CST Received: from sunrise(161.40.50.11) by rnet via smap (V1.3) id sma001187; Thu Nov 2 10:14:08 1995 Received: from darwin.rsoc.rockwell.com by rose.rsoc.rockwell.com (5.0/SMI-SVR4) id AA23457; Thu, 2 Nov 1995 10:14:07 -0600 Received: by darwin.rsoc.rockwell.com (5.0/SMI-SVR4) id AA23987; Thu, 2 Nov 1995 10:14:05 -0600 Date: Thu, 2 Nov 1995 10:14:05 -0600 From: friedric@rose.rsoc.rockwell.com (Robin Friedrich) Message-Id: <9511021614.AA23987@darwin.rsoc.rockwell.com> To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Some notes on the matrix class X-Sun-Charset: US-ASCII Content-Length: 257 Sender: owner-matrix-sig@python.org Precedence: bulk > From: hinsenk@ere.umontreal.ca (Hinsen Konrad) |> It would be more in the spirit of Python to use poisonous snakes instead |> of guns ;-) A 16-ton weight actually. (sorry for the noise but this list has way too much signal!) keep up the good work. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Nov 2 12:58:20 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA16541 for matrix-sig-people; Thu, 2 Nov 1995 12:58:20 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id MAA16537 for ; Thu, 2 Nov 1995 12:58:15 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA01318; Thu, 2 Nov 95 09:57:46 PST Received: by kristen.llnl.gov (5.0/SMI-SVR4) id AA03825; Thu, 2 Nov 1995 09:57:44 +0800 Date: Thu, 2 Nov 1995 09:57:44 +0800 Message-Id: <9511021757.AA03825@kristen.llnl.gov> From: "P. Dubois" To: hinsenk@ere.umontreal.ca Cc: jjh@mama-bear.lcs.mit.edu, matrix-sig@python.org In-Reply-To: <199511021510.KAA15138@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Subject: [PYTHON MATRIX-SIG] floating point handling Content-Length: 789 Sender: owner-matrix-sig@python.org Precedence: bulk (1) I disagree with Konrad's remarks on floating point. I know of no production code that allows the generation of NaNs or any of the other IEEE nonsense. If such a thing occurs in one of our codes it is an error and we need to hear about it immediately. Knowing *where* the problem first occurred is crucial. We use whatever facilities are provided on a given platform (and sometimes there aren't any) to make sure that: a. Any floating point problems signal FPE (which we then catch), except: b. Fast-underflow-to-zero is turned on (Can make a real speed difference: if you don't do something about it, underflows result in system calls.) There is no standardization in this area. (2) I will supply a module that is equivalent to the Cray random number generator. Stand by. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 3 06:36:33 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id GAA25390 for matrix-sig-people; Fri, 3 Nov 1995 06:36:33 -0500 Received: from hermes.dur.ac.uk (hermes.dur.ac.uk [129.234.4.9]) by python.org (8.6.12/8.6.12) with ESMTP id GAA25384 for ; Fri, 3 Nov 1995 06:36:14 -0500 From: P.S.Craig@durham.ac.uk Received: from gauss.durham.ac.uk by hermes.dur.ac.uk id (8.6.12/ for dur.ac.uk) with SMTP; Fri, 3 Nov 1995 09:39:58 GMT Received: from markov (markov.dur) by uk.ac.durham.gauss; Fri, 3 Nov 95 09:39:57 GMT Message-Id: <26057.9511030939@markov> Subject: Re: [PYTHON MATRIX-SIG] floating point handling To: dubois@kristen.llnl.gov (P. Dubois) Date: Fri, 3 Nov 1995 09:38:16 +0000 (GMT) In-Reply-To: <9511021757.AA03825@kristen.llnl.gov> from "P. Dubois" at Nov 2, 95 09:57:44 am X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 1223 Sender: owner-matrix-sig@python.org Precedence: bulk P. Dubois writes: > > > (1) I disagree with Konrad's remarks on floating point. > > I know of no production code that allows the generation of NaNs or any > of the other IEEE nonsense. If such a thing occurs in one of our codes > it is an error and we need to hear about it immediately. Knowing *where* > the problem first occurred is crucial. > This is getting a bit silly. Whether the IEEE `nonsense' is appropriate or not is context dependent. I make heavy use of S-Plus (a statistics package with some APL like matrix addressing) and Matlab (a widely used matrix arithmetic package with many practical applications). Both of these return Inf for 1/0 and Nan for 0/0 and I don't think they have to generate exceptions to do it. It seems quite sensible to me that python should allow both modes of operation, i.e. IEEE extended arithmetic or exceptions. And if that means significant changes to the base language, I have to apologise to Guido for creating work. The problem for me is that python is a much better programming language than S-Plus or Matlab but that I need this matrix/tensor class to be as like them as possible before I switch to using it instead of them for my research. Cheers, Peter Craig ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 3 08:26:22 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id IAA25896 for matrix-sig-people; Fri, 3 Nov 1995 08:26:22 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id IAA25892 for ; Fri, 3 Nov 1995 08:26:17 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa08268; 3 Nov 95 8:22 EST Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id IAA16812; Fri, 3 Nov 1995 08:20:59 -0500 Message-Id: <199511031320.IAA16812@monty> To: P.S.Craig@durham.ac.uk cc: "P. Dubois" , matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] floating point handling In-reply-to: Your message of "Fri, 03 Nov 1995 09:38:16 GMT." <26057.9511030939@markov> References: <26057.9511030939@markov> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Fri, 03 Nov 1995 08:20:58 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk > It seems quite sensible to me that python should allow both modes of > operation, i.e. IEEE extended arithmetic or exceptions. And if that > means significant changes to the base language, I have to apologise to > Guido for creating work. The problem for me is that python is a much > better programming language than S-Plus or Matlab but that I need this > matrix/tensor class to be as like them as possible before I switch to > using it instead of them for my research. If you insist on having "Inf" for infinity and "NaN" for 1/0, all you probably need to do is hack the floatobject.c module a wee bit, to include representations for them and to return them in appropriate cases. I don't mind work but I have about 1,000,000 different projects to attend to (two of them are due today :-) and things go faster if people contribute patches instead of requesating changes. (Not that a contributed patch will automatically end up in the language -- that depends on the quality of the code and how well your feel for what's "right" in Python agrees with mine...) --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 3 10:02:21 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA26486 for matrix-sig-people; Fri, 3 Nov 1995 10:02:21 -0500 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id KAA26480 for ; Fri, 3 Nov 1995 10:02:16 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id JAA03652 (8.6.11/IDA-1.6); Fri, 3 Nov 1995 09:59:17 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA19512; Fri, 3 Nov 1995 09:59:14 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA18261; Fri, 3 Nov 1995 09:59:13 -0500 Date: Fri, 3 Nov 1995 09:59:13 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511031459.JAA18261@cyclone.ERE.UMontreal.CA> To: guido@CNRI.Reston.VA.US CC: matrix-sig@python.org In-reply-to: <199511031320.IAA16812@monty> (message from Guido van Rossum on Fri, 03 Nov 1995 08:20:58 -0500) Subject: Re: [PYTHON MATRIX-SIG] floating point handling Sender: owner-matrix-sig@python.org Precedence: bulk If you insist on having "Inf" for infinity and "NaN" for 1/0, all you probably need to do is hack the floatobject.c module a wee bit, to include representations for them and to return them in appropriate Indeed I don't think this is much work, having looked at the float implementation quite a bit before writing the complex module. But obviously there must be a way to enable or disable floating point exceptions (I thing everyone agrees they are useful). I am not quite sure what the appropriate way to do that would be - a variable in the sys module would be nice, but probably expensive to check after each float operation. A function with a boolean argument that sets a C variable might be better. Or maybe there is a better way that I just can't think of... The same flag could then be used in the complex and matrix modules. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 3 10:20:50 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA26591 for matrix-sig-people; Fri, 3 Nov 1995 10:20:50 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id KAA26587 for ; Fri, 3 Nov 1995 10:20:45 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa10648; 3 Nov 95 10:10 EST Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id KAA17260; Fri, 3 Nov 1995 10:08:40 -0500 Message-Id: <199511031508.KAA17260@monty> To: Hinsen Konrad cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] floating point handling In-reply-to: Your message of "Fri, 03 Nov 1995 09:59:13 EST." <199511031459.JAA18261@cyclone.ERE.UMontreal.CA> References: <199511031459.JAA18261@cyclone.ERE.UMontreal.CA> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Fri, 03 Nov 1995 10:08:39 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk > If you insist on having "Inf" for infinity and "NaN" for 1/0, all you > probably need to do is hack the floatobject.c module a wee bit, to > include representations for them and to return them in appropriate > > Indeed I don't think this is much work, having looked at the float > implementation quite a bit before writing the complex module. But > obviously there must be a way to enable or disable floating point > exceptions (I thing everyone agrees they are useful). I am not quite > sure what the appropriate way to do that would be - a variable in the > sys module would be nice, but probably expensive to check after each > float operation. A function with a boolean argument that sets a C > variable might be better. Or maybe there is a better way that I just > can't think of... The same flag could then be used in the complex and > matrix modules. Hmm... Globals that affect the behavior of low-level objects are rarely a good idea. For instance, some non-compute-intensive module may be using floats and expecting exceptions -- it will break if it is used in your program that turns off exceptions... How about introducing a new type, say ieeefloat, with the desired behavior? Its casting behavior could easily be designed to force mixed-mode arithmetic (between standard and ieee floats) to be done in ieee mode. --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 3 10:36:35 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA26672 for matrix-sig-people; Fri, 3 Nov 1995 10:36:35 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id KAA26668 for ; Fri, 3 Nov 1995 10:36:30 -0500 Received: from nineveh.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA04812; Fri, 3 Nov 95 10:34:52 EST Received: by nineveh.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA03746; Fri, 3 Nov 95 10:37:03 -0500 Message-Id: <9511031537.AA03746@nineveh.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Fri, 3 Nov 95 10:37:02 -0500 To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] IEEE exceptions, casting, matrix multiplies, and default types References: <199511031459.JAA18261@cyclone.ERE.UMontreal.CA> Sender: owner-matrix-sig@python.org Precedence: bulk I'd like to generalize this discussion a little bit. I've noticed four areas where the easiest solution would be to add a user-settable global variable to indicate a "desired" mode, yet I agree with Guido that as a general rule this is an extremely bad idea. 1) 1/0. returns Inf or raises exception [Note: Matrix_i creates a matrix of ints, Matrix_d creates one of doubles] 2) Matrix_i(1,2,3) * Matrix_d(1.,2.,3.) == Matrix_d(1.,4.,6.) or raises exception 3) a * b is element-wise or is linear algebra style multiply 4) add([1,2,3], [4,5,6]) == Matrix_?(5,7,9) where the question is what the ? should be. I'm busy this morning getting the 0.11 release of the matrixobject together, so I don't have time for much more comment, I just wanted to point out that this is a very general question (in particular when thinking about matrices), and if we're really lucky there might be a nice general purpose answer. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 3 12:02:51 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA27203 for matrix-sig-people; Fri, 3 Nov 1995 12:02:51 -0500 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id MAA27199 for ; Fri, 3 Nov 1995 12:02:45 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id MAA08285 (8.6.11/IDA-1.6); Fri, 3 Nov 1995 12:00:19 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA20667; Fri, 3 Nov 1995 12:00:18 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA25356; Fri, 3 Nov 1995 12:00:14 -0500 Date: Fri, 3 Nov 1995 12:00:14 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511031700.MAA25356@cyclone.ERE.UMontreal.CA> To: guido@CNRI.Reston.VA.US CC: matrix-sig@python.org In-reply-to: <199511031508.KAA17260@monty> (message from Guido van Rossum on Fri, 03 Nov 1995 10:08:39 -0500) Subject: Re: [PYTHON MATRIX-SIG] floating point handling Sender: owner-matrix-sig@python.org Precedence: bulk Hmm... Globals that affect the behavior of low-level objects are rarely a good idea. For instance, some non-compute-intensive module may be using floats and expecting exceptions -- it will break if it is used in your program that turns off exceptions... That's indeed a problem. In fact this is a well-known problem for APL users, since APL has a range of global variables that affect the meaning of all functions (i.e. the index origin can be set to zero or one). The solution that APL offers is that these variables can be made local and thereby changed for an individual function, but that is far from an ideal solution. How about introducing a new type, say ieeefloat, with the desired behavior? Its casting behavior could easily be designed to force mixed-mode arithmetic (between standard and ieee floats) to be done in ieee mode. I am sure two float types would create quite a bit of confusion for users, especially those who don't care about error handling and don't even want to understand the difference. For example, there would have to be two different input and output notations to make clear which version is used. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 3 12:44:02 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA27488 for matrix-sig-people; Fri, 3 Nov 1995 12:44:02 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id MAA27484 for ; Fri, 3 Nov 1995 12:43:59 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA14249; Fri, 3 Nov 95 09:42:48 PST Received: by kristen.llnl.gov (5.0/SMI-SVR4) id AA09062; Fri, 3 Nov 1995 09:42:46 +0800 Date: Fri, 3 Nov 1995 09:42:46 +0800 Message-Id: <9511031742.AA09062@kristen.llnl.gov> From: "P. Dubois" To: hinsenk@ere.umontreal.ca, matrix-sig@python.org In-Reply-To: <199511031505.KAA18617@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA) Subject: Re: [PYTHON MATRIX-SIG] Who wants to test my complex number module? Content-Length: 14417 Sender: owner-matrix-sig@python.org Precedence: bulk (This is the middle of a conversation about the complex module that Hinsen has developed, but some issues I wanted to mention to everyone). a. It is a surprising thing (to me) that while tan is in the Fortran standard, cot isn't. We put one in Basis because it drove us crazy. b. The conjugation function in Fortran is called conjg, not conj. We probably should have a functional equivalent, too, conjg(c). Actually, the conjugate not really an attribute of the object, it is a transform, and so I'd rather see a method which conjugates the given object and a conjg(c) which returns copy(c).conjg(). c. However, that leads us to the whole discussion of naming. I think if you read "Reusable Software" by Bertrand Meyer, and have some experience with constructing large scale reusable component libraries (which is what Python is generating on a frightening scale) it turns out that a consistent naming convention is very, very important. Some of the lessons the Eiffel community has learned are: 1. Abbreviations should be avoided. Basically, if you don't abbreviate you can't forget which abbreviation to use. So in the immediate case, I would favor conjugate rather than conjg. This tends to be against the traditions in a rapid prototyping language but the components we are building aren't the rapid prototype they are elements of the reusable software library. (Although, you can make a case that conjg isn't an abbreviation, it is the Fortran name which everyone knows!) 2. It is more important for the names to be consistent than traditional. For example, the size of an object like a list, stack, or queue is invariably called count, and the operation to place an element at a certain position is put. In stacks, put might more traditionally be called "push" and the top item "top" or "first" but an Eiffel user will know without looking that the top item is going to be called "item", as is the current item in a list. Fortunately, Python's name scoping mitigates quite a bit of the worry about name clashes, but the problem of a user learning a large body of components is still there. d. In the matrix class, one ought to worry about putting a name like "pp". Not only are some users going to accidentally call one of their variables pp, when you read it you haven't a clue what it is. PrintMatrix ? (Unless, has pp become a standard Python abbreviation for PrettyPrint?) e. About the printed representation of a complex: shouldn't it be complex(1.0, 1.1) not 1j1.1 ? There are two components to this thought. One is that the two parts should be presented with equal precision. Second is that it might come in handy if the printed form was readable back into Python, and there seems to be some "tradition" of this sort of convention in Python components already. The Fortran representation for code literals is (1.0, 1.1) which is also suggestive. f. Note to ourselves: the printing of large matrices needs to be more orderly and labeled. An example follows from Basis. Note the control variable available to change the printing precision on a "class-wide" basis. kristen[51] basis Basis (basis, Version 951101) Run at 09:20:46 on 11/03/95 on the sun4m machine, suffix 8544x Initializing Basis System Basis 11.0 Initializing 3-D Surface Plotting Routine Initializing 3-D Isosurface Plotting Routine Initializing Device Package EZD Graphics Devices 3.4 NCAR Initializing EZCURVE/NCAR Graphics EZN Graphics Interface Version 5.5 Initializing Singular Value Decomposition Initializing Polynomial Fitting Initializing PFB Interface PFB 4.1 Basis> real x = ones(5, 20) Basis> x x shape: (5,20) row col = 1 2 3 4 1: 1.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 2: 0.00000E+00 1.00000E+00 0.00000E+00 0.00000E+00 3: 0.00000E+00 0.00000E+00 1.00000E+00 0.00000E+00 4: 0.00000E+00 0.00000E+00 0.00000E+00 1.00000E+00 5: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 row col = 5 6 7 8 1: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 2: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 3: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 4: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 5: 1.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 row col = 9 10 11 12 1: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 2: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 3: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 4: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 5: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 row col = 13 14 15 16 1: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 2: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 3: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 4: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 5: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 row col = 17 18 19 20 1: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 2: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 3: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 4: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 5: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 Basis> [x, x+2.] [x,x+2.] shape: (5,20,2) Number of dimensions: 3, lengths: 5 20 2 INDEX(row,col,1) row col = 1 2 3 4 1: 1.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 2: 0.00000D+00 1.00000D+00 0.00000D+00 0.00000D+00 3: 0.00000D+00 0.00000D+00 1.00000D+00 0.00000D+00 4: 0.00000D+00 0.00000D+00 0.00000D+00 1.00000D+00 5: 0.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 row col = 5 6 7 8 1: 0.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 2: 0.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 3: 0.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 4: 0.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 5: 1.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 row col = 9 10 11 12 1: 0.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 2: 0.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 3: 0.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 4: 0.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 5: 0.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 row col = 13 14 15 16 1: 0.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 2: 0.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 3: 0.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 4: 0.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 5: 0.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 row col = 17 18 19 20 1: 0.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 2: 0.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 3: 0.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 4: 0.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 5: 0.00000D+00 0.00000D+00 0.00000D+00 0.00000D+00 INDEX(row,col,2) row col = 1 2 3 4 1: 3.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 2: 2.00000D+00 3.00000D+00 2.00000D+00 2.00000D+00 3: 2.00000D+00 2.00000D+00 3.00000D+00 2.00000D+00 4: 2.00000D+00 2.00000D+00 2.00000D+00 3.00000D+00 5: 2.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 row col = 5 6 7 8 1: 2.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 2: 2.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 3: 2.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 4: 2.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 5: 3.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 row col = 9 10 11 12 1: 2.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 2: 2.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 3: 2.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 4: 2.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 5: 2.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 row col = 13 14 15 16 1: 2.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 2: 2.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 3: 2.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 4: 2.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 5: 2.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 row col = 17 18 19 20 1: 2.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 2: 2.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 3: 2.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 4: 2.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 5: 2.00000D+00 2.00000D+00 2.00000D+00 2.00000D+00 ================================================ Basis> fuzz=14; x x shape: (5,20) row col = 1 2 1: 1.00000000000000E+00 0.00000000000000E+00 2: 0.00000000000000E+00 1.00000000000000E+00 3: 0.00000000000000E+00 0.00000000000000E+00 4: 0.00000000000000E+00 0.00000000000000E+00 5: 0.00000000000000E+00 0.00000000000000E+00 row col = 3 4 1: 0.00000000000000E+00 0.00000000000000E+00 2: 0.00000000000000E+00 0.00000000000000E+00 3: 1.00000000000000E+00 0.00000000000000E+00 4: 0.00000000000000E+00 1.00000000000000E+00 5: 0.00000000000000E+00 0.00000000000000E+00 row col = 5 6 1: 0.00000000000000E+00 0.00000000000000E+00 2: 0.00000000000000E+00 0.00000000000000E+00 3: 0.00000000000000E+00 0.00000000000000E+00 4: 0.00000000000000E+00 0.00000000000000E+00 5: 1.00000000000000E+00 0.00000000000000E+00 row col = 7 8 1: 0.00000000000000E+00 0.00000000000000E+00 2: 0.00000000000000E+00 0.00000000000000E+00 3: 0.00000000000000E+00 0.00000000000000E+00 4: 0.00000000000000E+00 0.00000000000000E+00 5: 0.00000000000000E+00 0.00000000000000E+00 row col = 9 10 1: 0.00000000000000E+00 0.00000000000000E+00 2: 0.00000000000000E+00 0.00000000000000E+00 3: 0.00000000000000E+00 0.00000000000000E+00 4: 0.00000000000000E+00 0.00000000000000E+00 5: 0.00000000000000E+00 0.00000000000000E+00 row col = 11 12 1: 0.00000000000000E+00 0.00000000000000E+00 2: 0.00000000000000E+00 0.00000000000000E+00 3: 0.00000000000000E+00 0.00000000000000E+00 4: 0.00000000000000E+00 0.00000000000000E+00 5: 0.00000000000000E+00 0.00000000000000E+00 row col = 13 14 1: 0.00000000000000E+00 0.00000000000000E+00 2: 0.00000000000000E+00 0.00000000000000E+00 3: 0.00000000000000E+00 0.00000000000000E+00 4: 0.00000000000000E+00 0.00000000000000E+00 5: 0.00000000000000E+00 0.00000000000000E+00 row col = 15 16 1: 0.00000000000000E+00 0.00000000000000E+00 2: 0.00000000000000E+00 0.00000000000000E+00 3: 0.00000000000000E+00 0.00000000000000E+00 4: 0.00000000000000E+00 0.00000000000000E+00 5: 0.00000000000000E+00 0.00000000000000E+00 row col = 17 18 1: 0.00000000000000E+00 0.00000000000000E+00 2: 0.00000000000000E+00 0.00000000000000E+00 3: 0.00000000000000E+00 0.00000000000000E+00 4: 0.00000000000000E+00 0.00000000000000E+00 5: 0.00000000000000E+00 0.00000000000000E+00 row col = 19 20 1: 0.00000000000000E+00 0.00000000000000E+00 2: 0.00000000000000E+00 0.00000000000000E+00 3: 0.00000000000000E+00 0.00000000000000E+00 4: 0.00000000000000E+00 0.00000000000000E+00 5: 0.00000000000000E+00 0.00000000000000E+00 "Compression" is also useful, can be turned off by user: Basis> real x= [1.,2.,3] // ones(1000) Basis> x x shape: (1003) 1: 1.00000000000000E+00 2.00000000000000E+00 3: 3.00000000000000E+00 1.00000000000000E+00 5:1000 1.00000000000000E+00 1.00000000000000E+00 1001: 1.00000000000000E+00 1.00000000000000E+00 1003: 1.00000000000000E+00 Basis> [x,x/2.] [x,x/2.] shape: (1003,2) row col = 1 2 1: 1.00000000000000D+00 5.00000000000000D-01 2: 2.00000000000000D+00 1.00000000000000D+00 3: 3.00000000000000D+00 1.50000000000000D+00 4:1003 1.00000000000000D+00 5.00000000000000D-01 Basis> ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 3 13:06:47 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA27633 for matrix-sig-people; Fri, 3 Nov 1995 13:06:47 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id NAA27629 for ; Fri, 3 Nov 1995 13:06:41 -0500 Received: from nineveh.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA05894; Fri, 3 Nov 95 13:04:57 EST Received: by nineveh.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA08561; Fri, 3 Nov 95 13:07:05 -0500 Message-Id: <9511031807.AA08561@nineveh.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Fri, 3 Nov 95 13:07:04 -0500 To: "P. Dubois" Subject: [PYTHON MATRIX-SIG] Naming conventions Cc: matrix-sig@python.org References: <9511031742.AA09062@kristen.llnl.gov> Sender: owner-matrix-sig@python.org Precedence: bulk > a. It is a surprising thing (to me) that while tan is in the Fortran standard, > cot isn't. We put one in Basis because it drove us crazy. I completely agree that we should expand upon the basic mathematical functions, not feeling limited by the current math.h C-library. > c. However, that leads us to the whole discussion of naming. I think if > you read "Reusable Software" by Bertrand Meyer, and have some experience > with constructing large scale reusable component libraries (which is > what Python is generating on a frightening scale) it turns out that > a consistent naming convention is very, very important. Some of the > lessons the Eiffel community has learned are: > > 1. Abbreviations should be avoided. Basically, if you don't abbreviate > you can't forget which abbreviation to use. So in the immediate case, > I would favor conjugate rather than conjg. This tends to be against > the traditions in a rapid prototyping language but the components > we are building aren't the rapid prototype they are elements of > the reusable software library. (Although, you can make a case that > conjg isn't an abbreviation, it is the Fortran name which everyone > knows!) As somebody who hates FORTRAN, I'd really like to argue strongly against assuming that everybody knows a FORTRAN name. I'd like to second your motion for "conjugate". I've been trying to follow exactly the convention that you mention for naming my basic ofuncs, thus I have "multiply", "divide", "leftShift", "andBoolean", "andLogical", and even things like "absolute", and "power" though I can see arguments for "pow" and "abs". > e. About the printed representation of a complex: shouldn't it be > complex(1.0, 1.1) not 1j1.1 ? There are two components to this thought. > One is that the two parts should be presented with equal precision. > Second is that it might come in handy if the printed form was readable > back into Python, and there seems to be some "tradition" of this sort > of convention in Python components already. The Fortran representation > for code literals is (1.0, 1.1) which is also suggestive. I believe that what's going on here is that Konrad is hoping that something like 1.0j1.0 (note the reasonable precision) will become acceptable as an input form to python for creating complex numbers. I know that printing out a matrix that has a lot of complex(1.0, 1.0) entries is hideous, so I'm in favor of this idea. > d. In the matrix class, one ought to worry about putting a name like > "pp". Not only are some users going to accidentally call one of their > variables pp, when you read it you haven't a clue what it is. > PrintMatrix ? (Unless, has pp become a standard Python abbreviation for > PrettyPrint?) > f. Note to ourselves: the printing of large matrices needs to be more > orderly and labeled. I agree, pp is hideous. It was only a hack so that I could easily write sample code and have the matrices printed out looking half-way decent. The whole issue of printing matrices needs to be addressed. One part of this issue is the desire that the basic printed representation of an object should correspond to a command that could be used to recreate that object, on the other hand, the well-labeled format you show from BASIS is generally what a user wants to see. I'm ambivalent on this issue. If somebody gives me a good routine for printing a matrix I'd love to add it to my code. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 3 13:57:20 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA28116 for matrix-sig-people; Fri, 3 Nov 1995 13:57:20 -0500 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id NAA28112 for ; Fri, 3 Nov 1995 13:57:17 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id NAA12188 (8.6.11/IDA-1.6); Fri, 3 Nov 1995 13:54:55 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA19028; Fri, 3 Nov 1995 13:54:54 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA02955; Fri, 3 Nov 1995 13:54:51 -0500 Date: Fri, 3 Nov 1995 13:54:51 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511031854.NAA02955@cyclone.ERE.UMontreal.CA> To: dubois@kristen.llnl.gov CC: matrix-sig@python.org In-reply-to: <9511031742.AA09062@kristen.llnl.gov> (dubois@kristen.llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] Who wants to test my complex number module? Sender: owner-matrix-sig@python.org Precedence: bulk a. It is a surprising thing (to me) that while tan is in the Fortran standard, cot isn't. We put one in Basis because it drove us crazy. I haven't ever had a pressing need for cot(), but since it costs virtually nothing to add such simple functions, there is little reason not to do so. I already added the inverse hyperbolic functions to my cmath module, because it seemed silly not to have them. I am perfectly willing to add more functions, also to the float math module. This leads to the question of what functions would be desirable. In addition to the full set of angular functions, I'd like to see the error function and the gamma function. Anything else? b. The conjugation function in Fortran is called conjg, not conj. We probably should have a functional equivalent, too, conjg(c). Actually, the conjugate not really an attribute of the object, it is a transform, and so I'd rather see a method which conjugates the given object and a conjg(c) which returns copy(c).conjg(). I had already expected that objection. In fact the only reason why I implemented conjugation as an attribute (which indeed it should not be) is that I didn't want to add functions to the complex module. I envisioned that maybe one day one could make complex numbers a standard part of Python, like floats, with their proper input notation, and then there would be no more complex module. It could be put into cmath, but it doesn't really belong there. I am open to any suggestion on where to put this function... 1. Abbreviations should be avoided. Basically, if you don't abbreviate you can't forget which abbreviation to use. So in the immediate case, That is also the position of Mathematica, and it works well in practice. The only "safe" abbreviations are those long established in mathematical notation, like "sin" or "log". On the other hand, does anyone really want SquareRoot()? Even Mathematica uses Sqrt[]. I think one needs a certain (very small) set of accepted abbreviations. The point is not just ease of typing, but also keeping moderately complex expressions short enough to be legible. the reusable software library. (Although, you can make a case that conjg isn't an abbreviation, it is the Fortran name which everyone knows!) I'd rather not make the assumption that everyone knows Fortran! In fact, I'd be happy if I could one day forget Fortran. e. About the printed representation of a complex: shouldn't it be complex(1.0, 1.1) not 1j1.1 ? There are two components to this thought. For my taste complex(1.0,1.1) is too long, especially in matrices. One is that the two parts should be presented with equal precision. Why? I find 3.1415926j0 much nicer than 3.1415926j0.0000000 (and shorter too). Second is that it might come in handy if the printed form was readable back into Python, and there seems to be some "tradition" of this sort Of course. My hope is that the output notation I use (which by the way is stolen from APL2 and J) could become the accepted input notation one day. of convention in Python components already. The Fortran representation for code literals is (1.0, 1.1) which is also suggestive. But it can never become an input notation, since (1.0, 1.1) is read as a tuple in Python. Even for output only it would be confusing. For flexible output, there should be some kind of formatting option, as it exists for floating point numbers. But that can't be implemented in a module, so it doesn't exist for now. f. Note to ourselves: the printing of large matrices needs to be more orderly and labeled. Definitely. Again APL is a good example to copy. Basis> real x = ones(5, 20) Basis> x x shape: (5,20) row col = 1 2 3 4 1: 1.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 2: 0.00000E+00 1.00000E+00 0.00000E+00 0.00000E+00 3: 0.00000E+00 0.00000E+00 1.00000E+00 0.00000E+00 4: 0.00000E+00 0.00000E+00 0.00000E+00 1.00000E+00 5: 0.00000E+00 0.00000E+00 0.00000E+00 0.00000E+00 ... I am not sure I'd like this to be the default output format. The labels are nice for big matrices, but for small matrices I'd prefer just the elements. Also, I'd prefer non-exponential notations as long as it is reasonable for all elements (as in this case). APL systems tend to choose the optimal output format based on many properties of the matrix, and I have really come to appreciate that. Unfortunately, J doesn't follow this tradition. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 3 14:05:48 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA28168 for matrix-sig-people; Fri, 3 Nov 1995 14:05:48 -0500 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id OAA28164 for ; Fri, 3 Nov 1995 14:05:43 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id OAA12505 (8.6.11/IDA-1.6); Fri, 3 Nov 1995 14:02:37 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA21488; Fri, 3 Nov 1995 14:02:35 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA03346; Fri, 3 Nov 1995 14:02:33 -0500 Date: Fri, 3 Nov 1995 14:02:33 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511031902.OAA03346@cyclone.ERE.UMontreal.CA> To: jjh@mama-bear.lcs.mit.edu CC: matrix-sig@python.org In-reply-to: <9511031807.AA08561@nineveh.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Fri, 3 Nov 95 13:07:04 -0500) Subject: Re: [PYTHON MATRIX-SIG] Naming conventions Sender: owner-matrix-sig@python.org Precedence: bulk whole issue of printing matrices needs to be addressed. One part of this issue is the desire that the basic printed representation of an object should correspond to a command that could be used to recreate that object, on the other hand, the well-labeled format you show from BASIS is generally what a user wants to see. I'm ambivalent on this issue. If somebody gives Python has repr() and str(), so we can have both. My suggestion is an input-like notation for repr(), and a human-readable one for str() and therefore print. The fact is that for matrices it doesn't make sense to use the same format for input and output. Matrix constants will only be used for small matrices, and they must consist of some linear arrangement. But much larger matrices will be generated by other means, and they have to be printed in a readable way. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 3 14:50:46 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA28473 for matrix-sig-people; Fri, 3 Nov 1995 14:50:46 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id OAA28466 for ; Fri, 3 Nov 1995 14:50:40 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA15918; Fri, 3 Nov 95 11:50:01 PST Received: by kristen.llnl.gov (5.0/SMI-SVR4) id AA13959; Fri, 3 Nov 1995 11:49:54 +0800 Date: Fri, 3 Nov 1995 11:49:54 +0800 Message-Id: <9511031949.AA13959@kristen.llnl.gov> From: "P. Dubois" To: hinsenk@ere.umontreal.ca Cc: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] printing complexes Content-Length: 1263 Sender: owner-matrix-sig@python.org Precedence: bulk That was an excellent point you just made about str() and repr(). I think. I'm so new to Python I'm not sure which is which (:-> The reason I was worried about the precision being the same on the two sides (and controllable) is simply the issue of making nice output where you want things to line up for comparison. It is jarring if you have a list and successive elements are all different in appearance, and it makes it harder to make a nice table. I didn't know about the J/APL2 convention, thanks for explaining it. FYI, in Basis I don't have complex constants, just IMAGINARY ones. You do things like: 1i 1i = ( 0.00000D+00, 1.00000D+00) Basis> 3 + 2.5i 3+2.5i = ( 3.00000D+00, 2.50000D+00) Basis> As you see, I use the Fortran printing convention which is, of course, unacceptable in Python. But in array printing I don't, I use spacing: iota(10) + 2i iota(10)+2i shape: (10) 1: 1.00000D+00, 2.00000D+00 2.00000D+00, 2.00000D+00 3: 3.00000D+00, 2.00000D+00 4.00000D+00, 2.00000D+00 5: 5.00000D+00, 2.00000D+00 6.00000D+00, 2.00000D+00 7: 7.00000D+00, 2.00000D+00 8.00000D+00, 2.00000D+00 9: 9.00000D+00, 2.00000D+00 1.00000D+01, 2.00000D+00 Basis> ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 3 16:15:26 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA29116 for matrix-sig-people; Fri, 3 Nov 1995 16:15:26 -0500 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id QAA29112 for ; Fri, 3 Nov 1995 16:15:23 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id QAA16854 (8.6.11/IDA-1.6); Fri, 3 Nov 1995 16:12:57 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA27971; Fri, 3 Nov 1995 16:12:54 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA10966; Fri, 3 Nov 1995 16:12:53 -0500 Date: Fri, 3 Nov 1995 16:12:53 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511032112.QAA10966@cyclone.ERE.UMontreal.CA> To: dubois@kristen.llnl.gov CC: matrix-sig@python.org In-reply-to: <9511031949.AA13959@kristen.llnl.gov> (dubois@kristen.llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] printing complexes Sender: owner-matrix-sig@python.org Precedence: bulk The reason I was worried about the precision being the same on the two sides (and controllable) is simply the issue of making nice output where you want things to line up for comparison. It is jarring if you have a list and successive elements are all different in appearance, and it makes it harder to make a nice table. For matrices (and possibly sequence types in general) this is indeed an issue. But what should be equal is not the precision of the real and imaginary part of each number, but the real parts of all numbers and the imaginary parts of all numbers. My suggestion for formatting float and complex numbers in matrices is the following: Define an "order" of output formats as 1) integer 2) non-exponential floating point 3) exponential notation Next, determine the appropriate format for each entry of the matrix, using the same criterion as for %g-format in C to distinguish between 2 and 3. Then find the highest "order" used in the matrix (and the widest output field if the order is 1 (integer)) and apply it to all elements. For complex numbers, do this procedure independently for the real and imaginary parts. The result is a format in which all columns are lined up properly and have equal widths, but which also is the most compact possible. FYI, in Basis I don't have complex constants, just IMAGINARY ones. That's an alternative I have considered, and if there is a general preference for it, I am perfectly happy with it. But it should be used equally for input and output. Still I prefer the APL2/J notation because it can never give the impression of an arithmetic impression. Consider printing an expression with complex constants and multiplications in it. I am sure some people would read 2+3i * 1-7i as equivalent to 2-4i, so you'd have to print all complex numbers in parentheses (which of course would also be obligatory for input). ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 3 16:25:50 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA29209 for matrix-sig-people; Fri, 3 Nov 1995 16:25:50 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id QAA29205 for ; Fri, 3 Nov 1995 16:25:47 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA16917; Fri, 3 Nov 95 13:25:08 PST Received: by kristen.llnl.gov (5.0/SMI-SVR4) id AA16768; Fri, 3 Nov 1995 13:25:07 +0800 Date: Fri, 3 Nov 1995 13:25:07 +0800 Message-Id: <9511032125.AA16768@kristen.llnl.gov> From: "P. Dubois" To: hinsenk@ere.umontreal.ca Cc: matrix-sig@python.org In-Reply-To: <199511032112.QAA10966@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA) Subject: Re: [PYTHON MATRIX-SIG] printing complexes Content-Length: 903 Sender: owner-matrix-sig@python.org Precedence: bulk Actually, allow me to admit that the notation 3 + 4i in Basis *is* arithmetic, so that indeed 2 + 3i * 1 - 7i is 2-4i. While I think I know more about how to right a parser now, back in 1984 I didn't, so I took this very lazy way of introducing complex numbers. That said, it should be noted that the overwhelming percentage of all our matrices and complex numbers are created by users accessing compiled variables in common blocks that have been filled by compiled routines. So actual occurrence of a complex literal AS INPUT is rare. I think that would be true for most Python users too. You get complex numbers by reading some data and doing an FFT or eigenproblem or calculating some field, usually. So the question of how to represent complex literals in Python input is probably not very important as long as you don't end up screwing something up, like making the scanner twice as costly etc. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 3 16:38:31 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA29320 for matrix-sig-people; Fri, 3 Nov 1995 16:38:31 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id QAA29316 for ; Fri, 3 Nov 1995 16:38:27 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA17003; Fri, 3 Nov 95 13:37:47 PST Received: by kristen.llnl.gov (5.0/SMI-SVR4) id AA17068; Fri, 3 Nov 1995 13:37:46 +0800 Date: Fri, 3 Nov 1995 13:37:46 +0800 Message-Id: <9511032137.AA17068@kristen.llnl.gov> From: "P. Dubois" To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] the imaginary constant idea is not good Content-Length: 272 Sender: owner-matrix-sig@python.org Precedence: bulk I think I'd better say that I didn't mean to suggest that the imaginary constants idea in Basis was a good idea; it isn't. It made it easy on me at the time, and as I said it has worked o.k. because it really isn't used much. Anybody know what is in Matlab or IDL? Paul ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 3 16:52:36 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA29425 for matrix-sig-people; Fri, 3 Nov 1995 16:52:36 -0500 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id QAA29420 for ; Fri, 3 Nov 1995 16:52:31 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id QAA17735 (8.6.11/IDA-1.6); Fri, 3 Nov 1995 16:44:48 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA05417; Fri, 3 Nov 1995 16:44:42 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA13884; Fri, 3 Nov 1995 16:44:41 -0500 Date: Fri, 3 Nov 1995 16:44:41 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511032144.QAA13884@cyclone.ERE.UMontreal.CA> To: dubois@kristen.llnl.gov CC: matrix-sig@python.org In-reply-to: <9511032125.AA16768@kristen.llnl.gov> (dubois@kristen.llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] printing complexes Sender: owner-matrix-sig@python.org Precedence: bulk know more about how to right a parser now, back in 1984 I didn't, so I took this very lazy way of introducing complex numbers. It's not bad at all for input. I just don't think it is the best for output. But it's certainly good enough. So actual occurrence of a complex literal AS INPUT is rare. True enough... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 3 17:06:01 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA29524 for matrix-sig-people; Fri, 3 Nov 1995 17:06:01 -0500 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id RAA29520 for ; Fri, 3 Nov 1995 17:05:54 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id RAA18135 (8.6.11/IDA-1.6); Fri, 3 Nov 1995 17:03:33 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA09825; Fri, 3 Nov 1995 17:03:31 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA15034; Fri, 3 Nov 1995 17:03:30 -0500 Date: Fri, 3 Nov 1995 17:03:30 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511032203.RAA15034@cyclone.ERE.UMontreal.CA> To: dubois@kristen.llnl.gov CC: matrix-sig@python.org In-reply-to: <9511032137.AA17068@kristen.llnl.gov> (dubois@kristen.llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] the imaginary constant idea is not good Sender: owner-matrix-sig@python.org Precedence: bulk Anybody know what is in Matlab or IDL? Matlab uses the same notation you chose for BASIS, for input and output. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 3 17:33:30 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA29806 for matrix-sig-people; Fri, 3 Nov 1995 17:33:30 -0500 Received: from bartel.jhuapl.edu (bartel.jhuapl.edu [128.244.146.143]) by python.org (8.6.12/8.6.12) with SMTP id RAA29802 for ; Fri, 3 Nov 1995 17:33:27 -0500 Message-Id: <199511032233.RAA29802@python.org> Received: by bartel.jhuapl.edu (1.38.193.4/16.2) id AA12409; Fri, 3 Nov 1995 17:36:48 -0500 Date: Fri, 3 Nov 1995 17:36:48 -0500 From: Chris Chase S1A To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] the imaginary constant idea is not good In-Reply-To: <9511032137.AA17068@kristen.llnl.gov> References: <9511032137.AA17068@kristen.llnl.gov> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk Paul> Anybody know what is in Matlab or IDL? IDL uses a function: complex(real, imaginary). Output is "(real, imaginary)" without the quotes. Arrays of complex numbers are easily formed by using arrays for the real and imaginary parts. Matlab uses a complex constant i or j (actually functions) with the special notation that allows affixing i or j to a numeric constant, i.e. 2i = 2*i. Thus, 2 + 3i * 1 - 7i is 2-4i. Parentheses must be used to obtain the other interpretation. The use of i or j as functions can be hidden by a local variable of the same name. Output uses the i suffix. Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 3 23:27:32 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id XAA31295 for matrix-sig-people; Fri, 3 Nov 1995 23:27:32 -0500 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id XAA31291 for ; Fri, 3 Nov 1995 23:27:25 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id MAA08435 (8.6.11/IDA-1.6); Fri, 3 Nov 1995 12:04:31 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA22331; Fri, 3 Nov 1995 12:04:29 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA26533; Fri, 3 Nov 1995 12:04:24 -0500 Date: Fri, 3 Nov 1995 12:04:24 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511031704.MAA26533@cyclone.ERE.UMontreal.CA> To: jjh@mama-bear.lcs.mit.edu CC: matrix-sig@python.org In-reply-to: <9511031537.AA03746@nineveh.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Fri, 3 Nov 95 10:37:02 -0500) Subject: Re: [PYTHON MATRIX-SIG] IEEE exceptions, casting, matrix multiplies, and default types Sender: owner-matrix-sig@python.org Precedence: bulk [Note: Matrix_i creates a matrix of ints, Matrix_d creates one of doubles] 2) Matrix_i(1,2,3) * Matrix_d(1.,2.,3.) == Matrix_d(1.,4.,6.) or raises exception I see no reason why matrices should have different coercion rules (or even none at all) than scalar numbers. So the result should be a float matrix. 4) add([1,2,3], [4,5,6]) == Matrix_?(5,7,9) where the question is what the ? should be. The rule should be that first the lists get transformed into matrices of a certain type, and then addition proceeds using the normal coercion rules. My suggestion for determining the global type of a list is that it should be complex if at least one element is complex, else float if at least one element is float, or otherwise integer. That of course leaves two of your points as starting points for extensive ideological fights ;-) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Nov 6 12:15:49 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA09735 for matrix-sig-people; Mon, 6 Nov 1995 12:15:49 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id MAA09731 for ; Mon, 6 Nov 1995 12:15:44 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA09048; Mon, 6 Nov 95 09:14:23 PST Received: by kristen.llnl.gov (5.0/SMI-SVR4) id AA17004; Mon, 6 Nov 1995 09:14:09 +0800 Date: Mon, 6 Nov 1995 09:14:09 +0800 Message-Id: <9511061714.AA17004@kristen.llnl.gov> From: "P. Dubois" To: hinsenk@ere.umontreal.ca Cc: jjh@mama-bear.lcs.mit.edu, matrix-sig@python.org In-Reply-To: <199511031704.MAA26533@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Subject: Re: [PYTHON MATRIX-SIG] casting and default types Content-Length: 1222 Sender: owner-matrix-sig@python.org Precedence: bulk I agree with Konrad's view: the type of the matrix to be created from a list should be the least type conforming to all the components, and the same should go for the result of an expression. One could have the Matrix_? operators which strictly required the correct kind of list as an argument, and just plain Matrix(list) would be the creation routine which was "clever" at figuring this out. In Basis, [1,2,3] is an integer array, [1,2,3.] is a real array. Combined with the expression rule, this works well. I think no coercion is probably workable but less desireable. Related to this is what such "coercion" means for characters. If you have a list [a,b,c] where a,b,c are all strings, Basis interprets the "type" of the resulting matrix to be "strings of length max(len(a),len(b), len(c)), and the component strings are copied into the new structure with right blank fill to this new length. (I keep mentioning Basis simply as a way of letting you all know what one way is that users found acceptable; this is a set of semantics that has 10 years of intensive user testing. When there is something the users didn't like about it, I'll tell you that too. In this area, there are no complaints whatsoever.) ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Nov 6 12:36:36 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA09892 for matrix-sig-people; Mon, 6 Nov 1995 12:36:36 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id MAA09888 for ; Mon, 6 Nov 1995 12:36:33 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA09652; Mon, 6 Nov 95 09:35:17 PST Received: by kristen.llnl.gov (5.0/SMI-SVR4) id AA17513; Mon, 6 Nov 1995 09:35:16 +0800 Date: Mon, 6 Nov 1995 09:35:16 +0800 Message-Id: <9511061735.AA17513@kristen.llnl.gov> From: "P. Dubois" To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] A related idea Content-Length: 556 Sender: owner-matrix-sig@python.org Precedence: bulk Suppose we have the matrix class working. Suppose there is a Fortran or C array in the program which you would like to "adopt" as a matrix in Python. The only difference between this and a matrix created a normal way is that you have to remember never to free the memory. So it seems you might be able to do this if you had a method of creation (from C) which took a pointer,size,type kind of description. You could then increment the reference counter one extra, so that the memory is never freed, or a "don't free" flag could be in the header. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Nov 7 09:56:34 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA15777 for matrix-sig-people; Tue, 7 Nov 1995 09:56:34 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id JAA15773 for ; Tue, 7 Nov 1995 09:56:29 -0500 Received: from nineveh.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA09734; Tue, 7 Nov 95 09:54:40 EST Received: by nineveh.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA11707; Tue, 7 Nov 95 09:56:40 -0500 Message-Id: <9511071456.AA11707@nineveh.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Tue, 7 Nov 95 09:56:39 -0500 To: "P. Dubois" Subject: Re: [PYTHON MATRIX-SIG] A related idea Cc: matrix-sig@python.org References: <9511061735.AA17513@kristen.llnl.gov> Sender: owner-matrix-sig@python.org Precedence: bulk > Suppose we have the matrix class working. What do you mean suppose ;) ? > Suppose there is a Fortran > or C array in the program which you would like to "adopt" as a > matrix in Python. The only difference between this and a matrix created a > normal way is that you have to remember never to free the memory. > So it seems you might be able to do this if you had a method > of creation (from C) which took a pointer,size,type kind of > description. You could then increment the reference counter one extra, so > that the memory is never freed, or a "don't free" flag could be in the > header. I agree that there should be a function to create a new matrix object from an existing array in C (or FORTRAN). What we disagree about is the semantics. I played around a while with the idea of "sharing" an array between python and C. I found that as a general rule I ran into memory leak problems (or worse). Using your scheme, python is never responsible for freeing the allocated memory, this means that the C code must be responsible for it, but the C code can't know when it is safe to free the memory because even when it's done with it there might still be a python reference to the memory. Let me know if I'm completely missing something here, and this memory leak issue isn't really a problem. I made the general decision to completely avoid these sorts of issues by (almost) never returning arrays that are references to other arrays. I found that doing a malloc and a memcpy is usually sufficiently faster than any operation I'm going to perform on the array that it could be ignored. In order for this to be a valid assumption, I need to be working on a system where I have "more than enough" memory hanging around. This is always true for me, but I'm interested in cases where it's not. Note: the following will be completely meaningless to you if you haven't played with the alpha matrix object, so please ignore. --- I'd add the following function to do what you're suggesting, but using my semantics (this is off the top of my head, so there're no guarantees it will work). PyObject *PyMatrix_FromDimsAndData(int nd, int *dimensions, char type, void *data) { PyMatrixObject *ret = PyMatrix_FromDims(nd, dimensions, type); memcpy(ret->data, data, NBYTES(ret)); return (PyObject *)ret; } I'll add something like this to the next release. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Nov 7 10:12:42 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA15935 for matrix-sig-people; Tue, 7 Nov 1995 10:12:42 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id KAA15931 for ; Tue, 7 Nov 1995 10:12:38 -0500 Received: from nineveh.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA09878; Tue, 7 Nov 95 10:10:45 EST Received: by nineveh.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA11715; Tue, 7 Nov 95 10:12:45 -0500 Message-Id: <9511071512.AA11715@nineveh.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Tue, 7 Nov 95 10:12:44 -0500 To: "P. Dubois" Subject: Re: [PYTHON MATRIX-SIG] casting and default types Cc: matrix-sig@python.org References: <9511061714.AA17004@kristen.llnl.gov> Sender: owner-matrix-sig@python.org Precedence: bulk I've had a couple of discussions with Jim Fulton about exactly this issue when I first grabbed his matrix object. If he's reading this, I would really like to get his opinion. I agree with you, and for my uses full automatic coercion as well as reasonable choices for matrix type in instantiation is a good idea. The current instantiation (and lack of coercion) system is Jim Fulton's (more or less) and he could do a much better job of defending it than me. The principle issue is that automatic type coercions can hide bugs. The other issue is that automatic coercion can hide efficiency problems. If I have a vector of ints that I'm frequently going to have to multiply by different vectors of floats, then it would be much more efficient for me to convert it to a vector of floats one time. If I have automatic type coercion, I might never even notice this problem (except that my multiply operation would be something like a factor of two slower). This is why I mentioned this issue as something that might be amenable to some sort of flag in the same way that IEEE exceptions could be (I was just noticing that a system of this sort is enabled on the DEC alpha's). If I want to be really careful, I'd insist that adding floats to ints raised an exception, and simlarly that dividing by zero would do the same. If I was willing to play fast and loose then I could disable these forms of "error checking" and I'd be off. Still not sure what's the "right" way, Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Nov 7 14:48:34 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA17636 for matrix-sig-people; Tue, 7 Nov 1995 14:48:34 -0500 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id OAA17629 for ; Tue, 7 Nov 1995 14:47:56 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id OAA02723 (8.6.11/IDA-1.6); Tue, 7 Nov 1995 14:43:39 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA18469; Tue, 7 Nov 1995 14:43:38 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA23753; Tue, 7 Nov 1995 14:43:37 -0500 Date: Tue, 7 Nov 1995 14:43:37 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511071943.OAA23753@cyclone.ERE.UMontreal.CA> To: jjh@mama-bear.lcs.mit.edu CC: matrix-sig@python.org In-reply-to: <9511071512.AA11715@nineveh.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Tue, 7 Nov 95 10:12:44 -0500) Subject: Re: [PYTHON MATRIX-SIG] casting and default types Sender: owner-matrix-sig@python.org Precedence: bulk The current instantiation (and lack of coercion) system is Jim Fulton's (more or less) and he could do a much better job of defending it than me. The principle issue is that automatic type coercions can hide bugs. The Whether or not coercion is a good idea is an old argument, but for Python it has long been decided: Python has coercion, and not having it for matrices would be simply inconsistent. other issue is that automatic coercion can hide efficiency problems. If I have a vector of ints that I'm frequently going to have to multiply by different vectors of floats, then it would be much more efficient for me to convert it to a vector of floats one time. If I have automatic type I'd suppose that people who need the highest possible performance are aware of this problem and are watching out for it. There is a profiler for Python that can help find such problems. I don't see this as a valid reason Python inconsistent and most users' life more difficult. This is why I mentioned this issue as something that might be amenable to some sort of flag in the same way that IEEE exceptions could be (I was just That's of course a possible solution, if someone thinks this is really important. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Nov 7 15:48:19 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA18045 for matrix-sig-people; Tue, 7 Nov 1995 15:48:19 -0500 Received: from qvarsx.er.usgs.GOV (qvarsx.er.usgs.gov [130.11.51.82]) by python.org (8.6.12/8.6.12) with ESMTP id PAA18041 for ; Tue, 7 Nov 1995 15:48:16 -0500 Received: from localhost (dsjfqvarsa [130.11.51.73]) by qvarsx.er.usgs.GOV (EMAIL 1.2.1) with ESMTP id UAA21889; Tue, 7 Nov 1995 20:41:36 GMT Message-Id: <199511072041.UAA21889@qvarsx.er.usgs.GOV> To: hinsenk@ere.umontreal.ca (Hinsen Konrad) cc: jjh@mama-bear.lcs.mit.edu cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] casting and default types In-reply-to: <199511071943.OAA23753@cyclone.ERE.UMontreal.CA> Date: Tue, 07 Nov 1995 15:46:35 -0500 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk First of all, I want to apologize for not participating in this list much lately. I've been buried in a project here and haven't had time to keep up. On Tue, 7 Nov 1995 14:43:37 -0500 Hinsen Konrad said: > > The current instantiation (and lack of coercion) system is Jim Fulton's > (more or less) and he could do a much better job of defending it than me. > The principle issue is that automatic type coercions can hide bugs. The > > Whether or not coercion is a good idea is an old argument, but > for Python it has long been decided: Python has coercion, and > not having it for matrices would be simply inconsistent. Actually, this is not true. There are some places where coersion takes place automatically, but there are lot's of places where it doesn't. In the case of numbers, witness the float() and int() functions. Similarly, lists are not automatically converted to tuples in many places where tuples are required. And so on. > other issue is that automatic coercion can hide efficiency problems. If I > have a vector of ints that I'm frequently going to have to multiply by > different vectors of floats, then it would be much more efficient for me to > convert it to a vector of floats one time. If I have automatic type > > I'd suppose that people who need the highest possible performance > are aware of this problem and are watching out for it. There > is a profiler for Python that can help find such problems. I don't > see this as a valid reason Python inconsistent and most users' > life more difficult. It's not just a matter of performance. It is also a matter of correctness in some cases. > This is why I mentioned this issue as something that might be amenable to > some sort of flag in the same way that IEEE exceptions could be (I was just > > That's of course a possible solution, if someone thinks this > is really important. Global variables are evil. I'd be much more in favor of having different kinds of matrices that automagically coerce, if people really want this. I have to bring myself up to date on the in's and outs of this coersion issue. I think my objection to Jim's original coersion proposal had to do with ranks, more than the actual data types. There are certainly situations where coersion makes sense. For example, in my Fortran interface, FIDL, I automatically convert between argument and expected numeric element types. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Nov 7 16:13:05 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA18229 for matrix-sig-people; Tue, 7 Nov 1995 16:13:05 -0500 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id QAA18225 for ; Tue, 7 Nov 1995 16:13:00 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id QAA05592 (8.6.11/IDA-1.6); Tue, 7 Nov 1995 16:03:00 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA07111; Tue, 7 Nov 1995 16:02:57 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA29127; Tue, 7 Nov 1995 16:02:56 -0500 Date: Tue, 7 Nov 1995 16:02:56 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511072102.QAA29127@cyclone.ERE.UMontreal.CA> To: jfulton@usgs.gov CC: jjh@mama-bear.lcs.mit.edu, matrix-sig@python.org In-reply-to: <199511072041.UAA21889@qvarsx.er.usgs.GOV> (jfulton@wrdmail.er.usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] casting and default types Sender: owner-matrix-sig@python.org Precedence: bulk Actually, this is not true. There are some places where coersion takes place automatically, but there are lot's of places where it doesn't. In the case of numbers, witness the float() and int() functions. Similarly, lists are not automatically converted to tuples I can't imagine a need for float() and int() except in order to "downgrade" a number. In all mathematical operations where floats are required, I can just as well use ints. Lists and tuples are another problem, of course. My point of view is that as far as coercion is concerned, matrices should behave just like scalars of the same type(s). It's not just a matter of performance. It is also a matter of correctness in some cases. I don't understand. The choice is automatic coercion or none at all, i.e. throwing an exception for mixed-type operations. I don't see how one or the other can lead to a wrong result; it's either the correct result or no result. Of course there is some chance of making a mistake and not noticing it due to automatic coercion, but then you already have the same risk with scalars. Besides, I just can't think of a reasonable example... Also, I'd expect users to be familiar with coercion. I haven't made a survey, but I'd claim that most languages have coercion between numerical types, similar to Python. That's certainly true for the languages most used for numerical work. I'd be much more in favor of having different kinds of matrices that automagically coerce, if people really want this. Which creates some question and probably a lot of confusion. What happens if coercing and non-coercing matrices are combined? What happens if a function written for one type of matrices is used with the other? How can the matrices be distinguished in output? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Nov 8 09:03:01 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA22195 for matrix-sig-people; Wed, 8 Nov 1995 09:03:01 -0500 Received: from qvarsx.er.usgs.GOV (qvarsx.er.usgs.gov [130.11.51.82]) by python.org (8.6.12/8.6.12) with ESMTP id JAA22191 for ; Wed, 8 Nov 1995 09:02:57 -0500 Received: from localhost (dsjfqvarsa [130.11.51.73]) by qvarsx.er.usgs.GOV (EMAIL 1.2.1) with ESMTP id NAA14836; Wed, 8 Nov 1995 13:56:11 GMT Message-Id: <199511081356.NAA14836@qvarsx.er.usgs.GOV> To: hinsenk@ere.umontreal.ca (Hinsen Konrad) cc: "James L Fulton, Hydrologist, Reston, VA " cc: jjh@mama-bear.lcs.mit.edu, matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] casting and default types In-reply-to: <199511072102.QAA29127@cyclone.ERE.UMontreal.CA> Date: Wed, 08 Nov 1995 09:01:11 -0500 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk On Tue, 7 Nov 1995 16:02:56 -0500 Hinsen Konrad said: > > Actually, this is not true. There are some places where coersion > takes place automatically, but there are lot's of places where it > doesn't. In the case of numbers, witness the float() and int() > functions. Similarly, lists are not automatically converted to tuples > > I can't imagine a need for float() and int() except in order to > "downgrade" a number. In all mathematical operations where floats > are required, I can just as well use ints. Lists and tuples are > another problem, of course. > > My point of view is that as far as coercion is concerned, matrices > should behave just like scalars of the same type(s). But what you are unwilling to believe is that scalars do not always coerce automatically. Try: spam = '0' * 20 and spam = '0' * 20.0 > It's not just a matter of performance. It is also a matter of > correctness in some cases. > > I don't understand. The choice is automatic coercion or none at all, > i.e. throwing an exception for mixed-type operations. I don't see > how one or the other can lead to a wrong result; it's either the > correct result or no result. No, another choice is generating an incorrect result. > Of course there is some chance of making a mistake and not noticing > it due to automatic coercion, but then you already have the same > risk with scalars. Besides, I just can't think of a reasonable > example... OK, how about this: 1 / 2 * 2 as opposed to: 1 / 2.0 * 2 These two expressions generate different results. If I wanted the first algorithm, I would need to explicitly force 2.0 to be an integer, but coersion has instead forced the first sub-expression to be a floating-point expression. There was discussion in the Python list a while back about problems caused by a new feature that automatically coerced floating point values passed to functions expecting integer arguments. Guido even expressed regret at being talked into adding the feature. Check the list archive for details. Even if an error is raised, it may be raised far from where the coersion that ultimately caused the error was done, making debugging of the error difficult. As I said in my other note, I object less to coersion of element types that to coersion of ranks. > Also, I'd expect users to be familiar with coercion. I haven't > made a survey, but I'd claim that most languages have coercion > between numerical types, similar to Python. That's certainly > true for the languages most used for numerical work. Again, user's familiar with Python should not expect coersion everywhere. User's familiar with Fortran should certianly not expect coersion everywhere. > I'd be much more in favor of having different kinds of matrices that > automagically coerce, if people really want this. > > Which creates some question and probably a lot of confusion. > What happens if coercing and non-coercing matrices are combined? > What happens if a function written for one type of matrices > is used with the other? How can the matrices be distinguished > in output? These are certainly valid points. Of course, the situation would be much worse with a global variable. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Nov 8 09:41:10 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA22457 for matrix-sig-people; Wed, 8 Nov 1995 09:41:10 -0500 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id JAA22453 for ; Wed, 8 Nov 1995 09:41:05 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id JAA23239 (8.6.11/IDA-1.6); Wed, 8 Nov 1995 09:37:26 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA17463; Wed, 8 Nov 1995 09:32:02 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA15432; Wed, 8 Nov 1995 09:32:01 -0500 Date: Wed, 8 Nov 1995 09:32:01 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511081432.JAA15432@cyclone.ERE.UMontreal.CA> To: jfulton@usgs.gov CC: matrix-sig@python.org In-reply-to: <199511081356.NAA14836@qvarsx.er.usgs.GOV> (jfulton@wrdmail.er.usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] casting and default types Sender: owner-matrix-sig@python.org Precedence: bulk > My point of view is that as far as coercion is concerned, matrices > should behave just like scalars of the same type(s). But what you are unwilling to believe is that scalars do not always coerce automatically. Not at all. I am just saying that if coercion is applied to a certain combination of scalar types, then the same coercion should be applied to arrays of the same type. As to which coercions should occur automatically, I'd say that int -> float -> complex should be automatic, but the reverse not, because it leads to an essential loss of information (int ->float can occasionally lead to a loss of precision, depending on how long the representations of int and float are, but I can live with that). > Of course there is some chance of making a mistake and not noticing > it due to automatic coercion, but then you already have the same > risk with scalars. Besides, I just can't think of a reasonable > example... OK, how about this: 1 / 2 * 2 as opposed to: 1 / 2.0 * 2 That's indeed a problem, but (in my point of view) not one caused by coercion, but by the unfortunate fact that / denotes quite different operations for integers and for floats. There was discussion in the Python list a while back about problems caused by a new feature that automatically coerced floating point values passed to functions expecting integer arguments. Guido even expressed regret at being talked into adding the feature. Check the list archive for details. I didn't know about that feature, and I certainly don't like it (see above). As I said in my other note, I object less to coersion of element types that to coersion of ranks. What exactly do you mean by "coercion of ranks"? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Nov 8 10:06:18 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA22543 for matrix-sig-people; Wed, 8 Nov 1995 10:06:18 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id KAA22539 for ; Wed, 8 Nov 1995 10:06:15 -0500 Received: from nineveh.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA16936; Wed, 8 Nov 95 10:04:17 EST Received: by nineveh.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA13100; Wed, 8 Nov 95 10:06:28 -0500 Message-Id: <9511081506.AA13100@nineveh.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Wed, 8 Nov 95 10:06:28 -0500 To: hinsenk@ere.umontreal.ca (Hinsen Konrad) Subject: Re: [PYTHON MATRIX-SIG] casting and default types Cc: matrix-sig@python.org References: <199511081432.JAA15432@cyclone.ERE.UMontreal.CA> Sender: owner-matrix-sig@python.org Precedence: bulk > What exactly do you mean by "coercion of ranks"? I'm responsible for this poor phrase. My initial matrix proposal had the phrase "dimension coercion" for the J-style rank concept (of which I was unaware at the time). The issue in a nutshell is that everybody (I think) agrees that: a = Matrix((1,2,3),(11,12,13)) b = Matrix(100,200,300) a+1 = Matrix((2,3,4),(12,13,14)) One question is should a+b be an error, or should it be a+b = Matrix((101,202,203),(111,212,313)) I believe that the second is correct, and this can be elegantly expressed by the concept of ranks. I don't think there's much disagreement on this either. Now the question comes, what if I ask for inverse( Matrix(((1,2),(11,12)), ((3,4),(13,14))) ). Using the rank notions in J, the argument should be considered a 1-frame of 2-cells. The result should be a 1-frame containing the inverse of each of these 2-cells. However, for a case like this one, the arguments in favor of treating this as an error (you can't take the inverse of a 3d matrix) seem worth considering. My current solution to this is that my optimized ofuncs use the J-style rank system, but only work on operations with unbounded rank like +, *, etc. This is as much an implementation issue as anything else. Other functions on matrices, like inverse (or my favorite, fft) are completely free to implement a rank system, or not as they see fit. So for the fft function, when given a matrix, it will return the fft of every 1-cell. The question is whether or not we should adopt such a system as a standard for matrix functions. Under the current setup there would be no way to enforce such a standard. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Nov 8 23:32:39 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id XAA27173 for matrix-sig-people; Wed, 8 Nov 1995 23:32:39 -0500 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id XAA27169 for ; Wed, 8 Nov 1995 23:32:30 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id KAA25830 (8.6.11/IDA-1.6); Wed, 8 Nov 1995 10:47:45 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id KAA07970; Wed, 8 Nov 1995 10:47:36 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id KAA19928; Wed, 8 Nov 1995 10:47:35 -0500 Date: Wed, 8 Nov 1995 10:47:35 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511081547.KAA19928@cyclone.ERE.UMontreal.CA> To: jjh@mama-bear.lcs.mit.edu CC: matrix-sig@python.org In-reply-to: <9511081506.AA13100@nineveh.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Wed, 8 Nov 95 10:06:28 -0500) Subject: Re: [PYTHON MATRIX-SIG] casting and default types Sender: owner-matrix-sig@python.org Precedence: bulk Using the rank notions in J, the argument should be considered a 1-frame of 2-cells. The result should be a 1-frame containing the inverse of each of these 2-cells. However, for a case like this one, the arguments in favor of treating this as an error (you can't take the inverse of a 3d matrix) seem worth considering. In my opinion, the experience of the APL/J community shows that using rank extension everywhere is the better solution. Although in principle that can hide errors, such errors don't seem to occur in practice. It happens that by some mistake you create an array with the wrong length (typically off by 1), but you never create an array of the wrong rank. Besides, you could always locate such a mistake after a rank-3 inversion, as the result is still rank-3 and could not be confused with the inverse of a rank-2 matrix. On the other hand, there are applications where rank extension on inverses etc. is useful. Of course you could always get the same effect with explicit loops, but the speed difference is enormous (e.g. for inverting 1000 4x4 matrices). My current solution to this is that my optimized ofuncs use the J-style rank system, but only work on operations with unbounded rank like +, *, etc. That includes all binary operations, which are the most complicated. Other functions on matrices, like inverse (or my favorite, fft) are completely free to implement a rank system, or not as they see fit. So for There should be some provision in the matrix module for applying a unary function with a given rank to an array of higher rank, which could also be applied to user-written Python functions. the fft function, when given a matrix, it will return the fft of every 1-cell. The question is whether or not we should adopt such a system as a FFT is a good example for an operation where the rank system is useful. There are of course many applications for rank-1 FFT with extension to higher ranks, i.e. what you implemented. But it also makes sense to have multidimensional FFTs. In the J system, FFT would be unbounded and therefore n-dimensional for a rank-n array. You could then specify a lower rank to get lower-dimensional FFTs. standard for matrix functions. Under the current setup there would be no way to enforce such a standard. I think it should be standard (i.e. provided for all built-in functions). It shouldn't be enforced for user-defined function, but it should be made easy. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Nov 9 14:12:55 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA32109 for matrix-sig-people; Thu, 9 Nov 1995 14:12:55 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id OAA32104 for ; Thu, 9 Nov 1995 14:12:48 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA01142; Thu, 9 Nov 95 11:10:46 PST Received: by kristen.llnl.gov (5.x/SMI-SVR4) id AA08047; Thu, 9 Nov 1995 11:10:44 -0800 Date: Thu, 9 Nov 1995 11:10:44 -0800 Message-Id: <9511091910.AA08047@kristen.llnl.gov> From: "P. Dubois" To: jjh@mama-bear.lcs.mit.edu Cc: matrix-sig@python.org In-Reply-To: <9511071456.AA11707@nineveh.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Tue, 7 Nov 95 09:56:39 -0500) Subject: Re: [PYTHON MATRIX-SIG] A related idea Sender: owner-matrix-sig@python.org Precedence: bulk Jim Hugunin replied, in response to my suggestion for providing an "adoption" method for compiled arrays: > Suppose we have the matrix class working. What do you mean suppose ;) ? Hey, I'm a mathematician, all my sentences start with suppose! > Suppose there is a Fortran > or C array in the program which you would like to "adopt" as a > matrix in Python. The only difference between this and a matrix created a > normal way is that you have to remember never to free the memory. > So it seems you might be able to do this if you had a method > of creation (from C) which took a pointer,size,type kind of > description. You could then increment the reference counter one extra, so > that the memory is never freed, or a "don't free" flag could be in the > header. I agree that there should be a function to create a new matrix object from an existing array in C (or FORTRAN). What we disagree about is the semantics. I played around a while with the idea of "sharing" an array between python and C. I found that as a general rule I ran into memory leak problems (or worse). Using your scheme, python is never responsible for freeing the allocated memory, this means that the C code must be responsible for it, but the C code can't know when it is safe to free the memory because even when it's done with it there might still be a python reference to the memory. Let me know if I'm completely missing something here, and this memory leak issue isn't really a problem. This is a good point which I hadn't thought of because the way I intend to use this "adopting" the arrays are the permanent state of a calculation and are never "released" by the compiled code. I was aware that if I wanted to ever change size etc. I would somehow need to tell Python about it. Some careful thinking is required in that case. The point isn't that I want to save memory but that I want to MODIFY the array the compiled code is using, such as an array in a Fortran common block. I don't have a problem copying it when I'm going to do something with it. But to use it as a method of input to a code, I need to be able to do something like: import phys1 #load up physics code phys1.x = # or phys1.x[3] = 1. Here I'm assuming x is created by the initialization of the module such that it has the "real" x in the compiled code as a data area. Obviously, one could instead provide methods (e.g., set_x, set_y, set_z) but it dispels the illusion that x is really a matrix attribute of a physics package. Maybe I'm not taking the right approach here. Suggestions appreciated. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 10 11:40:49 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA00470 for matrix-sig-people; Fri, 10 Nov 1995 11:40:49 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id LAA00465 for ; Fri, 10 Nov 1995 11:40:44 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA11735; Fri, 10 Nov 95 08:40:41 PST Received: by kristen.llnl.gov (5.x/SMI-SVR4) id AA18754; Fri, 10 Nov 1995 08:40:40 -0800 Date: Fri, 10 Nov 1995 08:40:40 -0800 Message-Id: <9511101640.AA18754@kristen.llnl.gov> From: "P. Dubois" To: matrix-sig@python.org Cc: equipe-basis@lists.llnl.gov Subject: [PYTHON MATRIX-SIG] [munro@icf.llnl.gov: Re: Python Matrix extension tutorial] Sender: owner-matrix-sig@python.org Precedence: bulk Dear Matrix-SIGers, Yesterday at LLNL Brian Yang taught a tutorial on the matrix extension to about 20 people. I asked that people give me comments to forward to the SIG. In general, we concluded that the extension does what we need it to do. Here are some of the comments for your consideration. I will send them as separate messages so as to compartmentalize the discussion a little. The first is from Dave Munro. Dave is a physicist who has developed his own interpreted language for postprocessing called Yorick. The code this is most often used to postprocess, Lasnex, is a Fortran program with lots of arrays, so Yorick has facilities especially devoted to manipulating arrays easily. Here is Dave's suggestion: ------- Start of forwarded message ------- In the handout at the meeting on page 7 is an example of the heavily used Yorick feature I mentioned: You have two arrays b with shape (2,3) c with shape (2,2,3) and you want to produce: d[i][j][k]= b[i][k] + c[i][j][k] {The example would be much clearer if the ranks were given as (2,3) and (2,4,3), so it only made sense one way.} In Python, this common operation is carried out by the expression: d= add[1,2](b,c) Yorick has a special syntax for this same operation: d= b(,-,) + c which I find much clearer; the "-" in an index list is a place holder indicating that the result is to have an additional dimension in the slot where the "-" was. Thus b(,-,) is a rank-3 array which can be directly compared with c. With this notation, b(,,-)+c is the same as b[j][k]+c[i][j][k] (the default), while b(,-,)+c produces b[i][k]+c[i][j][k] and b(-,,)+c produces b[i][j]+c[i][j][k]. I call the "-" a "pseudo-index" in Yorick. In order for this to work, you need to broaden the definition of when two arrays are conformable; instead of requiring an exact match for the shape of the largest common cells of the two operands, you need to treat 1-length dimensions as "wildcards" matching any length. Thus, if b has shape (2,3) then b(,-,) has shape (2,1,3) -- exactly the same as reshape(b, (2,1,3)) -- which you can consider conformable to the (2,4,3) array c by the rule that [i][0][k] matches [i][j][k]. Ordinary scalar broadcasting is just a specific case of this general conformability rule. ----- I have the impression that the syntax would present very little difficulty in Python; on the model of the All and Reverse index objects, it would be easy to introduce a Pseudo object which did the reshape: d= b[(All,Pseudo,All)] + c However, I imagine that the implementor will be loathe to change his notion of array conformability at this late date. Nevertheless, you might as well ask to see what he says; probably the change wouldn't actually break any existing correct code, but merely give meaning to what would previously have produced a runtime error. That "1-length wildcard" rule has been a tremendous winner for Yorick; if you can convince the Python guys to use it, I doubt they would ever regret it. In Yorick, the combination of the relaxed conformability rule, pseudo-indices and a general transpose operator that can produce any permutation of dimensions allows for nearly all meaningful combinations of arrays. ----- Yorick has one other indexing syntax which has proven useful, which I call "rubber indices". They address the problem of writing interpreted code which extracts slices of arrays when you don't know beforehand how many dimensions the array has. An example is an opacity array for which you know that the most slowly varying index represents photon wavelength, but there might be zero, one, two, or three faster varying dimensions representing position in space. These situations can probably be handled with Python's reshape operator, so again this is a question of ease of use. ------- End of forwarded message ------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 10 11:53:07 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA00561 for matrix-sig-people; Fri, 10 Nov 1995 11:53:07 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id LAA00557 for ; Fri, 10 Nov 1995 11:53:04 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA11890; Fri, 10 Nov 95 08:53:02 PST Received: by kristen.llnl.gov (5.x/SMI-SVR4) id AA18832; Fri, 10 Nov 1995 08:53:01 -0800 Date: Fri, 10 Nov 1995 08:53:01 -0800 Message-Id: <9511101653.AA18832@kristen.llnl.gov> From: "P. Dubois" To: matrix-sig@python.org Cc: equipe-basis@lists.llnl.gov Subject: [PYTHON MATRIX-SIG] [tbyang@icf.llnl.gov: Some subtlety in reaching items of a matrix] Sender: owner-matrix-sig@python.org Precedence: bulk The following comment by Brian Yang is not a surprise, as the difference in returning reference vs. returning copy was documented, but I thought I'd pass it on. ------- Start of forwarded message ------- I just found something about the matrix module which I was not aware of earlier: For a rank-3 matrix `c', there are some subtle difference between C[0][1] and c[(0,1)]. The following example illustrates this: -----------------------------------------> >>> c [[[4.0, 5.0, 6.0], [0.0, 4.0, 5.0]], [[1.0, 6.0, 5.0], [8.0, 5.0, 2.0]]] >>> c2=c[(0,1)] >>> c2 [0.0, 4.0, 5.0] >>> c1=c[0][1] >>> c1 [0.0, 4.0, 5.0] >>> c[(0,1,2)]=6 >>> c [[[4.0, 5.0, 6.0], [0.0, 4.0, 6.0]], [[1.0, 6.0, 5.0], [8.0, 5.0, 2.0]]] >>> c1 [0.0, 4.0, 6.0] >>> c2 [0.0, 4.0, 5.0] <-------------------------------------- Notice when c changes, c1 changes also, but not c2. The difference occurs because in `c1=c[0][1]' c is treated as a sequence, while in `c2=c[(0,1)]' c is treated as a mapping. And it just so happened that in the implementation of the methods, Hugunin decided to create a new matrix in the latter case, while for the former case he just created some new pointers pointing to the same data area as c. It appears very confusing to me. If we can have enough feed-back about this, maybe we can suggest him to change to one way or the other. ------- End of forwarded message ------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 10 12:10:03 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA00658 for matrix-sig-people; Fri, 10 Nov 1995 12:10:03 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id MAA00653 for ; Fri, 10 Nov 1995 12:10:00 -0500 Received: from ling-ling.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA05332; Fri, 10 Nov 95 12:08:08 EST Received: by ling-ling.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA00801; Fri, 10 Nov 95 12:08:37 -0500 Message-Id: <9511101708.AA00801@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Fri, 10 Nov 95 12:08:37 -0500 To: "P. Dubois" Subject: Re: [PYTHON MATRIX-SIG] [tbyang@icf.llnl.gov: Some subtlety in reaching items of a matrix] Cc: matrix-sig@python.org, equipe-basis@lists.llnl.gov References: <9511101653.AA18832@kristen.llnl.gov> Sender: owner-matrix-sig@python.org Precedence: bulk > For a rank-3 matrix `c', there are some subtle difference between > C[0][1] and c[(0,1)]. The following example illustrates this: First, I agree completely, this is terrible. Nevertheless, here's my justification: Returning a matrix by-copy is always the best thing to do conceptually (I guess I've played with too many functional languages). I find it a lot easier to think about my matrix code if I don't have to worry about references to objects. On the other hand, if you want to allow python indexing of the style a[i][j] to function with any sort of reasonable efficiency, then you need to have a[i] return by reference. I'm perfectly willing to be convinced to change over to always returning matrices by reference (with the understanding that the code will have many more bugs, and the understanding that if a = Matrix(1,2,3) then a[0] will return not a python float, but a 0d matrix with the same sort of reference style semantics) or to always returning arrays by copy with the understanding that a[i][j] will potentially be an incredibly inefficient way of accessing an array element. I'm not happy with my chosen compromise. I just can't come up with a better solution. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 10 12:20:17 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA00717 for matrix-sig-people; Fri, 10 Nov 1995 12:20:17 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id MAA00713 for ; Fri, 10 Nov 1995 12:20:13 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA12187; Fri, 10 Nov 95 09:20:09 PST Received: by kristen.llnl.gov (5.x/SMI-SVR4) id AA19051; Fri, 10 Nov 1995 09:20:09 -0800 Date: Fri, 10 Nov 1995 09:20:09 -0800 Message-Id: <9511101720.AA19051@kristen.llnl.gov> From: "P. Dubois" To: jjh@mama-bear.lcs.mit.edu Cc: matrix-sig@python.org, equipe-basis@lists.llnl.gov In-Reply-To: <9511101708.AA00801@ling-ling.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Fri, 10 Nov 95 12:08:37 -0500) Subject: Re: [PYTHON MATRIX-SIG] [tbyang@icf.llnl.gov: Some subtlety in reaching items of a matrix] Sender: owner-matrix-sig@python.org Precedence: bulk I think we will sooner or later get time to make the change discussed before so that c[0,1] is a euphemism for c[(0,1)]. Assuming that, allowing c[0][1] to be a reference while c[0,1] is a copy doesn't seem too hard to explain to people. And I think the latter will be used much more than the former for use in expressions. I think the compromise adopted is far better than the alternatives; this class has to have good performance and we certainly want easy ways to assign values to row and columns at vector speed. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 10 12:46:34 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA01013 for matrix-sig-people; Fri, 10 Nov 1995 12:46:34 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id MAA01009 for ; Fri, 10 Nov 1995 12:46:31 -0500 Received: from ling-ling.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA05484; Fri, 10 Nov 95 12:44:45 EST Received: by ling-ling.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA00962; Fri, 10 Nov 95 12:45:43 -0500 Message-Id: <9511101745.AA00962@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Fri, 10 Nov 95 12:45:43 -0500 To: tbyang@garion.llnl.gov (Tser-Yuan (Brian) Yang) Subject: Re: [PYTHON MATRIX-SIG] [tbyang@icf.llnl.gov: Some subtlety in reaching items of a matrix] Cc: equipe-basis@monroe.llnl.gov, matrix-sig@python.org References: <9511101737.AA00683@garion.llnl.gov.llnl.gov> Sender: owner-matrix-sig@python.org Precedence: bulk > If c[0,1] becomes a euphemism for c[(0,1)]. I will be very confused in > interpreting c[0], as it can correpond to both c[0,1] and c[0][1], if > they do different things. This is in a very good point! Just so you know how things will work (if no changes are made). c[0] will be a reference to the data in c, and c[0,] will be a copy of the data in c. This is incredibly ugly, but again, I don't know what to do about it. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 10 15:20:00 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA01918 for matrix-sig-people; Fri, 10 Nov 1995 15:20:00 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id PAA01913 for ; Fri, 10 Nov 1995 15:19:57 -0500 Received: from ling-ling.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA06295; Fri, 10 Nov 95 15:18:02 EST Received: by ling-ling.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA01218; Fri, 10 Nov 95 15:19:00 -0500 Message-Id: <9511102019.AA01218@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Fri, 10 Nov 95 15:18:59 -0500 To: "P. Dubois" Subject: Re: [PYTHON MATRIX-SIG] [munro@icf.llnl.gov: Re: Python Matrix extension tutorial] Cc: matrix-sig@python.org, equipe-basis@lists.llnl.gov References: <9511101640.AA18754@kristen.llnl.gov> Sender: owner-matrix-sig@python.org Precedence: bulk First off, I want to mention that I've looked at Yorick and find it a very impressive language for matrix manipulation. If I was still trying to choose between languages like octave or matlab I'd find Yorick a clear winner. The point of this python extension is to put these sorts of features within the python language so that so that we can simultaneously have access to the full power of that programming language (I know of no other "matrix" language in which you can write a full-scale web browser). > In the handout at the meeting on page 7 is an example of the heavily > used Yorick feature I mentioned: You have two arrays > > b with shape (2,3) > c with shape (2,2,3) > > and you want to produce: > > d[i][j][k]= b[i][k] + c[i][j][k] > > {The example would be much clearer if the ranks were given as (2,3) > and (2,4,3), so it only made sense one way.} In Python, this common > operation is carried out by the expression: > > d= add[1,2](b,c) > > Yorick has a special syntax for this same operation: > > d= b(,-,) + c > > which I find much clearer I think that Dave has a different concept of the operation being indicated by add[1,2](b,c) than I do, and than is really suggested by the notion of ranks. add[1,2] means add the vectors in b to the matrices in c. The notion of producing d[i][j][k]= b[i][k] + c[i][j][k] is conceptually different. Nevertheless, I am intrigued by the idea of adding this to the language. > I have the impression that the syntax would present very little > difficulty in Python; on the model of the All and Reverse index > objects, it would be easy to introduce a Pseudo object which did the > reshape: > > d= b[(All,Pseudo,All)] + c > > However, I imagine that the implementor will be loathe to change his > notion of array conformability at this late date. Nevertheless, you > might as well ask to see what he says; probably the change wouldn't > actually break any existing correct code, but merely give meaning to > what would previously have produced a runtime error. That "1-length > wildcard" rule has been a tremendous winner for Yorick; if you can > convince the Python guys to use it, I doubt they would ever regret it. I think that this makes sense, and I'll have a go at implementing it for my next release. The current version of the matrix object is 0.11 for a reason. I'm perfectly happy to consider any and all suggested changes to the language. I think I need to play with this one a little bit to see if I like it (for most of the things that I do ranks really are the right conceptual model), and I'll throw it out so that other people can begin playing with it and form their own opinions. > Yorick has one other indexing syntax which has proven useful, which I > call "rubber indices". They address the problem of writing > interpreted code which extracts slices of arrays when you don't know > beforehand how many dimensions the array has. An example is an > opacity array for which you know that the most slowly varying index > represents photon wavelength, but there might be zero, one, two, or > three faster varying dimensions representing position in space. These > situations can probably be handled with Python's reshape operator, so > again this is a question of ease of use. I think that I like this idea. Let me know if this is what you mean. Currently I have (for c being a 3d array): c[[0,]] <--> c[[0,All, All]] Are you suggesting something like: c[[Rubber, 0]] <--> c[[All, All, 0]]? If you're in fact suggesting something else, I'd like to hear more. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 10 16:42:49 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA00481 for matrix-sig-people; Fri, 10 Nov 1995 16:42:49 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id QAA00476 for ; Fri, 10 Nov 1995 16:42:45 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id QAA23417 (8.6.11/IDA-1.6); Fri, 10 Nov 1995 16:36:22 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA19949; Fri, 10 Nov 1995 16:36:21 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA24733; Fri, 10 Nov 1995 16:36:20 -0500 Date: Fri, 10 Nov 1995 16:36:20 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511102136.QAA24733@cyclone.ERE.UMontreal.CA> To: dubois@kristen.llnl.gov CC: matrix-sig@python.org, equipe-basis@lists.llnl.gov In-reply-to: <9511101640.AA18754@kristen.llnl.gov> (dubois@kristen.llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] [munro@icf.llnl.gov: Re: Python Matrix extension tutorial] Sender: owner-matrix-sig@python.org Precedence: bulk Yorick has one other indexing syntax which has proven useful, which I call "rubber indices". They address the problem of writing interpreted code which extracts slices of arrays when you don't know beforehand how many dimensions the array has. An example is an opacity array for which you know that the most slowly varying index represents photon wavelength, but there might be zero, one, two, or three faster varying dimensions representing position in space. These situations can probably be handled with Python's reshape operator, so again this is a question of ease of use. Maybe it helps to point out how this is handled in J and in my Array module. There indexing is not seen as something special, but just as one of many structural functions. Consequently it has no special syntax (in my array module I have added bracket indexing as a kind of syntactic sugar for the function take()), but is an ordinary function with two arguments: the array to be indexed from, and an index array. Accessing the first (or second etc.) dimension of an array of arbitrary rank is then done by specifying a negative rank for the indexing function. Negative function ranks basically mean that you do not specify the rank of the cells that the function operates on, but the rank of the frames of these cells. Maybe something similar could be implemented in Python, although I don't see with what kind of syntax one could indicate negative ranks for bracket indexing. Personally, I'd like to see an additional indexing function similar in spirit to the J version. I realise it can't replace bracket indexing, since you can't assign to parts of a matrix this way, but it would be a useful addition for many cases. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 10 17:12:55 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA00674 for matrix-sig-people; Fri, 10 Nov 1995 17:12:55 -0500 Received: from alfven.jhuapl.edu (alfven.jhuapl.edu [128.244.146.147]) by python.org (8.6.12/8.6.12) with SMTP id RAA00670 for ; Fri, 10 Nov 1995 17:12:52 -0500 Message-Id: <199511102212.RAA00670@python.org> Received: by alfven.jhuapl.edu (1.38.193.4/16.2) id AA02672; Fri, 10 Nov 1995 17:20:00 -0500 Date: Fri, 10 Nov 1995 17:20:00 -0500 From: Chris Chase S1A To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Yorick, Python and Array.py Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk >>>>> "D" == Dave Munro D> Here is Dave's suggestion: [deleted material about pseudo indexes and comformability in Yorick] Two weeks ago someone pointed me toward the Yorick language for its array capabilities. After studying it, I found that Yorick has the most flexible array subscripting syntax of any language that I have seen. The advantages of Yorick can be added to Python but it requires some major work on some internals. In a reply to one of my previous posts, Guido already said he doesn't mind someone submitting internal changes that keep the language backward compatible. I have been working on these modifications and hope to finish them soon (I don't have much time available for this SIG and tinkering around with Python). I will post a summary of the changes in a separate message that I attempting to get some immediate feedback before I finish my mods. D> In order for this to work, you need to broaden the definition of when D> two arrays are conformable; instead of requiring an exact match for D> the shape of the largest common cells of the two operands, you need to D> treat 1-length dimensions as "wildcards" matching any length. I already have tried this by making a small modification to Array.py, the prototype from Konrad Hinsen . If anyone is interested I can send you the modified version. Array.py already has a concept of wildcard for extending a shorter frame for one argument to match a longer frame of another argument. It does this by combining the lower-rank array will then be combined with each matching ranked cell in the higher ranked array. Yorick calls this broadcasting because the lower ranked cell is "broadcasted" or replicated along the missing higher dimensions. Yorick extends this one step further. Yorick will broadcast along any size 1 dimension to match the common dimension of the other arguments. When combining a lower rank array with a higher rank we can think of the missing higher dimensions as being size 1. (i.e. equivalent to appending 1's to the shape of the shorter frame). Then the broadcasting method will have the same behavior already exhibited by Array.py. However, it has the additional benefit of broadcasting other unit dimensions (a simple viewpoint unit dimensions have similar behave like a scalar.) As an example, using the modified Array.py: b = Array([ [2,3,7], [9,8,2] ]) d = Array([1,2,3]) e = Array([[1],[2]]) # This could already be done. It adds to each row. >>> b+d 3 5 10 10 10 5 # This is new. It adds a column vector containing 1,2 to each column. >>> b+e 3 4 8 11 10 4 It makes outer products extremely simple: >>> d+e 2 3 4 3 4 5 >>> d*e 1 2 3 2 4 6 Pseudo indexes make this type of thing even easier. In the above if e=Array([1,2]) then I could have added e along the columns using: b+e(-,) This is more useful in even higher dimensions. It saves pairs of transposition operators and/or reshaping which can be confusing. D> However, I imagine that the implementor will be loathe to change his D> notion of array conformability at this late date. Nevertheless, you D> might as well ask to see what he says; probably the change wouldn't D> actually break any existing correct code. There is no single "implementor" here. My understanding is that the Matrix module concept is still in flux - hence the need for this SIG. There have been several implementations and design concepts suggested on this list. I have not seen any proposal to accept as a standard a particular Array module yet. D> In Yorick, the combination of the relaxed conformability rule, D> pseudo-indices and a general transpose operator that can produce any D> permutation of dimensions allows for nearly all meaningful D> combinations of arrays. I do like Yorick's completely general transpose() operator which uses a list of permutation cycles combined with circular shifts and negative dimension numbers (taken relative to the rank) to conveniently specify any arbitrary transposition on dimensions. D> Yorick has one other indexing syntax which has proven useful, which I D> call "rubber indices". This is another very useful feature. The idea is that you can have left or right justified index lists relative to array rank. Unspecified dimensions indexes default to the entire dimension. Optionally, the unspecified dimensions can be collapsed into a single dimension that is the product of the collapsed dimensions. Yorick provides a very nice syntax for a rubber index ".." and "*" to collapse the missing indexes. This becomes very useful when the dimension is unknown to the user and provides much simpler and clearer code than a complex sequence of take() and reshape() operators. b(..,2,3) or b(*,2,3) index along the last two dimensions. b(1,..,5) indexes along the first and last dimension. Indexes before the rubber index are left-justified while those following the rubber index are right-justified with respect to the shape of the indexed array. One other subscripting feature that Yorick (Matlab has this also) has is the ability to specify a stride with slice notation. E.g. 1:10:2 specifies a range the has increments of 2. The stride can also be negative. This is like the range() function. I list the additions that I am experimenting with in another post. Chris ------- end ------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 10 17:34:55 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA00807 for matrix-sig-people; Fri, 10 Nov 1995 17:34:55 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id RAA00803 for ; Fri, 10 Nov 1995 17:34:52 -0500 Received: from ling-ling.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA07130; Fri, 10 Nov 95 17:33:02 EST Received: by ling-ling.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA01268; Fri, 10 Nov 95 17:33:53 -0500 Message-Id: <9511102233.AA01268@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Fri, 10 Nov 95 17:33:52 -0500 To: chris.chase@jhuapl.edu Subject: Re: [PYTHON MATRIX-SIG] Yorick, Python and Array.py Cc: matrix-sig@python.org References: <199511102212.RAA00670@python.org> Sender: owner-matrix-sig@python.org Precedence: bulk > The advantages of Yorick can be added to Python but it requires some > major work on some internals. In a reply to one of my previous posts, > Guido already said he doesn't mind someone submitting internal changes > that keep the language backward compatible. I'm not sure that this is true. I've found that for my matrix object (on which I have received almost entirely good reviews from the testers) that this sort of indexing can be added very easily to python using mapping semantics (as Dave mentioned): ie. a[[All,Empty,All]] <--> a(,-,) I do think that it would be nice to remove that extra set of brackets, which Guido is also in favor of; however, I'm just not convinced that any further modifications are needed. For your other examples: a[[Star,2,3]] <--> a(*,2,3) a[[Slice(1,10,2)]] <--> a(1:10:2) I realize that the syntax in Yorick is a little bit more consise, but I'm just not convinced that such fundamental changes to the python core are in fact necessary. Just my two cents worth, Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 10 19:12:03 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id TAA01446 for matrix-sig-people; Fri, 10 Nov 1995 19:12:03 -0500 Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.6.12/8.6.12) with ESMTP id TAA01440 for ; Fri, 10 Nov 1995 19:11:55 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id OAA07141 (8.6.11/IDA-1.6); Thu, 9 Nov 1995 14:47:33 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA19450; Thu, 9 Nov 1995 14:47:11 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA14508; Thu, 9 Nov 1995 14:47:10 -0500 Date: Thu, 9 Nov 1995 14:47:10 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511091947.OAA14508@cyclone.ERE.UMontreal.CA> To: dubois@kristen.llnl.gov CC: matrix-sig@python.org In-reply-to: <9511091910.AA08047@kristen.llnl.gov> (dubois@kristen.llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] A related idea Sender: owner-matrix-sig@python.org Precedence: bulk The point isn't that I want to save memory but that I want to MODIFY the array the compiled code is using, such as an array in a Fortran common block. We should make a difference between Fortran and C interfacing (C standing for all languages that can handle dynamic memory management in a reasonable way). Fortran arrays are always static, so there is no memory leak problem. Increasing the reference count to make sure Python never tries to free the memory should be sufficient. As for C, the best way would be to leave allocation completely to Python. The matrix module would provide functions to allocate and free matrices that can be called from the C code just like malloc() and free(). These functions would take care of the reference from the C code. Of course the C code is also free to handle its own matrices and manage them completely (e.g. workspace in library functions). These would never be accessed from Python; in fact, Python wouldn't even know about their existence. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Nov 11 01:01:44 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id BAA02641 for matrix-sig-people; Sat, 11 Nov 1995 01:01:44 -0500 Received: from retro.jhuapl.edu (retro.jhuapl.edu [128.244.146.239]) by python.org (8.6.12/8.6.12) with ESMTP id BAA02636 for ; Sat, 11 Nov 1995 01:01:40 -0500 Message-Id: <199511110601.BAA02636@python.org> Received: by retro.jhuapl.edu (1.37.109.16/16.2) id AA053859687; Sat, 11 Nov 1995 01:01:27 -0500 Date: Sat, 11 Nov 1995 01:01:27 -0500 From: Chris Chase S1A To: matrix-sig@python.org Cc: James Hugunin Subject: Re: [PYTHON MATRIX-SIG] Yorick, Python and Array.py In-Reply-To: <9511102233.AA01268@ling-ling.lcs.mit.edu.LCS.MIT.EDU> References: <199511102212.RAA00670@python.org> <9511102233.AA01268@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk >>>>> "JH" == James Hugunin writes: JH> I'm not sure that this is true. I've found that for my matrix object (on JH> which I have received almost entirely good reviews from the testers) that JH> this sort of indexing can be added very easily to python using mapping JH> semantics (as Dave mentioned): JH> ie. a[[All,Empty,All]] <--> a(,-,) I guess I am not clear on this. Are All and Empty special objects defined by your matrix module? Aside: Actually, if the additions I had in mind were made (like slice notation) I would write the righthand side as a[:,-,:] or a[:,None,:], avoiding a new syntactic element for an empty argument (not seen elsewhere in the language). If you are concerned about too many extras added to the language, I would suggest that None be equivalent to "-", removing the requirement for your Empty object. JH> I do think that it would be nice to remove that extra set of brackets, JH> which Guido is also in favor of; however, I'm just not convinced that any JH> further modifications are needed. As you have indicated, there are clever workarounds that can accomplish the Yorick type of additions without internal language changes. However, I do not think that we should rule out internal Python additions if they are beneficial but don't break, limit, or violate the spirit/flavor of the language. JH> a[[Star,2,3]] <--> a(*,2,3) Okay. This is good. Would you use something like Dot for the Yorick ".." pseudo index? How do you keep the definitions of Star and Dot from being hidden inadvertently by other definitions in the users code (like an imported star catalog module that defines a Star object)? JH> a[[Slice(1,10,2)]] <--> a(1:10:2) Would the following also work? a[::2,:-4,::-1] <--> a[[Slice(None,None,2),Slice(None,-4),Slice(None,None,-1)]] This is product indexing on a, i.e. the indexes of the resulting elements are taken from the Cartesian product of the index vectors. This example takes every other on the first dimension, up to the fourth from the last on the second dimension, and reverses the last dimension. Of course one may want to mix scalars, slices, and matrices as indexes in the same subscripting expression: a[(1,2,3),5,::-1,b] <--> a[[[1,2,3],5,Slice(None,None,-1),b]] Are you also considering supporting indexing a multi-dimensional array in flattened form when only a single index vector is given? E.g. for the rank 3 a: a[[Slice(1,None,2)]] accessing every other element of a in flattened form. This is useful and common in other array languages. The danger here is that a[[[1,2,3]]] could be mistaken for a[[1,2,3]], where the first is a one-dimensional 3 element vector index into a flattened array and the second is a multi-dimensional index using a scalar for each dimension. (Dropping the extra grouping, the difference a[(1,2,3)] and a[1,2,3] is slightly more evident). JH> I realize that the syntax in Yorick is a little bit more consise, but I'm JH> just not convinced that such fundamental changes to the python core are in JH> fact necessary. Necessary, perhaps not. Worthwhile, I would definitely say yes. (Especially if someone volunteers the work for the changes). At the least, I feel the following would be worthwhile: 1) drop the extra grouping inside [] for multi-dimensional subscripts. 2) support slice expressions with strides wherever a 'test' syntactical element is possible. This internalizes your Slice() functionality. It is a natural extension of slice syntax already available. But requires a special check for calling the current __slice__ functionality for sequences. Mabye: 3?) '-' for pseudo index [not a big benefit but it only takes 3 lines of code, one in Grammar and 2 in compile.c] I have already started implementing these. I think that your workaround for the rubber indexes is just as good as what I had started to implement (probably better in the how it fits into the language philosophy). Aside: It would have been useful if the Python methods for sequence and dictionary types (e.g. __slice__) had originally been defined to take variable length argument lists instead of defined length and type argument lists. Then additions for slices with strides and multi-dimensional indexing would have been easier to add without breaking existing code (argument checking would be up to the method. If more arguments then desired were passed, e.g. 3 instead of 2 for a slice, then either the extra arguments would be ignored or an exception signaled. Similarly for subscripting.). Chris Chase ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Nov 11 12:18:07 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA04154 for matrix-sig-people; Sat, 11 Nov 1995 12:18:07 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id MAA04150 for ; Sat, 11 Nov 1995 12:18:02 -0500 Received: from ling-ling.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA11250; Sat, 11 Nov 95 12:16:06 EST Received: by ling-ling.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA01573; Sat, 11 Nov 95 12:17:11 -0500 Message-Id: <9511111717.AA01573@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Sat, 11 Nov 95 12:17:10 -0500 To: chris.chase@jhuapl.edu Subject: Re: [PYTHON MATRIX-SIG] Yorick, Python and Array.py Cc: matrix-sig@python.org References: <199511102212.RAA00670@python.org> <9511102233.AA01268@ling-ling.lcs.mit.edu.LCS.MIT.EDU> <9511110559.AA08773@mama-bear.LCS.MIT.EDU> Sender: owner-matrix-sig@python.org Precedence: bulk I almost completely agree with your comments (now that I think I better understand what you're talking about). Here is how I see things. Please shout out if there are any major errors. I'm creating special objects All, Empty, Star, and Dot in order to capture the capabilities of Yorick for array subscripting. I've created a slice object to generalize the notion of start:stop:step indexing. Right now, my indexing system can take any combination of arbitrary sequences of ints, slices, or single ints as a full product form index into the matrix. The principle problems with this system are potential naming conflicts (or really ugly names like Matrix.Star) and the ugliness of Slice(None,None,2) vs. ::2. You are working on an extension to the python grammar to remove the extra brackets, make ::2 <--> some builtin python slice object, and "-" equivalent to some builtin python pseudo index object. I'd love to see all of these things happen, and I think that they'd fit in perfectly with my existing matrix object. I'm curious as to how you're implementing your slices. Are you creating a new python C object? This is actually the next step for me with my matrix object, and I'd love to see what you've done so far so that I can try and keep my work compatible with yours (and not duplicate too much effort). -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Nov 12 12:24:09 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA08604 for matrix-sig-people; Sun, 12 Nov 1995 12:24:09 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id MAA08600 for ; Sun, 12 Nov 1995 12:24:05 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id MAA20838 (8.6.11/IDA-1.6); Sun, 12 Nov 1995 12:22:20 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA07878; Sun, 12 Nov 1995 12:22:19 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA19828; Sun, 12 Nov 1995 12:22:18 -0500 Date: Sun, 12 Nov 1995 12:22:18 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511121722.MAA19828@cyclone.ERE.UMontreal.CA> To: jjh@mama-bear.lcs.mit.edu CC: chris.chase@jhuapl.edu, matrix-sig@python.org In-reply-to: <9511111717.AA01573@ling-ling.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Sat, 11 Nov 95 12:17:10 -0500) Subject: Re: [PYTHON MATRIX-SIG] Yorick, Python and Array.py Sender: owner-matrix-sig@python.org Precedence: bulk I'm creating special objects All, Empty, Star, and Dot in order to capture the capabilities of Yorick for array subscripting. I confess that I have not yet completely studied all the discussion about indexing, but let's not get carried away to fast with implementations. We are not creating matrices for Yorick users, but for Python, so the notation we introduce should make sense without referring to Yorick. "All" and "Empty" are fine, but how should anyone make sense of "Star" and "Dot" without learning Yorick first? We should first agree on what features we want (preferably based on real use in other matrix languages) and then think about a notation for them that fits into Pythons general structure. For example, I don't like the idea of having '-' standing for anything else than a mathematical operation; once we start with this, we'll soon have a Perl-compatible letter soup. I've created a slice object to generalize the notion of start:stop:step indexing. That's not the worst solution, but maybe there is a better one. How about permitting lists of indices? That would include the generalized slices if range() is used to create the list. But it would also allow much more general indexing schemes without any further effort. It is probably a bit less efficient, but are generalized slices really used in time-critical parts of matrix algorithms? (Honest question, I don't know!) The principle problems with this system are potential naming conflicts (or really ugly names like Matrix.Star) and the ugliness of Slice(None,None,2) vs. ::2. Indeed. It would not be too difficult to allow ::2 as valid syntax, but that raises the question of what this means for other Python objects. For consistency it should be implemented everywhere. Is anyone volunteering to do that? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Nov 12 13:32:03 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA08811 for matrix-sig-people; Sun, 12 Nov 1995 13:32:03 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id NAA08807 for ; Sun, 12 Nov 1995 13:32:00 -0500 Received: from ling-ling.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA16595; Sun, 12 Nov 95 13:29:47 EST Received: by ling-ling.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA01870; Sun, 12 Nov 95 13:30:59 -0500 Message-Id: <9511121830.AA01870@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Sun, 12 Nov 95 13:30:58 -0500 To: hinsenk@ere.umontreal.ca (Hinsen Konrad) Subject: Re: [PYTHON MATRIX-SIG] Yorick, Python and Array.py Cc: matrix-sig@python.org References: <199511121722.MAA19828@cyclone.ERE.UMontreal.CA> Sender: owner-matrix-sig@python.org Precedence: bulk > I'm creating special objects All, Empty, Star, and Dot in order to capture > the capabilities of Yorick for array subscripting. > > I confess that I have not yet completely studied all the discussion > about indexing, but let's not get carried away to fast with > implementations. We are not creating matrices for Yorick users, but > for Python, so the notation we introduce should make sense without > referring to Yorick. "All" and "Empty" are fine, but how should > anyone make sense of "Star" and "Dot" without learning Yorick first? I love the discussions that this list has generated, nonetheless I have found that sometimes they remain a little bit too divorced from actual implementations. I'm trying to implement some of the more interesting ideas in the hopes of myself and others being able to play with them, not as an attempt to impose a standard! > We should first agree on what features we want (preferably based on > real use in other matrix languages) and then think about a notation > for them that fits into Pythons general structure. For example, I > don't like the idea of having '-' standing for anything else than > a mathematical operation; once we start with this, we'll soon have > a Perl-compatible letter soup. This is a very good point. > I've created a slice object to generalize the notion of start:stop:step indexing. > > That's not the worst solution, but maybe there is a better one. How > about permitting lists of indices? That would include the generalized > slices if range() is used to create the list. But it would also allow > much more general indexing schemes without any further effort. It is > probably a bit less efficient, but are generalized slices really used > in time-critical parts of matrix algorithms? (Honest question, I don't > know!) Gee Konrad, I thought you had been playing with my matrices. I already allow lists of indices, and it's true, you can create generalized slices using range. The slice object is there in order to make things a lot easier. One thing that I really like about slices is that you can leave off the start or end points, and have them assigned appropriately for the array (just as in normal python slice indexing). This is a LOT easier to use than range. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Nov 12 13:49:18 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA08864 for matrix-sig-people; Sun, 12 Nov 1995 13:49:18 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id NAA08860 for ; Sun, 12 Nov 1995 13:49:15 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id NAA22593 (8.6.11/IDA-1.6); Sun, 12 Nov 1995 13:47:32 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA21743; Sun, 12 Nov 1995 13:47:32 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA22095; Sun, 12 Nov 1995 13:47:31 -0500 Date: Sun, 12 Nov 1995 13:47:31 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511121847.NAA22095@cyclone.ERE.UMontreal.CA> To: jjh@mama-bear.lcs.mit.edu CC: matrix-sig@python.org In-reply-to: <9511121830.AA01870@ling-ling.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Sun, 12 Nov 95 13:30:58 -0500) Subject: Re: [PYTHON MATRIX-SIG] Yorick, Python and Array.py Sender: owner-matrix-sig@python.org Precedence: bulk implementations. I'm trying to implement some of the more interesting ideas in the hopes of myself and others being able to play with them, not as an attempt to impose a standard! Fine, but there is always the danger that the first working implementation sets a de-facto standard. That's how we got "industry standards" like DOS, Windows, or the IBM PC architecture. Maybe it would be a good idea to use very complicated names for things that are just experimental. That way noone would want to keep them ;-) Gee Konrad, I thought you had been playing with my matrices. I already Not as much as I would like... easier. One thing that I really like about slices is that you can leave off the start or end points, and have them assigned appropriately for the array (just as in normal python slice indexing). This is a LOT easier to use than range. True. So how about allowing three-argument slices? Fixing the parser is easy; implementing the feature for other sequence types is probably some more work, but feasible. There is one semantic problem, which also exists for your Slice() objects: how is assignment to such generalized slices defined if the number of elements in the new-value object doesn't match the number of elements in the slice? Assigning to "traditional" slices simply changes the length of a sequence. Of course there is always the possibility of raising an error, but maybe there is some useful interpretation of this. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Nov 13 06:58:42 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id GAA12398 for matrix-sig-people; Mon, 13 Nov 1995 06:58:42 -0500 Received: from muecke.appl-math.tu-muenchen.de (muecke.appl-math.tu-muenchen.de [129.187.242.1]) by python.org (8.6.12/8.6.12) with SMTP id GAA12391 for ; Mon, 13 Nov 1995 06:58:01 -0500 Received: from piranha.appl-math.tu-muenchen.de by muecke.appl-math.tu-muenchen.de via SMTP (931110.SGI/911001.SGI) for matrix-sig@python.org id AA15406; Mon, 13 Nov 95 12:57:50 +0100 Received: by piranha.appl-math.tu-muenchen.de (940816.SGI.8.6.9/911001.SGI) for matrix-sig@python.org id MAA15261; Mon, 13 Nov 1995 12:57:38 +0100 From: "Thomas Schwaller" Message-Id: <9511131257.ZM15259@piranha.appl-math.tu-muenchen.de> Date: Mon, 13 Nov 1995 12:57:32 +0100 X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Speed...cg...and more! Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-matrix-sig@python.org Precedence: bulk Hi all. Unfortunately I missed the first 5 weeks or so :-( of the very interesting but sometimes quite heavy discussion! So let me begin slow.... ;-) 1) The ofunc object is very interesting, but I miss a feature which I always missed also in the mathmodule.c of the Python distribution. When I want to use e.g. sin(a) of an not predefined type (the new math functions in Matrix just extend the standard ones by comples numbers, matrices,..) I would like that sin (and all the other ones) test at last if the object has a sin attribute and execute it. In Mathmodule.c this was just a minor hack but here I don't now. Why do I want that behaviour. I write a lot of math stuff (ince contributes a 1D automatic differentiation module) and I also want to be able to call sin(a) of these types. In case of the Matrix objects one problem we will ultimately be faced with is the problem of using e.g. exp(A) but not in the pointwise sense but in the mathematical sense (defined by power series for example) So what to do then. Extending exp in python is done very eay: from Matrix import * Matrix_sin=sin def sin(x): """uses builtin sin function or attribute sin""" try: return Matrix_sin(x) except AttributeError: if hasattr(x, 'sin'): return x.sin() else: raise AttributeError, 'sin not available for this type' but I think this should be in the python code, so I just can do something like class dummy: def __init__(self, x) self.x=x def sin(self): return sin(x), sin(x-1) a=dummy() print sin(a) This is a thing which should be there once and for all. 2) When writing mathematical stuff with matrices one uses very often the multiplication (in the Matrix sense) with transposed matrices. Something like: s = Multiply(A.transpose(), r) Is it possible to write that directly with the available methods. When doing this kind of operation you don't really need to trnaspose the matrix, just switch the order of the the loops. When you do a lot of mulitplications with a lot of transposed matrices this is quite important. Please tell me when I'm missing something here. As far as I understand, at the moment you can influence the indices, but can you also influence the order of execution whitout producing a new matrix on the fly? As a goodie here's a simple conjugate gradient method for testing: (Hope it' correct!) from Matrix import * import sys def cg(A, b, x=None, eps=1e-12, k=None, printer=None): m, n = A.shape if k == None: k = m*2 # indeed much too high! if x == None: x, r = zeros(n,1), b else : r = b - Multiply(A, x) p = Multiply(A.transpose(), r) gamma = dot(p, p) for j in range(k): q = Multiply(A, p) alpha = gamma / dot(q, q) x = x + alpha*p r = r - alpha*q s = Multiply(A.transpose(), r) gamma_new = dot(s, s) beta = gamma_new / gamma gamma = gamma_new if gamma[0] <= eps*eps: return x p = s + beta*p if printer: printer(x, r, s, p, q, beta, gamma) raise error,'cg-method diverges with %d iterations and eps = %.0e' % (k,eps) def cg_test(): def pr(x, r, s, p, q, beta, gamma): print gamma[0] n =20 b= ones(n, 1) for i in range(100): a = rand(n, n) a=0.5*(a+a.transpose()) x = cg(a, b) sys.stdout.write('.') sys.stdout.flush() cg_test() 3) As a matter of style, perhaps we should discuss how to write such things. I guess as soon as the matrixmodule will be in the standard distribution there will be a lot of matlab style writing. We should try to keep the procedures, classes ... as uniform as possible, so that everybody can rely on a certain behaviour. We want to be better than Matlab with such a powerfull Language as Python, don't we? 4) Last but not least I need a simple example how to use matrixobjects writing own extension. Suppose I have a method for comuting the first eigenvalue of a sqaure matrix: int n=1000; double **a=....; double lambda1=first_eigenvalue(a, n); What is the standard way of using that in python? a=Matrix_(n,n) lambda1=first_eigenvalue(a) I mean how should I produce (access) a double** having a matrixobject? (this is not a question about how to write an extenssion, just about how to use the matrixobject :-)) 5) Are the handouts of the mentionned talk about the matrix module available for all? Would be nice! Tom ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Nov 13 08:32:46 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id IAA12647 for matrix-sig-people; Mon, 13 Nov 1995 08:32:46 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id IAA12643 for ; Mon, 13 Nov 1995 08:32:43 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id IAA03838 (8.6.11/IDA-1.6); Mon, 13 Nov 1995 08:29:36 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id IAA16739; Mon, 13 Nov 1995 08:29:34 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id IAA29436; Mon, 13 Nov 1995 08:29:33 -0500 Date: Mon, 13 Nov 1995 08:29:33 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511131329.IAA29436@cyclone.ERE.UMontreal.CA> To: et@appl-math.tu-muenchen.de CC: matrix-sig@python.org In-reply-to: <9511131257.ZM15259@piranha.appl-math.tu-muenchen.de> (et@appl-math.tu-muenchen.de) Subject: Re: [PYTHON MATRIX-SIG] Speed...cg...and more! Sender: owner-matrix-sig@python.org Precedence: bulk 1) The ofunc object is very interesting, but I miss a feature which I always missed also in the mathmodule.c of the Python distribution. When I want to use e.g. sin(a) of an not predefined type (the new math functions in Matrix just extend the standard ones by comples numbers, matrices,..) I am working on a "general" math module that provides all the standard functions for arbitrary objects. It calls functions from math for ints and floats, functions from cmath for complex numbers, functions from matrix for matrices, and looks for a method in all other objects. Originally I wanted to do that in Python, but that turned out to detetriorate performance drastically. The C module I am working on now should have no noticeable overhead. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Nov 13 09:40:53 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA13053 for matrix-sig-people; Mon, 13 Nov 1995 09:40:53 -0500 Received: from muecke.appl-math.tu-muenchen.de (muecke.appl-math.tu-muenchen.de [129.187.242.1]) by python.org (8.6.12/8.6.12) with SMTP id JAA13049 for ; Mon, 13 Nov 1995 09:40:45 -0500 Received: from piranha.appl-math.tu-muenchen.de by muecke.appl-math.tu-muenchen.de via SMTP (931110.SGI/911001.SGI) for matrix-sig@python.org id AA18737; Mon, 13 Nov 95 15:40:52 +0100 Received: by piranha.appl-math.tu-muenchen.de (940816.SGI.8.6.9/911001.SGI) for matrix-sig@python.org id PAA16803; Mon, 13 Nov 1995 15:40:45 +0100 From: "Thomas Schwaller" Message-Id: <9511131540.ZM16801@piranha.appl-math.tu-muenchen.de> Date: Mon, 13 Nov 1995 15:40:36 +0100 X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Speed...cg...and more! Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-matrix-sig@python.org Precedence: bulk Sorry for the typos in my last message. Here are some other things I forgot there. 1) What about a Matrix constructor which can use constant values and functions for initailising. e.g. from math import * from Matrix import * f=lambda i, j: sin(i)*cos(j) a=Matrix(1000,1000, func=f) #square 1000x1000 matrix b=a=Matrix_d(10,10,10, func=lambda i, j, k: sin(i)*cos(j)*exp(k) #10x10x10 tensor If we do that in the standard way it will not be fast enough. I nearly always have this problem when using some procedure which need functions as input. I C-code it's no problem at all and after introducing it in python it's just to slow (e.g. minimize a function...). Imagine that the function is evaluated 100000 times! What to do. If we could compile simple functions on the fly this would be just fine. Not a general Python Compiler, but just a tiny one for certain things. Other possibility: Dynamic loading (not really acceptable for such things (interactivity is lost a little bit) Or write some new module for that type of work (Some Parser thigs!) The above example would go as b=Matrix(1000,1000, "sin(i)*cos(j)") or Matrix(1000,1000, "i,j->sin(i)*cos(j)") Comments? Tom ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Nov 13 10:53:17 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA00212 for matrix-sig-people; Mon, 13 Nov 1995 10:53:17 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id KAA00208 for ; Mon, 13 Nov 1995 10:53:12 -0500 Received: from ling-ling.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA21521; Mon, 13 Nov 95 10:49:46 EST Received: by ling-ling.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA02186; Mon, 13 Nov 95 10:51:05 -0500 Message-Id: <9511131551.AA02186@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Mon, 13 Nov 95 10:51:04 -0500 To: hinsenk@ere.umontreal.ca (Hinsen Konrad) Subject: Re: [PYTHON MATRIX-SIG] Speed...cg...and more! Cc: et@appl-math.tu-muenchen.de, matrix-sig@python.org References: <199511131329.IAA29436@cyclone.ERE.UMontreal.CA> Sender: owner-matrix-sig@python.org Precedence: bulk > 1) The ofunc object is very interesting, but I miss a feature which I always > missed also in the mathmodule.c of the Python distribution. When I want to > use e.g. sin(a) of an not predefined type (the new math functions in > Matrix just extend the standard ones by comples numbers, matrices,..) > > I am working on a "general" math module that provides all the standard > functions for arbitrary objects. It calls functions from math for ints > and floats, functions from cmath for complex numbers, functions from > matrix for matrices, and looks for a method in all other objects. > > Originally I wanted to do that in Python, but that turned out to > detetriorate performance drastically. The C module I am working on now > should have no noticeable overhead. And here I've been working on adding these very same features to my ofuncobjects. I'm really think that this is the "right" place to do this, and then to produce a single omathmodule.c that does the right thing efficiently for all object types. Let's talk off-line about this Konrad so we don't end up duplicating each others efforts here. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Nov 13 11:22:16 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA00474 for matrix-sig-people; Mon, 13 Nov 1995 11:22:16 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id LAA00470 for ; Mon, 13 Nov 1995 11:22:14 -0500 Received: from ling-ling.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA21767; Mon, 13 Nov 95 11:18:59 EST Received: by ling-ling.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA02200; Mon, 13 Nov 95 11:20:18 -0500 Message-Id: <9511131620.AA02200@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Mon, 13 Nov 95 11:20:16 -0500 To: "Thomas Schwaller" Subject: Re: [PYTHON MATRIX-SIG] Speed...cg...and more! Cc: matrix-sig@python.org References: <9511131257.ZM15259@piranha.appl-math.tu-muenchen.de> Sender: owner-matrix-sig@python.org Precedence: bulk > 1) See my previous post. > 2) When writing mathematical stuff with matrices one uses very often > the multiplication (in the Matrix sense) with transposed matrices. > Something like: > s = Multiply(A.transpose(), r) The current implementation of matrix multiply sucks! I do very little linear algebra work, so I haven't gotten around to doing this properly yet. I'll bear this in mind when I do get around to doing things right. The following is the current implementation of matrix multiplication. def Multiply(a,b): return add.inner(multiply, a, b.transpose()) You should be able to use: s = add.inner(multiply, A, r).transpose() I'm sure that the extra transpose at the end will add very little overhead to your code (compared to the matrix multiplication itself). Thanks for the CG code, this is exactly the sort of stuff I'd love to have people start building into standard libraries! > 3) As a matter of style, perhaps we should discuss how to write such things. > I guess as soon as the matrixmodule will be in the standard distribution > there will be a lot of matlab style writing. We should try to keep the > procedures, classes ... as uniform as possible, so that everybody can > rely on a certain behaviour. We want to be better than Matlab with such a > powerfull Language as Python, don't we? Good point. I'd love to see people out there write some proposed style guides for matrix functions! > 4) Last but not least I need a simple example how to use matrixobjects > writing own extension. This is currently very poorly supported in the current 0.11 alpha release (expect this to change in later releases). Look at matrixmodule.c matrix_sort() for a much more complicated example than your own. Also, matrices are stored as contiguous memory blocks with a single pointer, to get the traditional C **, you need to generate it yourself (this too will be supported in a friendly way when I have time). > 5) Are the handouts of the mentionned talk about the matrix module available > for all? Would be nice! I agree! And from your followup: > f=lambda i, j: sin(i)*cos(j) > a=Matrix(1000,1000, func=f) #square 1000x1000 matrix Here's how I'd do that using ofuncs i = mrange(1000) j = mrange(1000) a = multiply.outer(sin(i), cos(j)) > b=a=Matrix_d(10,10,10, func=lambda i, j, k: sin(i)*cos(j)*exp(k) i = mrange(10) j = mrange(10) k = mrange(10) a1 = multiply.outer(sin(i), cos(j)) #note, this tempory is only needed for clarity b = a = multiply.outer(a1, exp(k)) Okay, I admit that this is uglier than you'd like, but it runs nearly as fast as the hand-coded C would. It might be possible to write some sort of matrix constructor function that could "compile" to this ofunc form on the fly. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Nov 13 15:05:50 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA01861 for matrix-sig-people; Mon, 13 Nov 1995 15:05:50 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id PAA01857 for ; Mon, 13 Nov 1995 15:05:46 -0500 Received: from ling-ling.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA23502; Mon, 13 Nov 95 15:03:50 EST Received: by ling-ling.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA02648; Mon, 13 Nov 95 15:05:10 -0500 Message-Id: <9511132005.AA02648@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Mon, 13 Nov 95 15:05:10 -0500 To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Matrix multiplies and syntactic nits Sender: owner-matrix-sig@python.org Precedence: bulk I'm working on putting out the 0.2 version of my matrix object, and I'm having a surprisingly hard time resolving some very silly naming issues. 1) There is a single python file currently called Matrix.py which imports everything you could ever want for basic functions on matrix objects. This module has a similar role as Tkinter does in the Tk module system. It defines functions and a number of useful constants. Should I rename this module to Mx.py, or something similar so that people will be more inclined to use "import Mx" rather than "from Matrix import *" (which is what I use all the time). This idea comes directly from Tkinter related discussions on the main list. 2) I have two possible ways to implement matrix multiplies: a) a % b I personally never need the modulo operator for matrices, and it does have enough of a resemblence to X to be reasonably mnemonic. However, this breaks the current conceptual elegance of the system where every operator works on matrices (more or less) exactly the way it works on python scalars. b) a.matrixMultiply(b) This is the obvious solution, however it does make matrix equations extremely ugly to type. Possibly an abbreviation is called for in this particular case? Any opinions? Note: There seem to be enough people with very reasonable objections to the idea of controlling "*" with a global variable that I don't consider this one of the possible solutions. 3) String representations of matrices I'm not talking about the normally printed representation here, that's a major issue, not a nit. I'm trying to decide between two string representations and again I just wanted some feedback. The following are 2x3 matrices of integers. a) Matrix_i((1,2,3),(4,5,6)) b) matrix([[1,2,3],[4,5,6]], 'i') Opinions on which you should get with str(M)? Thanks for the feedback - Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Nov 13 15:53:08 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA02128 for matrix-sig-people; Mon, 13 Nov 1995 15:53:08 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id PAA02124 for ; Mon, 13 Nov 1995 15:53:04 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id PAA21796 (8.6.11/IDA-1.6); Mon, 13 Nov 1995 15:51:15 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA06137; Mon, 13 Nov 1995 15:51:14 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA25492; Mon, 13 Nov 1995 15:51:13 -0500 Date: Mon, 13 Nov 1995 15:51:13 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511132051.PAA25492@cyclone.ERE.UMontreal.CA> To: jjh@mama-bear.lcs.mit.edu CC: matrix-sig@python.org In-reply-to: <9511132005.AA02648@ling-ling.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Mon, 13 Nov 95 15:05:10 -0500) Subject: Re: [PYTHON MATRIX-SIG] Matrix multiplies and syntactic nits Sender: owner-matrix-sig@python.org Precedence: bulk defines functions and a number of useful constants. Should I rename this module to Mx.py, or something similar so that people will be more inclined to use "import Mx" rather than "from Matrix import *" (which is what I use I don't see any need to do this. Anyone can always define a shorter name by writing import Matrix Mx = Matrix So I'd keep the "official" name understandable. 2) I have two possible ways to implement matrix multiplies: a) a % b NOOOOOOOOOO! enough of a resemblence to X to be reasonably mnemonic. However, this breaks the current conceptual elegance of the system where every operator works on matrices (more or less) exactly the way it works on python scalars. It's not just a matter of conceptual elegance, but of being able to write code that works identically with matrices and scalars. And of easily understanding what a certain piece of code does. b) a.matrixMultiply(b) This is the obvious solution, however it does make matrix equations extremely ugly to type. Possibly an abbreviation is called for in this particular case? Probably yes. But I'd still keep the long name for those who want to write clear code. The following are 2x3 matrices of integers. a) Matrix_i((1,2,3),(4,5,6)) b) matrix([[1,2,3],[4,5,6]], 'i') Opinions on which you should get with str(M)? I'd prefer the first. In fact, I'd prefer a name like "IntMatrix" instead of "Matrix_i". It is not at all obvious that the attached letter indicates the type. I don't like the second notation at all; it somehow implies that 'i' is a string argument that could have any value. If you want a notation that indicates the type as an argument, why not use types.IntType instead of 'i'? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Nov 13 18:08:41 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA03222 for matrix-sig-people; Mon, 13 Nov 1995 18:08:41 -0500 Received: from bartel.jhuapl.edu (bartel.jhuapl.edu [128.244.146.143]) by python.org (8.6.12/8.6.12) with SMTP id SAA03218 for ; Mon, 13 Nov 1995 18:08:38 -0500 Message-Id: <199511132308.SAA03218@python.org> Received: by bartel.jhuapl.edu (1.38.193.4/16.2) id AA04649; Mon, 13 Nov 1995 18:12:44 -0500 Date: Mon, 13 Nov 1995 18:12:44 -0500 From: Chris Chase S1A To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] flattened array indexing Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk I had suggested before that the "[]" syntax support both multi-dimensional product indexing and flattened array indexing. However, with the proposed M[i,j] <-> M[(i,j)] trying to also implement flattened indexing with the same syntax could cause some problems. Instead it might be better to have a separate M.flat(i) method for flattened indexing. I would then assume when i is a scalar M[i] is "hierarchical" indexing like Jim Fulton has suggested (i.e. M[i] returns a one lower dimensional array) I assume that if M has 3 dimensions then M[1,2] would be an error for not specifying the correct number of indexes. A question: M 1-dimensional. i = (1,2,3) Is the following a "wrong number of indexes" error or will 1D arrays be treated differently? M[i] <-> M[(1,2,3)] Would one have to resort to: M[(i,)] <-> M[((1,2,3))] ? Chris Chase ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Nov 16 09:00:44 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA04505 for matrix-sig-people; Thu, 16 Nov 1995 09:00:44 -0500 Received: from muecke.appl-math.tu-muenchen.de (muecke.appl-math.tu-muenchen.de [129.187.242.1]) by python.org (8.6.12/8.6.12) with SMTP id IAA04477 for ; Thu, 16 Nov 1995 08:57:53 -0500 Received: from piranha.appl-math.tu-muenchen.de by muecke.appl-math.tu-muenchen.de via SMTP (931110.SGI/911001.SGI) for matrix-sig@python.org id AA25573; Thu, 16 Nov 95 14:57:34 +0100 Received: by piranha.appl-math.tu-muenchen.de (940816.SGI.8.6.9/911001.SGI) for matrix-sig@python.org id OAA27143; Thu, 16 Nov 1995 14:57:32 +0100 From: "Thomas Schwaller" Message-Id: <9511161457.ZM27141@piranha.appl-math.tu-muenchen.de> Date: Thu, 16 Nov 1995 14:57:24 +0100 X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: matrix-sig@python.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-matrix-sig@python.org Precedence: bulk Hi all, when writing my first matrix algorithms I faced a "problem" of "inconsistency" concerning matrices and vectors. Suppose a naive user being used to Matlab, Scilab or octave. He/she want's to to his/her first matrix computations: a=Matrix([1,2,3],[2,3,4],[4,5,6]) b=ones(3) Multiply(a,b) Traceback (innermost last): File "", line 1, in ? File "/home/et/Python-1.3/Lib/Matrix.py", line 114, in Multiply return add.inner(multiply, a, b.transpose()) Matrix.error: 1st dimension invalid Aha he/she thinks, b ia a row vector. So let's transpose it b=ones(3).transpose() Traceback (innermost last): File "", line 1, in ? Matrix.error: 1st dimension invalid So wat the hell is going on here, (s)he thinks. So let's try b=ones(3, 1) Multiply(a,b) [[6.0], [9.0], [15.0]] Ah yes, well let's compute a dot product of b and compare it with something: if dot(b,b) == 3.0: print 'ok' else : print 'not ok' not ok Hm.. print dot(b,b) [3.0] Ah, I see.. if dot(b,b)[0] == 3.0: print 'ok' else : print 'not ok' ok At this point our potential user who heard really hot things about the python matrix class will probably not use it anymore. What can we do to change his/her opinion, because WE know it's still a very hot topic and we all love it, don't we? :-) ### END OF THE NIGHTMARE TALE ;;;;-)) TOM ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Nov 16 09:37:13 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA04663 for matrix-sig-people; Thu, 16 Nov 1995 09:37:13 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id JAA04659 for ; Thu, 16 Nov 1995 09:37:09 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id JAA19902 (8.6.11/IDA-1.6); Thu, 16 Nov 1995 09:34:27 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA19952; Thu, 16 Nov 1995 09:34:27 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA21776; Thu, 16 Nov 1995 09:34:26 -0500 Date: Thu, 16 Nov 1995 09:34:26 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511161434.JAA21776@cyclone.ERE.UMontreal.CA> To: et@appl-math.tu-muenchen.de CC: matrix-sig@python.org In-reply-to: <9511161457.ZM27141@piranha.appl-math.tu-muenchen.de> (et@appl-math.tu-muenchen.de) Sender: owner-matrix-sig@python.org Precedence: bulk a=Matrix([1,2,3],[2,3,4],[4,5,6]) b=ones(3) Multiply(a,b) Traceback (innermost last): File "", line 1, in ? File "/home/et/Python-1.3/Lib/Matrix.py", line 114, in Multiply return add.inner(multiply, a, b.transpose()) Matrix.error: 1st dimension invalid This ought to work, but Multiply() is still a bit broken. Try add.inner(multiply, a, b) which does what you want. Multiply does the same, but first transposes b (don't ask me why!), which produces the error message. Aha he/she thinks, b ia a row vector. So let's transpose it b=ones(3).transpose() This ought to work, and return b. So transpose() is also broken... At this point our potential user who heard really hot things about the python matrix class will probably not use it anymore. At this point the matrix class is still in its early stages! Patience... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Nov 16 12:00:38 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA05481 for matrix-sig-people; Thu, 16 Nov 1995 12:00:38 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id MAA05476 for ; Thu, 16 Nov 1995 12:00:33 -0500 Received: from ling-ling.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA19928; Thu, 16 Nov 95 11:57:06 EST Received: by ling-ling.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA01382; Thu, 16 Nov 95 11:58:16 -0500 Message-Id: <9511161658.AA01382@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Thu, 16 Nov 95 11:58:15 -0500 To: "Thomas Schwaller" Subject: [PYTHON MATRIX-SIG] Linear Algebra Cc: matrix-sig@python.org References: <9511161457.ZM27141@piranha.appl-math.tu-muenchen.de> Sender: owner-matrix-sig@python.org Precedence: bulk Warning, I spend almost all of my time doing signal processing type work, so you should expect that the linear algebra side of my matrix object is not quite right. Please keep sending these sorts of comments so that I can figure out what the linear algebra types want (except for * <--> matrixMultiply) and implement it properly. Konrad summed up most of my response in the following lines: > This ought to work, but Multiply() is still a bit broken. > This ought to work, and return b. So transpose() is also broken... > At this point the matrix class is still in its early stages! > Patience... I'll fix both of these for the next release (I'd been hoping for this evening, but Friday is looking more likely). Keep the bug reports coming! -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Nov 16 12:01:40 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA05481 for matrix-sig-people; Thu, 16 Nov 1995 12:00:38 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id MAA05476 for ; Thu, 16 Nov 1995 12:00:33 -0500 Received: from ling-ling.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA19928; Thu, 16 Nov 95 11:57:06 EST Received: by ling-ling.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA01382; Thu, 16 Nov 95 11:58:16 -0500 Message-Id: <9511161658.AA01382@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Thu, 16 Nov 95 11:58:15 -0500 To: "Thomas Schwaller" Subject: [PYTHON MATRIX-SIG] Linear Algebra Cc: matrix-sig@python.org References: <9511161457.ZM27141@piranha.appl-math.tu-muenchen.de> Sender: owner-matrix-sig@python.org Precedence: bulk Warning, I spend almost all of my time doing signal processing type work, so you should expect that the linear algebra side of my matrix object is not quite right. Please keep sending these sorts of comments so that I can figure out what the linear algebra types want (except for * <--> matrixMultiply) and implement it properly. Konrad summed up most of my response in the following lines: > This ought to work, but Multiply() is still a bit broken. > This ought to work, and return b. So transpose() is also broken... > At this point the matrix class is still in its early stages! > Patience... I'll fix both of these for the next release (I'd been hoping for this evening, but Friday is looking more likely). Keep the bug reports coming! -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Nov 16 12:29:24 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA05688 for matrix-sig-people; Thu, 16 Nov 1995 12:29:24 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id MAA05684 for ; Thu, 16 Nov 1995 12:29:18 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id MAA25852 (8.6.11/IDA-1.6); Thu, 16 Nov 1995 12:25:06 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA01528; Thu, 16 Nov 1995 12:25:05 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA03038; Thu, 16 Nov 1995 12:25:04 -0500 Date: Thu, 16 Nov 1995 12:25:04 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199511161725.MAA03038@cyclone.ERE.UMontreal.CA> To: jjh@mama-bear.lcs.mit.edu CC: matrix-sig@python.org In-reply-to: <9511161658.AA01382@ling-ling.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Thu, 16 Nov 95 11:58:15 -0500) Subject: Re: [PYTHON MATRIX-SIG] Linear Algebra Sender: owner-matrix-sig@python.org Precedence: bulk quite right. Please keep sending these sorts of comments so that I can figure out what the linear algebra types want (except for * <--> matrixMultiply) and implement it properly. Let's start with something easy: transpose(). It should work for arrays of any rank and reverse the shape. A useful addition would be an optional argument that is a list (or tuple or array) representing a permutation of range(len(x.shape)). That would allow an arbitrary reshuffling of axes. I'll send some comments on multiplication later. For now just that: Multiply() and Dot() are really one and the same function, if done correctly. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 17 12:44:15 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA12169 for matrix-sig-people; Fri, 17 Nov 1995 12:44:15 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id MAA12165 for ; Fri, 17 Nov 1995 12:44:12 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA18457; Fri, 17 Nov 95 09:43:43 PST Received: by kristen.llnl.gov (5.x/SMI-SVR4) id AA29337; Fri, 17 Nov 1995 09:43:42 -0800 Date: Fri, 17 Nov 1995 09:43:42 -0800 Message-Id: <9511171743.AA29337@kristen.llnl.gov> From: "P. Dubois" To: jjh@mama-bear.lcs.mit.edu Cc: matrix-sig@python.org In-Reply-To: <9511132005.AA02648@ling-ling.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Mon, 13 Nov 95 15:05:10 -0500) Subject: Re: [PYTHON MATRIX-SIG] Matrix multiplies and syntactic nits Sender: owner-matrix-sig@python.org Precedence: bulk Jim Hugunin wrote: > The following are 2x3 matrices of integers. > a) Matrix_i((1,2,3),(4,5,6)) > b) matrix([[1,2,3],[4,5,6]], 'i') > Opinions on which you should get with str(M)? I prefer Matrix_i. I would really prefer something explicit like IntegerMatrix, CharacterMatrix, etc, with Matrix reserved as euphemism for RealMatrix or as a "smart" constructor as I discussed before. It turns out you just don't use this kind of constructor much anyway, as most matrices are produced by array functions/operators, not literal lists. In the particular case of python there is some history having to do with using characters to represent popular types, so the present choice is reasonable and I don't feel strongly about this. "If it isn't an abbreviation, you don't have to remember which abbreviation it is." Paul ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Nov 17 16:32:20 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA13743 for matrix-sig-people; Fri, 17 Nov 1995 16:32:20 -0500 Received: from mama-bear.LCS.MIT.EDU (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id QAA13739 for ; Fri, 17 Nov 1995 16:32:17 -0500 Received: from ling-ling.lcs.mit.edu.LCS.MIT.EDU by mama-bear.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA07160; Fri, 17 Nov 95 16:29:57 EST Received: by ling-ling.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA01814; Fri, 17 Nov 95 16:31:15 -0500 Message-Id: <9511172131.AA01814@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Fri, 17 Nov 95 16:31:14 -0500 To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] prototype release status report Sender: owner-matrix-sig@python.org Precedence: bulk Just to keep people informed of the current status of the next release of my matrix object (which I had been claiming would be ready last night). I read Tom's message yesterday and realized that I really need to get my matrixMultiply and transpose operators working properly ASAP. I spent most of the day today crunching code to make this happen, and it's still not quite working. The challenge is to support APL style matrixMultiply[1,2] (and permutations of that) to multiply an array of vectors by an array of matrices in the proper fashion. I should have this working sometime this weekend and then I'll make the new release available. Sorry to keep people waiting. -Jim ---------- Quick preview of the most exciting things in the next release: type coercion is implemented for matrices, ie. Matrix_i(1,2,3)*Matrix_f(1.,2.,3.) = Matrix_f(1.,4.,9.) matrix printing is now controlled by a callback to a user-settable python function for prettier matrix printing. a**b works for pow(a,b) (Thanks to Konrad) a[1,2,3] works for multidimensional indexing (Thanks to Konrad) pickling matrices works and much more. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Nov 20 10:09:13 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA26087 for matrix-sig-people; Mon, 20 Nov 1995 10:09:13 -0500 Received: from muecke.appl-math.tu-muenchen.de ([129.187.242.1]) by python.org (8.6.12/8.6.12) with SMTP id KAA26083 for ; Mon, 20 Nov 1995 10:07:58 -0500 Received: from piranha.appl-math.tu-muenchen.de by muecke.appl-math.tu-muenchen.de via SMTP (931110.SGI/911001.SGI) for matrix-sig@python.org id AA23777; Mon, 20 Nov 95 16:07:20 +0100 Received: by piranha.appl-math.tu-muenchen.de (940816.SGI.8.6.9/911001.SGI) for matrix-sig@python.org id QAA10267; Mon, 20 Nov 1995 16:07:19 +0100 From: "Thomas Schwaller" Message-Id: <9511201607.ZM10265@piranha.appl-math.tu-muenchen.de> Date: Mon, 20 Nov 1995 16:07:08 +0100 X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Matrix Plot module (small announcement) Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-matrix-sig@python.org Precedence: bulk Hi folks, I promised to rewrite the plmodule for use with the matrixmodule. The next message will contain the first test version of the module. This one contains a test script and 'HOW TO BUILD' remarks. Trivial for most of us, but nevertheless... GENERAL REMARKS on plplot: (read also plmodule.c) PlPlot is certainly not the last word on Plotting (somebody should do an OpenGL plotting module!), but as a starting point it might be useful. I like it because it has a Tk widget with it. John Interrante and myself wrote it as the same time but he will not support it anymore, that's why I am doing this job... :-) I put both distributions together, but this new version is a really a new thing. All List inputs are now replaced by matrix inputs (Not backward compatible !) Could have made boths inputs (matrix and python lists ) possible, but matrix input is much faster and is the first choice here (MatLab style), so why people should use the slower lists instead? Further I added __doc__ strings everywhere (this where the previous C-comments available now with python! :-)) The Input is not realy tested. At the moment it assumes that you are doing the right thing. Comments are welcome here... I hope I well get feedback from people using that module (which was done quite fast, but I don't want let you wait! :-)) Send me as much scripts using that stuff (perhaps some Matlab alike procedures) I some people want to do the missing things, you are welcome. Mai place will be ftp://ftp.appl-math.tu-muenchen.de/pub/et/plmodule.tgz Enjoy it... Tom HOW TO BUILD: 1) PLplot is available by anonymous ftp from dino.ph.utexas.edu in the plplot/ directory. 2) Install plplot (configure it with double precision and tk support) You will get a libplplotdtk.a (or build a shared library) 3) Rebuid python with Tk and add extern int Pltk_Init(Tcl_Interp*); if (Pltk_Init(interp) == TCL_ERROR) return TCL_ERROR; in tkappinit.c and add -lplplotdtk (+ perhaps appropriate path) in Setup at the right place. 3a) Build the plmodule as shared library or as built in module. 4) Add the following to your TKinter.py class PLWin(Widget): def __init__(self, master=None, cnf={}, **kw): Widget.__init__(self, master, 'plframe', cnf, kw) class PLXWin(Widget): def __init__(self, master=None, cnf={}, **kw): Widget.__init__(self, master, 'PLXWin', cnf, kw) #! /usr/local/bin/python from pl import * from Matrix import * from Tkinter import * class Test: def plot0(self): pladv(0) plcol(1) plvpor(0.0, 1.0, 0.0, 1.0) plwind(0.0, 1.0, 0.0, 1.0) plbox("bc", 0.0, 0, "bc", 0.0, 0) plsvpa(50.0, 150.0, 100.0, 150.0) plwind(0.0, 1.0, 0.0, 1.0) plbox("bc", 0.0, 0, "bc", 0.0, 0) plptex(0.5, 0.5, 1.0, 0.0, 0.5, "BOX at (50,150,100,150)") pleop() def plot1(self): x = mrange(0.1, 6.1, 0.1) y = pow(x, 2) xs1 = zeros(6) ys1 = zeros(6) xmin, xmax = x[0], x[59] ymin, ymax = y[0], y[59] for i in range(6): xs1[i] = x[i * 10 + 3] ys1[i] = y[i * 10 + 3] plcol(1) plenv(xmin, xmax, ymin, ymax, 0, 0) plcol(2) pllab("(x)", "(y)", "#frPlot Example 1 : y=x#u2") plcol(9) plpoin(xs1, ys1, 10) plcol(4) plline(x, y) pleop() def plot2(self): plcol(1) plenv(0.0, 2*pi, -1.0, 1.0, 0, 1) plcol(2) pllab("(x)", "sin(x)", "#frPlot Example 2 : Sin Function") plcol(3) x = mrange(0, 2*pi, 2*pi/500) plline(x, sin(x)) pleop() def plot5(self): n = 2047 data = sin(mrange(0, 2*pi, 2*pi/n)) plcol(1) plhist(data, -1.1, 1.1, 44, 0) plcol(2) pllab("#frValue", "#frFrequency", "#frPlot Example 5 : Probability function of Oscillator") pleop() def plot8(self): m, n = 35, 46 x=mrange(-1, 1, 2.0/m) y=mrange(-1, 1, 2.0/n) r=sqrt(add.outer(x*x, y*y)) z=exp(-r*r)*cos(2*pi*r) pladv(0) plvpor(0.0, 1.0, 0.0, 0.9) plwind(-1.0, 1.0, -0.9, 1.1) plcol(1) plw3d(1.0, 1.0, 1.0, -1.0, 1.0, -1.0, 1.0, -1.0, 1.0, 60.0, 120) plbox3('bnstu', 'x axis', 0.0, 0, 'bnstu', 'y axis', 0.0, 0, 'bcdmnstuv', 'z axis', 0.0, 0) plcol(2) plplot3d(x, y, z, 3, 0) plcol(3) plmtex('t', 1.0, 0.5, 0.5, '#frPlot Example 8 : Alt=60, Az=120') pleop() def plot9(self): mypltr = lambda x, y: (2.0/34 * x -1.0, 2.0/34 * y -1.0) nx, ny = 35, 46 mark, space = 1500, 1500 clevel = Matrix(-1.0, -0.8, -0.6, -0.4, -0.2, 0.0, 0.2, 0.4, 0.6, 0.8, 1.0) x=mrange(-1, 1, 2.0/nx) y=mrange(-1, 1, 2.0/ny)-1.0 z=subtract.outer(x*x, y*y) w=multiply.outer(2*x, y) plenv(-1.0, 1.0, -1.0, 1.0, 0, 0) plcol(2) plcont(z, 1, nx, 1, ny, clevel, mypltr) plstyl(1, mark, space) plcol(3) plcont(w, 1, nx, 1, ny, clevel, mypltr) plstyl(0, mark, space) plcol(1) pllab("X Coordinate", "Y Coordinate", "Streamlines of flow") pleop() def makePlotMenu(self, mBar): Plot = Menubutton(mBar, text='Plot', underline='0') Plot.pack(side='left', padx='2m') Plot.menu = Menu(Plot) Plot.menu.add('command', label='Example0', command=self.plot0) Plot.menu.add('command', label='Example1', command=self.plot1) Plot.menu.add('command', label='Example2', command=self.plot2) #Plot.menu.add('command', label='Example3', command=self.plot3) Plot.menu.add('command', label='Example5', command=self.plot5) Plot.menu.add('command', label='Example8', command=self.plot8) Plot.menu.add('command', label='Example9', command=self.plot9) Plot.menu.add_separator() Plot.menu.add('command', label='Quit', underline='0', background='red', activebackground='green', command='exit') Plot['menu'] = Plot.menu return Plot def __init__(self, master, width=550, height=412, bg='black', relief='raised'): self.mBar = Frame(master, relief=relief, bd='2') self.mBar.pack(side='top', fill='x') self.Plot = self.makePlotMenu(self.mBar) self.glx = PLWin(master, width=`width`, height=`height`, bg=bg) self.glx.pack() self.QUIT = Button(master, text='QUIT', fg='red', command=master.quit, relief=relief) self.QUIT.pack(side='bottom', fill='both') if __name__ == '__main__': root=Tk() test = Test(root, 600, 600, 'black', 'raised') root.mainloop() ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Nov 20 13:19:53 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA26948 for matrix-sig-people; Mon, 20 Nov 1995 13:19:53 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id NAA26944 for ; Mon, 20 Nov 1995 13:19:48 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA25389; Mon, 20 Nov 95 10:18:42 PST Received: by kristen.llnl.gov (5.x/SMI-SVR4) id AA07428; Mon, 20 Nov 1995 10:18:38 -0800 Date: Mon, 20 Nov 1995 10:18:38 -0800 Message-Id: <9511201818.AA07428@kristen.llnl.gov> From: "P. Dubois" To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Uniform Random Number Generator extension class Sender: owner-matrix-sig@python.org Precedence: bulk On the matrix extension site I have posted URNG.tar. It has been tested with the 0.11 version of the matrix extension. This class supplies two modules: URNGmodule.c Uniform random number generator objects can be created with either a seed initialized from the clock or a seed supplied by the creating call. The random number generator objects contain methods to return either a single random number or a vector of random numbers (i.e., a Matrix of shape (n,)). The individual generators are completely independent and do not affect the stream produced by other generators. The generator is a Cray ranf() compatible generator. Ranf.py This is the interface intended for users. Here is the entire text: import URNG import Matrix # x=CreateGenerator(seed) creates an random number generator stream # seed < 0 ==> Use the default initial seed value. # seed = 0 ==> Set a "random" value for the seed from the system clock. # seed > 0 ==> Set seed directly (32 bits only). # x.ranf() samples from that stream. # x.sample(n) returns a vector from that stream. # # ranf() returns a stream of random numbers # random_sample(n) returns a vector of length n filled with random numbers # random_matrix(n,m,...) returns an arbitrarily shaped matrix filled with # random numbers. CreateGenerator = URNG.CreateGenerator standard_generator = CreateGenerator(-1) def ranf(): "Returns a single random number from the standard generator." return standard_generator.ranf() def random_sample(n): """Returns a vector of length n of random numbers from the standard generator.""" return standard_generator.sample(n) def random_matrix(*n): """Returns a matrix with shape given by n of random numbers from the standard generator.""" if not n: raise 'random_matrix cannot be an empty shape' m = 1 for i in n: m = m * i x = Matrix.reshape(random_sample(m),n) return x ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Nov 20 22:10:34 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id WAA00885 for matrix-sig-people; Mon, 20 Nov 1995 22:10:34 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id WAA00881 for ; Mon, 20 Nov 1995 22:10:30 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA03343; Mon, 20 Nov 95 22:09:49 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) id WAA07337; Mon, 20 Nov 1995 22:14:39 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199511210314.WAA07337@maigret> Subject: Re: [PYTHON MATRIX-SIG] Matrix Plot module (small announcement) To: et@appl-math.tu-muenchen.de (Thomas Schwaller) Date: Mon, 20 Nov 1995 22:14:39 -0500 (EST) Cc: matrix-sig@python.org In-Reply-To: <9511201607.ZM10265@piranha.appl-math.tu-muenchen.de> from "Thomas Schwaller" at Nov 20, 95 04:07:08 pm X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 255 Sender: owner-matrix-sig@python.org Precedence: bulk >3a) Build the plmodule as shared library or as built in module. NOTE: It is important to add the -DPL_DOUBLE flag to the plmodule line in the Setup file. Otherwise you'll end up w/ a module which works very very strangely, as I found out. =) --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Nov 27 12:31:43 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA01081 for matrix-sig-people; Mon, 27 Nov 1995 12:31:43 -0500 Received: from muecke.appl-math.tu-muenchen.de (muecke.appl-math.tu-muenchen.de [129.187.242.1]) by python.org (8.6.12/8.6.12) with SMTP id MAA01071 for ; Mon, 27 Nov 1995 12:29:59 -0500 Received: from piranha.appl-math.tu-muenchen.de by muecke.appl-math.tu-muenchen.de via SMTP (931110.SGI/911001.SGI) for matrix-sig@python.org id AA29136; Mon, 27 Nov 95 18:30:36 +0100 Received: by piranha.appl-math.tu-muenchen.de (940816.SGI.8.6.9/911001.SGI) for matrix-sig@python.org id SAA22005; Mon, 27 Nov 1995 18:30:35 +0100 From: "Thomas Schwaller" Message-Id: <9511271830.ZM22003@piranha.appl-math.tu-muenchen.de> Date: Mon, 27 Nov 1995 18:30:28 +0100 X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] plotmodule (Matrix version 0.2) Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-matrix-sig@python.org Precedence: bulk Hi everybody, I'm just putting plplot0.2.tgz at the same place where the matrixmodule is located. Inputs are now transformed to matrices if this is possible. I really didnt test that feature up to know, because I'm just using matrices for the moment. There's also a GIF of pltest.py running on my Linux box. For that demo you need the Tix widget set. If you don't have it just use the older demo I posted to the SIG, I just love this Notebook widgets! :-) Still hoping somebody will send me some other stuff written with the matrix- and pl-modules. plplot0.2.tgz still isn't transferred! If you have a faster connection to me, you can also get it at ftp://ftp.appl-math.tu-muenchen.de/pub/et/plplot0.2.tgz Tom ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Dec 1 18:40:37 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA15178 for matrix-sig-people; Fri, 1 Dec 1995 18:40:37 -0500 Received: from mama-bear (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id SAA15174 for ; Fri, 1 Dec 1995 18:40:30 -0500 Received: from ling-ling.lcs.mit.edu.LCS.MIT.EDU by mama-bear (4.1/SLS-4.1.1) id AA19259; Fri, 1 Dec 95 18:39:58 EST Received: by ling-ling.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA04760; Fri, 1 Dec 95 18:39:42 -0500 Message-Id: <9512012339.AA04760@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Fri, 1 Dec 95 18:39:41 -0500 To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available Sender: owner-matrix-sig@python.org Precedence: bulk I can't completely believe I finally finished this latest release. I'm heading off for the python workshop on Monday, and I wanted to get this out before I left. The whole package can be found at the same site as before, (if you don't know the site you need to send me mail saying that you agree to test and review the object and I'll give you the site). The NEWS for this release follows. The two major incompatible changes at the very beginning are why this release took so long (that and the fact that I have a few other demands on my time). Please let me know what you think of these changes! I finally feel good about the feature set and performance of the object. If nobody has any major complaints (or REALLY good suggestions like Yorick pseudo and rubber indices, curse them) then I don't expect the basic structure of the object to change much in the future. As before, don't hesitate to send me your gripes, your bugs, and your praise (if ever merited). However, I'll be gone for most of next week at the python workshop, so I might be slow in replying. -Jim ********** Release 0.20 of the matrix object Major incompatible changes (I expect that these might elict some interesting discussion): 1) Matrix indexing now returns by reference to the original data, not by value. As a side effect of this change, arbitrary sequences are not allowed in multidimensional indices, but only single indices, slices, RubberIndex and None. Note: there is a new method, take, which will allow you to index a matrix with an arbitrary sequence as before. This change was motivated both by the possible speed increases (about 40% for some code) and more importantly by issues of clarity of expression. I couldn't come up with any other way to make the following hold: m[0][1] is an efficient way to index a multi-dimensional array. m[0,1] == m[0][1] m[0,] == m[0] 2) No more ranks of operators, instead Yorick style pseudo indices are used. As a side effect of this, outer product is now a convenience function rather than a method on ofuncs. For functions of unbounded rank (currently the only kind of function supported by my omathmodule) Yorick pseudo indices support a superset of what was possible to express using ranks and outer products. There is no fundamental reason not to have both pseudo indices and ranks, I just think that it's conceptually cleaner not to mix the two up. Here's a fun thing you can do with pseudo indices that is a lot uglier with ranks and outer products: x=mrange(-1, 1, 2.0/m)[All,None] y=mrange(-1, 1, 2.0/n) r=sqrt(x**2+y**2) z=exp(-r**2)*cos(2*pi*r) This is modified from a piece of Tom Schwaller's code for producing a 2d surface plot. Notice that this is much more efficient than generating x and y as full 2d matrices. Major improvements: 1) This new release is about 40% faster than 0.11 (on a benchmark code I was given by the folks at LBL). 2) No longer need to say a[[1,2,3]] for multidimensional indexing, now a[1,2,3] works just fine (thanks Konrad!) 3) Type coercion is performed more or less according to the same rules that python uses. So I can add a matrix of ints to one of floats, and I wind up with a matrix of floats at the end. 4) Finally there's a matrixMultiply method for matrices that behaves reasonably for all combinations of matrix, vector, and scalar arguments. Minor improvements: 1) pickling of matrices works (and uses a binary format with endianness labeled for both efficiency and portability). 2) The omath module supports trig functions as object methods (if m has method sin, then I can take sin(m) and get the sine of m. This also works on matrices of objects. omathmodule can be used now a a replacement for cmathmodule and mathmodule. 3) transpose now does the appropriate arbitrary permutation of dimensions. 4) More matrix methods, and better implementations of existing ones. Notes 1) The code hasn't actually grown to more than double it's previous size. This is just the effect of including all of Konrad's patches to the python Grammar as complete files rather than diffs. Hopefully, this will be able to be cleanly seperated from the matrix object, but I just don't have the time to deal with that right now. Things I plan to fix ASAP 1) Integrate Chris Chase's grammar patches to replace Slice(None,None,2) with ::2. 2) Make matrixMultiply work on matrices larger than 2d (this requires a notion of rank). 3) Fix the memory leaks 4) Make some real documentation (I already have doc strings for all the matrix methods, I'm hoping to steal some code to convert this to "real" documentation at the workshop). ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Dec 3 23:21:48 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id XAA22015 for matrix-sig-people; Sun, 3 Dec 1995 23:21:48 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id XAA22011 for ; Sun, 3 Dec 1995 23:21:42 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id XAA16932 (8.6.11/IDA-1.6); Sun, 3 Dec 1995 23:20:10 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id XAA18911; Sun, 3 Dec 1995 23:20:08 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id LAA21925; Sun, 3 Dec 1995 11:59:56 -0500 Date: Sun, 3 Dec 1995 11:59:56 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199512031659.LAA21925@cyclone.ERE.UMontreal.CA> To: jjh@mama-bear.lcs.mit.edu CC: matrix-sig@python.org In-reply-to: <9512012339.AA04760@ling-ling.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Fri, 1 Dec 95 18:39:41 -0500) Subject: Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available Sender: owner-matrix-sig@python.org Precedence: bulk Major incompatible changes (I expect that these might elict some interesting discussion): OK, let's start ;-) 1) Matrix indexing now returns by reference to the original data, not by value. As a side effect of this change, arbitrary sequences are not allowed in multidimensional indices, but only single indices, slices, RubberIndex and None. I don't see any problem with this. Of course the documentation (to be written...) should be clear so that everyone knows the implications of this, but since Python variables in general hold references, it shouldn't be a surprise for anyone. This change was motivated both by the possible speed increases (about 40% for some code) and more importantly by issues of clarity of expression. I couldn't come up with any other way to make the following hold: m[0][1] is an efficient way to index a multi-dimensional array. m[0,1] == m[0][1] m[0,] == m[0] I don't see why these are important features, but of course they don't do any harm. 2) No more ranks of operators, instead Yorick style pseudo indices are used. As a side effect of this, outer product is now a convenience function rather than a method on ofuncs. As another side effect, we have lost some important functionality. For functions of unbounded rank (currently the only kind of function supported by my omathmodule) Yorick pseudo indices support a superset of what was possible to express using ranks and outer products. There is no fundamental reason not to have both pseudo indices and ranks, I just think that it's conceptually cleaner not to mix the two up. The pseudo indices are not a replacement for ranks, but just a convenient notation for some structural functions. I like this notation very much, but we still need ranks. And as you said, there is no reason not to have both. Apart from the problem of functions with finite rank (which we will certainly need e.g. for matrix inversion), it is not true that pseudo indiced support a superset of what is possible with ranks and the associated structural functions. In fact, it is just the opposite, as I will demonstrate. First let me explain how the new pseudo indices fit into the J-style array system. J doesn't have the concept of "indexing"; instead there are a few structural functions that can be applied with different ranks to construct all kinds of rearrangements of array elements. All the (pseudo) indices we currently have can be mapped onto these structural functions as follows: - Explicit numerical indices, lists, slices and the rubber index are equivalent to the function take() (see my Array.py) with various rank. - The pseudo-index "None" is equivalent to the J-function "itemize" that adds an axis of length one to its argument. I didn't implement this in Array.py, but it is a trivial addition. - The contracting rubber index (* in Yorick, I haven't found it yet in Python)is equivalent to the function ravel() with appropriate rank. As an example, the outer product mentioned above could be (inefficiently) implemented in my Array.py as: def outer(f, a, b): return f(reshape[0](a, shape(b)), b) So every construction with (pseudo) indices can be emulated with three structural functions of suitable rank. On the other hand, there are constructions that cannot be expressed with pseudo indices, for example all cases where a function rank is not a constant but a variable. In fact these can be handled in Python -- as opposed to Yorick -- by remembering that a[x,y,z] is really just a shorthand for a[(x,y,z)] and that the index tuple can be constructed from an expression, but that seems more like an implementation accident than a feature to me. More importantly, pseudo indices don't help in the least when it comes to reductions and general inner products that require the specification of an axis. Yorick does this by putting the function into the index list -- a clever idea, but I'd prefer to keep a clear distinction between functions and arguments. The current matrix module allows a second parameter to reduce() that indicates the axis, which is exactly how APL handles the problem. But both these methods are restrictive. They are sufficient only for reductions with functions of unbounded rank. I can give an example of an actual application where this is not enough: I had a sequence of n rotation matrices, stored in a rank-3 array D of shape (n, 3, 3). Then the net rotation of this sequence could be obtained by matrixMultiply.reduce(D). Since matrix multiplication is a rank-2 function, I couldn't express this with the current matrix module. The bottom line: pseudo indices are convenient shorthands for the most common structural functions, but they are no replacement for the general function rank scheme we had in version 0.11. And now for something completely different ------------------------------------------ 1) Names This may seem to be a minor issue, but we should tackle this before we all get used to the current ones. I strongly dislike the type-specific constructors (also used for output). They should be IntegerMatrix, FloatMatrix, ComplexMatrix, CharacterMatrix, and GeneralMatrix. Anyone is free to define shorter names for efficient typing, if desired. I even propose a more radical renaming. Many people associate "matrix" with the 2D-matrices from linear algebra. So it would be better to call our general objects "arrays", and leave the name "matrix" for linear-algebra type objects that are restricted to rank 2 and use * for matrix multiplication. They could be implemented in Python based on arrays. The pseudo-index 'None' is very confusing. An index "2" picks item number two from an axis, an index "All" picks all items from an axis. Consequently "None" should pick no item from an axis, which is course is a pointless operation. What it really does is create a new axis, so it should be named "New". "RubberIndex" is not confusing, but a bit strange, so it's worth thinking about alternatives. How about "Skip", in the sense "skip all following axes for explicit consideration"? For the "*" in Yorick I propose "Contract". 2) Constructors Currently there are two constructors, matrix() and Matrix(), with slightly different behaviour: matrix() takes a single argument that is a (possibly nested) Python sequence object, e.g. a list. Matrix() takes an arbitrary number of arguments that are made into a list before being passed to matrix(). So Matrix(1,2,3) is equivalent to matrix([1,2,3]). I find it confusing to have to constructors with almost the same name and almost the same function. I propose to have only one, of whatever name, which behaves like matrix(). The reason is that many matrix functions accept nested lists or tuples instead of matrix arguments, e.g. matrix([2,3]) + [5,4] works. So does matrix([2,3]) + matrix([5,4]), which is equivalent. But Matrix([2,3]) + [5,4] and Matrix([2,3]) + Matrix([5,4]) are not equivalent (well, they are in this simple exaple due to broadcasting, but not in general). So matrix() can be considered a conversion function from lists and tuples to arrays, which is needed anyway. I realize that the calling style of Matrix() has the advantage of saving one pair of brackets, but I don't consider this important enough to create the current confusion. I guess that's enough for today ;-) I wish the lucky participants of the workshop much fun and hope that they don't have to eat spam for lunch ;-) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Dec 4 14:26:33 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA24719 for matrix-sig-people; Mon, 4 Dec 1995 14:26:33 -0500 Received: from rnet.rose.rsoc.rockwell.com (rose.rsoc.rockwell.com [161.40.39.100]) by python.org (8.6.12/8.6.12) with SMTP id OAA24715 for ; Mon, 4 Dec 1995 14:26:28 -0500 Received: by rnet.rose.rsoc.rockwell.com (4.1/SMI-4.1) id AA00366; Mon, 4 Dec 95 13:25:18 CST Received: from sunrise(161.40.50.11) by rnet via smap (V1.3) id sma000364; Mon Dec 4 13:25:13 1995 Received: from feynman.rsoc.rockwell.com by rose.rsoc.rockwell.com (5.0/SMI-SVR4) id AA00997; Mon, 4 Dec 1995 13:25:10 -0600 Received: by feynman.rsoc.rockwell.com (5.0/SMI-SVR4) id AA00688; Mon, 4 Dec 1995 13:25:11 -0600 Date: Mon, 4 Dec 1995 13:25:11 -0600 From: forrest@rose.rsoc.rockwell.com (Dave Forrest) Message-Id: <9512041925.AA00688@feynman.rsoc.rockwell.com> To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Renaming and Constructors X-Sun-Charset: US-ASCII Content-Length: 1856 Sender: owner-matrix-sig@python.org Precedence: bulk > From owner-matrix-sig@python.org Mon Dec 4 12:06 CST 1995 > Date: Sun, 3 Dec 1995 11:59:56 -0500 > From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad) [snip] > > I strongly dislike the type-specific constructors (also used for > output). They should be IntegerMatrix, FloatMatrix, ComplexMatrix, > CharacterMatrix, and GeneralMatrix. Anyone is free to define shorter > names for efficient typing, if desired. > > I even propose a more radical renaming. Many people associate "matrix" > with the 2D-matrices from linear algebra. So it would be better to Yes!, Yes we do! (Some of us even thought that that was the reason for having a Matrix SIG) > call our general objects "arrays", and leave the name "matrix" for > linear-algebra type objects that are restricted to rank 2 and use * > for matrix multiplication. They could be implemented in Python based > on arrays. I thought that we had decided a while back to call it "Tensor" because "Array" was a more general, computer-science concept. [snip] > 2) Constructors > [snip] > I find it confusing to have to constructors with almost the same name > and almost the same function. I propose to have only one, of whatever > name, which behaves like matrix(). The reason is that many matrix > functions accept nested lists or tuples instead of matrix arguments, > e.g. matrix([2,3]) + [5,4] works. So does matrix([2,3]) + > matrix([5,4]), which is equivalent. But Matrix([2,3]) + [5,4] and > Matrix([2,3]) + Matrix([5,4]) are not equivalent (well, they are in > this simple exaple due to broadcasting, but not in general). So > matrix() can be considered a conversion function from lists and tuples > to arrays, which is needed anyway. I agree. David Forrest ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Dec 4 17:14:39 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA25498 for matrix-sig-people; Mon, 4 Dec 1995 17:14:39 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id RAA25493 for ; Mon, 4 Dec 1995 17:14:25 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id RAA13512 (8.6.11/IDA-1.6); Mon, 4 Dec 1995 17:12:33 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA22393; Mon, 4 Dec 1995 17:12:32 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA14614; Mon, 4 Dec 1995 17:12:32 -0500 Date: Mon, 4 Dec 1995 17:12:32 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199512042212.RAA14614@cyclone.ERE.UMontreal.CA> To: forrest@rose.rsoc.rockwell.com CC: matrix-sig@python.org In-reply-to: <9512041925.AA00688@feynman.rsoc.rockwell.com> (forrest@rose.rsoc.rockwell.com) Subject: Re: [PYTHON MATRIX-SIG] Renaming and Constructors Sender: owner-matrix-sig@python.org Precedence: bulk I thought that we had decided a while back to call it "Tensor" because "Array" was a more general, computer-science concept. But what we have now corresponds to that concept. In contrary, a tensor is something completely different: a mathematical object with certain transformation behaviour under coordinate system transforms. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Dec 5 03:30:42 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id DAA27194 for matrix-sig-people; Tue, 5 Dec 1995 03:30:42 -0500 Received: from atr-sw.atr-sw.atr.co.jp (atr-sw.atr-sw.atr.co.jp [133.186.20.117]) by python.org (8.6.12/8.6.12) with ESMTP id DAA27188 for ; Tue, 5 Dec 1995 03:28:22 -0500 Received: from ciris21.atr-sw.atr.co.jp (ciris21.atr-sw.atr.co.jp [133.186.20.41]) by atr-sw.atr-sw.atr.co.jp (8.6.11+2.5Wb2/3.4Wbeta3/95031616ATR-SW) with ESMTP id RAA00535 for ; Tue, 5 Dec 1995 17:23:22 +0900 Received: from atr-sw by ciris21.atr-sw.atr.co.jp (940816.SGI.8.6.9/6.4J.6) id RAA27076; Tue, 5 Dec 1995 17:23:21 +0900 Message-Id: <199512050823.RAA27076@ciris21.atr-sw.atr.co.jp> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] A bug fix for release 0.20 Date: Tue, 05 Dec 1995 17:23:21 +0900 From: "Perry A. Stoll" Sender: owner-matrix-sig@python.org Precedence: bulk While Jim Hugunin is gone at the Python conference, I thought I'd share a bug fix with the list. Finding this bug was accidental (trying to cast a matrix of ints to types.ListType), but unavoidable (it core dumped). Here is a brief example which demonstrates this: import Matrix, types Matrix.mrange(1,5).cast(types.ListType) The fix is simple once you've found it ;-) It is only one line in the file Objects/matrixtypes.c. Here are the diffs: 564c566 < sizeof(PyObject), 'O', PyMatrix_OBJECT}; --- > sizeof(PyObject *), 'O', PyMatrix_OBJECT}; i.e. the size of entries in an ObjectMatrix should be the size of a pointer to a PyObject struct, not the size of a PyObject struct. And now for more poking around in the guts of the matrix lib... -Perry Stoll Research Associate, CSRL Advanced Telecommunications Research 2-2 Hikaridai, Seika-cho Soraku-gun, Kyoto 619-02 JAPAN VOICE: +81-774-95-1217 FINGER: stoll@atrwide.atr.co.jp FAX : +81-774 95-1208 EMAIL: stoll@atr-sw.atr.co.jp PGP 2.6 Key fingerprint = AF 56 5C D8 5F 78 BA FD 21 6E 2A 68 C4 55 9E B0 ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Dec 5 08:33:46 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id IAA28034 for matrix-sig-people; Tue, 5 Dec 1995 08:33:46 -0500 Received: from eeel.nist.gov (sparky.eeel.nist.gov [129.6.64.163]) by python.org (8.6.12/8.6.12) with SMTP id IAA28030 for ; Tue, 5 Dec 1995 08:33:41 -0500 Received: from acdc.eeel.nist.gov by eeel.nist.gov (4.1/SMI-3.2-del.6) id AA26006; Tue, 5 Dec 95 08:32:37 EST Received: by acdc.eeel.nist.gov (4.1/SMI-3.2-del.5) id AA03465; Tue, 5 Dec 95 08:28:54 EST Date: Tue, 5 Dec 95 08:28:54 EST Message-Id: <9512051328.AA03465@acdc.eeel.nist.gov> From: Michael McLay To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Re: Python vs Perl: benchmarking for speed? In-Reply-To: References: <496if1$8hg@mrnews.mro.dec.com> <49m6j5$m7v@mrnews.mro.dec.com> <49of3l$86g@csnews.cs.colorado.edu> Sender: owner-matrix-sig@python.org Precedence: bulk Hinsen Konrad (hinsenk@cyclone.ERE.UMontreal.CAwrites: > It seems to me that the argument you replied to was a purely > hypothetical one, to illustrate the usefulness of tabulated > benchmarks. Noone suggests that Perl takes ten times more time > for a given problem than Python. In fact, I would be very much > surprised if the difference between Perl and Python would ever > reach a factor of ten in any direction for any problem. What an invitation. How about doing that benchmark comparison of Python matrix math vs. Perl matrix math that the world has been waiting for. I can't wait to see the results:-) Michael ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Dec 5 08:44:00 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id IAA28084 for matrix-sig-people; Tue, 5 Dec 1995 08:44:00 -0500 Received: from rnet.rose.rsoc.rockwell.com (rose.rsoc.rockwell.com [161.40.39.100]) by python.org (8.6.12/8.6.12) with SMTP id IAA28079 for ; Tue, 5 Dec 1995 08:43:56 -0500 Received: by rnet.rose.rsoc.rockwell.com (4.1/SMI-4.1) id AA03068; Tue, 5 Dec 95 07:42:37 CST Received: from sunrise(161.40.50.11) by rnet via smap (V1.3) id sma003065; Tue Dec 5 07:41:55 1995 Received: from feynman.rsoc.rockwell.com by rose.rsoc.rockwell.com (5.0/SMI-SVR4) id AA07778; Tue, 5 Dec 1995 07:41:54 -0600 Received: by feynman.rsoc.rockwell.com (5.0/SMI-SVR4) id AA03099; Tue, 5 Dec 1995 07:41:53 -0600 Date: Tue, 5 Dec 1995 07:41:53 -0600 From: forrest@rose.rsoc.rockwell.com (Dave Forrest) Message-Id: <9512051341.AA03099@feynman.rsoc.rockwell.com> To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Re: Python vs Perl: benchmarking for speed? X-Sun-Charset: US-ASCII Content-Length: 879 Sender: owner-matrix-sig@python.org Precedence: bulk > From owner-matrix-sig@python.org Tue Dec 5 07:35 CST 1995 [snip] > > Hinsen Konrad (hinsenk@cyclone.ERE.UMontreal.CAwrites: > > It seems to me that the argument you replied to was a purely > > hypothetical one, to illustrate the usefulness of tabulated > > benchmarks. Noone suggests that Perl takes ten times more time > > for a given problem than Python. In fact, I would be very much > > surprised if the difference between Perl and Python would ever > > reach a factor of ten in any direction for any problem. > > What an invitation. How about doing that benchmark comparison of > Python matrix math vs. Perl matrix math that the world has been > waiting for. I can't wait to see the results:-) IMHO, this "dialogue" is not relevant to the Matrix discussion. Please take it elsewhere. David Forrest ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Dec 7 12:47:19 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA10284 for matrix-sig-people; Thu, 7 Dec 1995 12:47:19 -0500 Received: from mama-bear (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id MAA10280 for ; Thu, 7 Dec 1995 12:47:13 -0500 Received: from ling-ling.lcs.mit.edu.LCS.MIT.EDU by mama-bear (4.1/SLS-4.1.1) id AA26673; Thu, 7 Dec 95 12:46:36 EST Received: by ling-ling.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA07309; Thu, 7 Dec 95 12:47:03 -0500 Message-Id: <9512071747.AA07309@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Thu, 7 Dec 95 12:47:03 -0500 To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Ranks and pseudo indices References: <199512031659.LAA21925@cyclone.ERE.UMontreal.CA> Sender: owner-matrix-sig@python.org Precedence: bulk Konrad writes: > 2) No more ranks of operators, instead Yorick style pseudo indices are > used. As a side effect of this, outer product is now a convenience function > rather than a method on ofuncs. > > As another side effect, we have lost some important functionality. > > For functions of unbounded rank (currently the only kind of function > supported by my omathmodule) Yorick pseudo indices support a superset of > what was possible to express using ranks and outer products. There is no > fundamental reason not to have both pseudo indices and ranks, I just think > that it's conceptually cleaner not to mix the two up. > > The pseudo indices are not a replacement for ranks, but just a convenient > notation for some structural functions. I like this notation very much, > but we still need ranks. And as you said, there is no reason not to have both. > I'll look at your specific comments, but I don't yet find them convincing. > So every construction with (pseudo) indices can be emulated with three > structural functions of suitable rank. On the other hand, there are > constructions that cannot be expressed with pseudo indices, for > example all cases where a function rank is not a constant but a > variable. In fact these can be handled in Python -- as opposed to > Yorick -- by remembering that a[x,y,z] is really just a shorthand for > a[(x,y,z)] and that the index tuple can be constructed from an > expression, but that seems more like an implementation accident than a > feature to me. Actually, I do think that this is a feature, but I'd be curious for a good example of a case of this. > More importantly, pseudo indices don't help in the least when it comes > to reductions and general inner products that require the specification > of an axis. Yorick does this by putting the function into the index > list -- a clever idea, but I'd prefer to keep a clear distinction > between functions and arguments. The current matrix module allows > a second parameter to reduce() that indicates the axis, which is > exactly how APL handles the problem. I don't think that you're actually complaining about my current solution to this problem are you? > But both these methods are restrictive. They are sufficient only > for reductions with functions of unbounded rank. I can give an > example of an actual application where this is not enough: I had > a sequence of n rotation matrices, stored in a rank-3 array D of > shape (n, 3, 3). Then the net rotation of this sequence could > be obtained by matrixMultiply.reduce(D). Since matrix > multiplication is a rank-2 function, I couldn't express this > with the current matrix module. This is unfortunate, but unless I hear of many more of these problems, I don't expect this to go away any time soon. It's REALLY hard to add in function of finite rank to the Optimized FUNCtion scheme I'm currently using. I spent a week and a half trying to make this happen and wound up with an ugly useless mess. Since I can imagine very few cases where this sort of behavior is important, I'm very tempted to leave ofuncs as they are right now, only operating on unbounded functions. > The bottom line: pseudo indices are convenient shorthands for > the most common structural functions, but they are no replacement > for the general function rank scheme we had in version 0.11. You haven't convinced me of this (except in the cases of functions with finite rank) and I have a large number of reasons to ignore them. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Dec 7 13:04:11 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA10362 for matrix-sig-people; Thu, 7 Dec 1995 13:04:11 -0500 Received: from mama-bear (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id NAA10357 for ; Thu, 7 Dec 1995 13:04:09 -0500 Received: from ling-ling.lcs.mit.edu.LCS.MIT.EDU by mama-bear (4.1/SLS-4.1.1) id AA26856; Thu, 7 Dec 95 13:03:36 EST Received: by ling-ling.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA07317; Thu, 7 Dec 95 13:04:04 -0500 Message-Id: <9512071804.AA07317@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Thu, 7 Dec 95 13:04:03 -0500 To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Renaming and Constructors References: <9512041925.AA00688@feynman.rsoc.rockwell.com> Sender: owner-matrix-sig@python.org Precedence: bulk > 1) Names > > This may seem to be a minor issue, but we should tackle this before > we all get used to the current ones. I agree completely! > I strongly dislike the type-specific constructors (also used for > output). They should be IntegerMatrix, FloatMatrix, ComplexMatrix, > CharacterMatrix, and GeneralMatrix. Anyone is free to define shorter > names for efficient typing, if desired. Does a FloatMatrix contain C floats or doubles (both are possible in matrices)? Please define a complete naming scheme that can be compared to the current (admittedly cryptic) typecodes. Note: The current output is only being used because nobody has yet given me a good replacement for the PrintMatrix function. I deliberately pulled this function out into python code so that other people could do it right for me. > I even propose a more radical renaming. Many people associate "matrix" > with the 2D-matrices from linear algebra. So it would be better to > call our general objects "arrays", and leave the name "matrix" for > linear-algebra type objects that are restricted to rank 2 and use * > for matrix multiplication. They could be implemented in Python based > on arrays. The only problem that I have with array is that python already has arrays with a known semantics. The fundamental rule in python is to never break existing code. I know this is annoying, but the only other option would be to make the current matrices completely backward compatible with the existing arrays. (Which is possibly possible...) Any other naming suggestions? (I agree that tensor is as bad as matrix) > The pseudo-index 'None' is very confusing. An index "2" picks item > number two from an axis, an index "All" picks all items from an > axis. Consequently "None" should pick no item from an axis, which is > course is a pointless operation. What it really does is create a new > axis, so it should be named "New". "RubberIndex" is not confusing, > but a bit strange, so it's worth thinking about alternatives. How > about "Skip", in the sense "skip all following axes for explicit > consideration"? For the "*" in Yorick I propose "Contract". I have no good opinions on this issue. The obvious reason that I chose None is that it is a built-in python object, so it is always in the current namespace. This namespace issue is an important one. I really don't think that we are going to be adding any of the three of these to the python core any time soon. One solution would be to use "new", "ellipses", and "contract" and to have Matrix.py map these strings to objects. Note: I picked RubberIndex in the hopes of choosing something sufficiently long and hard to type that nobody would assume I meant it to be the final solution. I had hoped that this might be turned into the syntax ".." one day, but after the python workshop I doubt that this will happen any time soon. > 2) Constructors > > Currently there are two constructors, matrix() and Matrix(), with > slightly different behaviour: matrix() takes a single argument that is > a (possibly nested) Python sequence object, e.g. a list. Matrix() > takes an arbitrary number of arguments that are made into a list > before being passed to matrix(). So Matrix(1,2,3) is equivalent > to matrix([1,2,3]). > > I find it confusing to have to constructors with almost the same name > and almost the same function. I propose to have only one, of whatever > name, which behaves like matrix(). The reason is that many matrix > functions accept nested lists or tuples instead of matrix arguments, > e.g. matrix([2,3]) + [5,4] works. So does matrix([2,3]) + > matrix([5,4]), which is equivalent. But Matrix([2,3]) + [5,4] and > Matrix([2,3]) + Matrix([5,4]) are not equivalent (well, they are in > this simple exaple due to broadcasting, but not in general). So > matrix() can be considered a conversion function from lists and tuples > to arrays, which is needed anyway. > > I realize that the calling style of Matrix() has the advantage of > saving one pair of brackets, but I don't consider this important > enough to create the current confusion. This is a real problem. I'll repeat my reasons for creating Matrix, but otherwise, I'm willing to go along with the consensus of the list on this one. I was looking for the cleanest and easiest to read notation for a matrix in the unfortunate world where I couldn't add a new bracket type to python (okay, there aren't even any brackets to add so what am I complaining about). I noticed that by using the variable number of arguments notation, I got something that looked a lot more like just a new kind of brackets. Let me know who else agrees with Konrad here. > I guess that's enough for today ;-) I wish the lucky participants > of the workshop much fun and hope that they don't have to eat > spam for lunch ;-) Had much fun, never ate SPAM - Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Dec 7 14:35:37 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA10678 for matrix-sig-people; Thu, 7 Dec 1995 14:35:37 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id OAA10674 for ; Thu, 7 Dec 1995 14:35:27 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id OAA28669 (8.6.11/IDA-1.6); Thu, 7 Dec 1995 14:34:13 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA19484; Thu, 7 Dec 1995 14:34:12 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA10658; Thu, 7 Dec 1995 14:34:11 -0500 Date: Thu, 7 Dec 1995 14:34:11 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199512071934.OAA10658@cyclone.ERE.UMontreal.CA> To: jjh@mama-bear.lcs.mit.edu CC: matrix-sig@python.org In-reply-to: <9512071747.AA07309@ling-ling.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Thu, 7 Dec 95 12:47:03 -0500) Subject: Re: [PYTHON MATRIX-SIG] Ranks and pseudo indices Sender: owner-matrix-sig@python.org Precedence: bulk > variable. In fact these can be handled in Python -- as opposed to > Yorick -- by remembering that a[x,y,z] is really just a shorthand for > a[(x,y,z)] and that the index tuple can be constructed from an > expression, but that seems more like an implementation accident than a > feature to me. Actually, I do think that this is a feature, but I'd be curious for a good example of a case of this. OK, let's make it a feature. I can't remember an example right now, but I know that I used variables to specify ranks in J a few times. Besides, my philosophy is the opposite: there should be good arguments for restrictions, not for generality. Sooner or later someone will reach the limits of everything, so you should not impose narrow limits arbitrarily just because you can't think of any problem affected by them. > between functions and arguments. The current matrix module allows > a second parameter to reduce() that indicates the axis, which is > exactly how APL handles the problem. I don't think that you're actually complaining about my current solution to this problem are you? In the sense of being restrictive, yes. Many APL users have found this handling of reduction restrictive, which is why J's rank scheme is generally appreciated, even by those who are put off by J's other features. This is unfortunate, but unless I hear of many more of these problems, I don't expect this to go away any time soon. It's REALLY hard to add in function of finite rank to the Optimized FUNCtion scheme I'm currently using. I spent a week and a half trying to make this happen and wound up I confess that I have never looked at their implementation, but I don't see where the problem is. It is even possible to implement the mapping from finite-rank functions to unbounded-rank functions in Python relatively easily, but that means for-loops and the associated loss of efficiency. I'll see if I can figure out how the OFUNCs work. with an ugly useless mess. Since I can imagine very few cases where this sort of behavior is important, I'm very tempted to leave ofuncs as they are right now, only operating on unbounded functions. If you want more cases, try to locate an archive of comp.lang.apl of the time when J came out. There were long discussions about the merits of the rank system back then. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Dec 7 15:15:46 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA10848 for matrix-sig-people; Thu, 7 Dec 1995 15:15:46 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id PAA10844 for ; Thu, 7 Dec 1995 15:15:39 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id PAA00861 (8.6.11/IDA-1.6); Thu, 7 Dec 1995 15:14:34 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA00327; Thu, 7 Dec 1995 15:14:32 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA13383; Thu, 7 Dec 1995 15:14:31 -0500 Date: Thu, 7 Dec 1995 15:14:31 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199512072014.PAA13383@cyclone.ERE.UMontreal.CA> To: jjh@mama-bear.lcs.mit.edu CC: matrix-sig@python.org In-reply-to: <9512071804.AA07317@ling-ling.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Thu, 7 Dec 95 13:04:03 -0500) Subject: Re: [PYTHON MATRIX-SIG] Renaming and Constructors Sender: owner-matrix-sig@python.org Precedence: bulk Does a FloatMatrix contain C floats or doubles (both are possible in matrices)? Please define a complete naming scheme that can be compared to Really? I didn't know that. Anyway, "float" should mean what it means in normal Python, i.e. C double. C floats could be called "ShortFloat", for example. As for the other typecodes, I am not sure what they all mean (is there a list somewhere?). But let's start: 'c' Character 'u' Unsigned (is that really useful?) '1' ??? (maybe Boolean?) 's' String 'i' Integer 'l' LongInteger 'f' ShortFloat 'd' Float 'F' ??? (maybe ShortComplex?) 'D' Complex 'O' General The only problem that I have with array is that python already has arrays with a known semantics. The fundamental rule in python is to never break existing code. I know this is annoying, but the only other option would be to make the current matrices completely backward compatible with the existing arrays. (Which is possibly possible...) So that means we can not call any module "array", but we can still use the name "array" within the modules. A suggestions: - "arrays_and_matrices" for the C module that nobody imports directly anyway. - "Array" for the Python wrapper with array semantics; "Array" in the names of the constructors. - "Matrix" for the Python wrapper with 2D-matrix semantics; "Matrix" in the names of the constructors. That leaves the possibility of confusion between "array" and "Array", but I can live with that. I doubt the current Python arrays will be attractive to anyone who has the new module. namespace. This namespace issue is an important one. I really don't think that we are going to be adding any of the three of these to the python core any time soon. They don't have to be. They just have to be in the namespace of the modules "Array" and "Matrix". > I guess that's enough for today ;-) I wish the lucky participants > of the workshop much fun and hope that they don't have to eat > spam for lunch ;-) Had much fun, never ate SPAM - Jim That's cheating ;-) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Dec 8 15:50:12 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA00759 for matrix-sig-people; Fri, 8 Dec 1995 15:50:12 -0500 Received: from alfven.jhuapl.edu (alfven.jhuapl.edu [128.244.146.147]) by python.org (8.6.12/8.6.12) with SMTP id PAA00755 for ; Fri, 8 Dec 1995 15:50:07 -0500 Message-Id: <199512082050.PAA00755@python.org> Received: by alfven.jhuapl.edu (1.38.193.4/16.2) id AA08007; Fri, 8 Dec 1995 15:59:02 -0500 Date: Fri, 8 Dec 1995 15:59:02 -0500 From: Chris Chase S1A To: matrix-sig@python.org Cc: James Hugunin Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available In-Reply-To: <9512012339.AA04760@ling-ling.lcs.mit.edu.LCS.MIT.EDU> References: <9512012339.AA04760@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk James> The NEWS for this release follows. The two major incompatible James> changes at the very beginning are why this release took so long James> (that and the fact that I have a few other demands on my time). James> Please let me know what you think of these changes! James> As before, don't hesitate to send me your gripes, your bugs, James> and your praise (if ever merited). However, I'll be gone for James> most of next week at the python workshop, so I might be slow in James> replying. James> -Jim James> Major incompatible changes (I expect that these might elict some James> interesting discussion): James> 1) Matrix indexing now returns by reference to the original James> data, not by value. As a side effect of this change, James> arbitrary sequences are not allowed in multidimensional James> indices, but only single indices, slices, RubberIndex and James> None. I personally prefer general product indexing that allow multidimensional indexes. These could be added to the reference version of indexing but they would require extra baggage by retaining a copy of the index vector. This would not be very efficient but is not a reason to eliminate it since it would not affect the efficiency of the other types of indexes. Are references better than copying for indexing? I can see that a speed increase is natural when using only references rather than creating a copy. But are references always efficient? If a multidimensional slice reference is passed as an argument to another routine that accesses the object multiple times then it may end up being less efficient. To avoid this would require that these routines make contiguous copies of these types of arguments (whether or not the copy is needed since the reference nature of the object is transparent to the user). Similarly a copy to contiguous storage would need to be done before passing a reference to an external FORTRAN or C routine. On the other hand, I can see that references can save memory for very large matrix objects. Additionally, a copy can be performed at user discretion. I would think that assignment to references that change the originally indexed object may lead to some surprises for some users used to the implementations of other matrix languages like Matlab and IDL. I am not an expert on this. What are the other pros and cons about references versus copies being returned by indexing? James> Note: there is a new method, take, which will allow you to James> index a matrix with an arbitrary sequence as before. This then requires a sequence of take() plus [] indexing to obtain general product indexing. How does one do assignment using a multi-dimensional index vector? James> This change was motivated both by the possible speed increases James> (about 40% for some code) and more importantly by issues of James> clarity of expression. I couldn't come up with any other way James> to make the following hold: James> m[0][1] is an efficient way to index a multi-dimensional array. James> m[0,1] == m[0][1] James> m[0,] == m[0] It will be difficult to maintain a natural connection between hierarchical index using [][] versus multi-dimensional indexing. For example: m[0:4,1] != m[0:4][1] James> 2) No more ranks of operators, instead Yorick style pseudo James> indices are used. As a side effect of this, outer product James> is now a convenience function rather than a method on James> ofuncs. James> For functions of unbounded rank (currently the only kind of James> function supported by my omathmodule) Yorick pseudo indices James> support a superset of what was possible to express using ranks James> and outer products. There is no fundamental reason not to have James> both pseudo indices and ranks, I just think that it's James> conceptually cleaner not to mix the two up. Function ranks offer more generality than what can be done with pseudo indices. I would prefer that the matrix module keep function ranks rather than limit the full generality of the object. Although I do not what it takes to add it cleanly. I have not had time to look deeply into your code other than browsing the matrixobject.c (the current state of the code made for rather dense reading). James> 1) This new release is about 40% faster than 0.11 (on a James> benchmark code I was given by the folks at LBL). What was the benchmark? >> 1) Names >> >> This may seem to be a minor issue, but we should tackle this before >> we all get used to the current ones. James> I agree completely! >> I strongly dislike the type-specific constructors (also used for >> output). They should be IntegerMatrix, FloatMatrix, ComplexMatrix, >> CharacterMatrix, and GeneralMatrix. Anyone is free to define shorter >> names for efficient typing, if desired. James> Does a FloatMatrix contain C floats or doubles (both are James> possible in matrices)? Please define a complete naming scheme James> that can be compared to the current (admittedly cryptic) James> typecodes. Perhaps a naming scheme similar to FORTRAN's INTEGER*8, REAL*4 REAL*8. The advantage of this is that the precision is known. With ANSI C the precision of floats and doubles is not specified. ANSI C only enforces a minimum precision on an implementation. >> I even propose a more radical renaming. Many people associate "matrix" >> with the 2D-matrices from linear algebra. So it would be better to >> call our general objects "arrays", and leave the name "matrix" for >> linear-algebra type objects that are restricted to rank 2 and use * >> for matrix multiplication. They could be implemented in Python based >> on arrays. I would prefer the term "array" with a specialized "matrix" module providing linear algebra matrix functions and a matrix multiplication operator that take 2D arrays as arguments. But it was pointed out that we can not use "array" as it is already a different module in Python. I don't suppose the module produced by this SIG could completely replace the current array module (without breadking its old behavior)? James> Note: I picked RubberIndex in the hopes of choosing something James> sufficiently long and hard to type that nobody would assume I James> meant it to be the final solution. I had hoped that this might James> be turned into the syntax ".." one day, but after the python James> workshop I doubt that this will happen any time soon. What happened at the workshop to rule out the ".." or "*" syntax for rubber indexes? Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Dec 8 18:17:02 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA01359 for matrix-sig-people; Fri, 8 Dec 1995 18:17:02 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id SAA01355 for ; Fri, 8 Dec 1995 18:16:52 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id SAA04860 (8.6.11/IDA-1.6); Fri, 8 Dec 1995 18:16:03 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id SAA07397; Fri, 8 Dec 1995 18:16:03 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id SAA05920; Fri, 8 Dec 1995 18:16:02 -0500 Date: Fri, 8 Dec 1995 18:16:02 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199512082316.SAA05920@cyclone.ERE.UMontreal.CA> To: chris.chase@jhuapl.edu CC: matrix-sig@python.org In-reply-to: <199512082050.PAA00755@python.org> (message from Chris Chase S1A on Fri, 8 Dec 1995 15:59:02 -0500) Subject: Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available Sender: owner-matrix-sig@python.org Precedence: bulk It will be difficult to maintain a natural connection between hierarchical index using [][] versus multi-dimensional indexing. For example: m[0:4,1] != m[0:4][1] I agree, and we shouldn't insist on this as a feature. It might even be better (in terms of error checking) to forbid an index that doesn't cover all dimensions. Right now, a[0] is just a shorthand for a[0, RubberIndex], and I don't think this shorthand is necessary. Perhaps a naming scheme similar to FORTRAN's INTEGER*8, REAL*4 REAL*8. The advantage of this is that the precision is known. With ANSI C the precision of floats and doubles is not specified. ANSI C only enforces a minimum precision on an implementation. And since Python is written in C, we can't promise anything more. Besides, giving a single number still doesn't specify the precision of a floating point number. The memory space could be distributed differently between mantissa and exponent. My major concern is that array names correspond with the standard Python number type names as far as possible. Anything else just generates confusion. operator that take 2D arrays as arguments. But it was pointed out that we can not use "array" as it is already a different module in As I pointed out before, it is only the module name which may not be "array". Even "Array" is OK. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Dec 8 19:06:06 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id TAA01539 for matrix-sig-people; Fri, 8 Dec 1995 19:06:06 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id TAA01535 for ; Fri, 8 Dec 1995 19:06:02 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA12115; Fri, 8 Dec 95 16:05:57 PST Received: from kristen (localhost) by kristen.llnl.gov (5.x/SMI-SVR4) id AA17281; Fri, 8 Dec 1995 16:05:55 -0800 Message-Id: <30C8D2E2.6190@llnl.gov> Date: Fri, 08 Dec 1995 16:05:54 -0800 From: "Paul. Dubois" Organization: Lawrence Livermore National Laboratory X-Mailer: Mozilla 2.0b2 (X11; I; SunOS 5.4 sun4m) Mime-Version: 1.0 To: Hinsen Konrad Cc: chris.chase@jhuapl.edu, matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available References: <199512082316.SAA05920@cyclone.ERE.UMontreal.CA> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk A followup to the remarks about precision. In fact, Fortran 90 goes to considerable lengths to try to improve this situation but it isn't REAL*8 that is the solution. Fortran 90 uses something called a Kind specifier, and Kind specifiers are calculable from desired characteristics at compile time. Actually knowing that something is a certain number of bits doesn't tell you what you need to know, as there is a tradeoff between precision and dynamic range. Anyway, you end up saying REAL(kind) x, y, z. I believe Jim has made fundamentally the right decision in deciding that the abstraction he is after is "things that store a homogenous bunch of stuff in contiguous memory". This would be a parameterized type in other languages, such as Eiffel's ARRAY[T], where one actually instantiates ARRAY[DOUBLE], ARRAY[INTEGER], ARRAY[CHARACTER], ARRAY[TELEPHONE_BOOK], etc. Since a large use of this abstraction is connection to C and Fortran libraries, one has to be precise about it in a cross-language way and I think the present proposal is correct. The question of what language must be used to express this at the Python level is an interesting one. While I favor names like IntegerMatrix, ..., reserving Matrix to be a "smart" operator that decides on the type from the arguments, there is a place for a version with a calculable type or type code, so that one can get different things on different platforms with the same source by doing tests on the system type. I think Array is a better name than Matrix and if 'array' weren't already taken would have no hesitation in going ahead. If we picked Array as the name that is now Matrix, what would we call that which is now 'matrix'? ArrayFromSequence ? -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Dec 9 11:06:38 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA03136 for matrix-sig-people; Sat, 9 Dec 1995 11:06:38 -0500 Received: from harfang.CC.UMontreal.CA ([132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id LAA03132 for ; Sat, 9 Dec 1995 11:06:35 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id LAA17952 (8.6.11/IDA-1.6); Sat, 9 Dec 1995 11:05:46 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id LAA00160; Sat, 9 Dec 1995 11:05:45 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id LAA12973; Sat, 9 Dec 1995 11:05:43 -0500 Date: Sat, 9 Dec 1995 11:05:43 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199512091605.LAA12973@cyclone.ERE.UMontreal.CA> To: dubois1@llnl.gov CC: matrix-sig@python.org In-reply-to: <30C8D2E2.6190@llnl.gov> (dubois1@llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available Sender: owner-matrix-sig@python.org Precedence: bulk interesting one. While I favor names like IntegerMatrix, ..., reserving Matrix to be a "smart" operator that decides on the type from the arguments, there is a place for a version with a calculable type or type code, so that one can get different things on different platforms with the same source by doing tests on the system type. The "smart" version could take an optional argument indicating the type somehow. In addition to the typecodes, it would be nice if it would accept the return values of the builtin function type() . Then you could write something like list = [1,3,2] Array(list, type(list[0])) The typecode will still be needed for the types that have no Python equivalents, like C-floats. I think Array is a better name than Matrix and if 'array' weren't already taken would have no hesitation in going ahead. If we picked Once more, it is only the module name "array" that we can't use. Nothing else is global in Python. Array as the name that is now Matrix, what would we call that which is now 'matrix'? ArrayFromSequence ? I still propose that "Array" should behave like "matrix" does now. That way we need only one constructor. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Dec 9 11:11:11 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA03136 for matrix-sig-people; Sat, 9 Dec 1995 11:06:38 -0500 Received: from harfang.CC.UMontreal.CA ([132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id LAA03132 for ; Sat, 9 Dec 1995 11:06:35 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id LAA17952 (8.6.11/IDA-1.6); Sat, 9 Dec 1995 11:05:46 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id LAA00160; Sat, 9 Dec 1995 11:05:45 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id LAA12973; Sat, 9 Dec 1995 11:05:43 -0500 Date: Sat, 9 Dec 1995 11:05:43 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199512091605.LAA12973@cyclone.ERE.UMontreal.CA> To: dubois1@llnl.gov CC: matrix-sig@python.org In-reply-to: <30C8D2E2.6190@llnl.gov> (dubois1@llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available Sender: owner-matrix-sig@python.org Precedence: bulk interesting one. While I favor names like IntegerMatrix, ..., reserving Matrix to be a "smart" operator that decides on the type from the arguments, there is a place for a version with a calculable type or type code, so that one can get different things on different platforms with the same source by doing tests on the system type. The "smart" version could take an optional argument indicating the type somehow. In addition to the typecodes, it would be nice if it would accept the return values of the builtin function type() . Then you could write something like list = [1,3,2] Array(list, type(list[0])) The typecode will still be needed for the types that have no Python equivalents, like C-floats. I think Array is a better name than Matrix and if 'array' weren't already taken would have no hesitation in going ahead. If we picked Once more, it is only the module name "array" that we can't use. Nothing else is global in Python. Array as the name that is now Matrix, what would we call that which is now 'matrix'? ArrayFromSequence ? I still propose that "Array" should behave like "matrix" does now. That way we need only one constructor. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Dec 9 16:07:49 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA04124 for matrix-sig-people; Sat, 9 Dec 1995 16:07:49 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id QAA04120 for ; Sat, 9 Dec 1995 16:07:44 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA14361; Sat, 9 Dec 95 16:06:08 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) id QAA05656; Sat, 9 Dec 1995 16:12:05 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199512092112.QAA05656@maigret> Subject: [PYTHON MATRIX-SIG] Display of 1 & 2D matrices? To: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Sat, 9 Dec 1995 16:12:04 -0500 (EST) Cc: guido@CNRI.Reston.VA.US, matrix-sig@python.org In-Reply-To: <199510021627.MAA14222@cyclone.ERE.UMontreal.CA> from "Hinsen Konrad" at Oct 2, 95 12:27:22 pm X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 196 Sender: owner-matrix-sig@python.org Precedence: bulk Pardon this digression please: Has anyone started working on display modules for 1 and 2D matrices using any of the gui systems available for Python (Tk-based, X-based, whatever-based)? --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Dec 9 16:09:18 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA04124 for matrix-sig-people; Sat, 9 Dec 1995 16:07:49 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id QAA04120 for ; Sat, 9 Dec 1995 16:07:44 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA14361; Sat, 9 Dec 95 16:06:08 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) id QAA05656; Sat, 9 Dec 1995 16:12:05 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199512092112.QAA05656@maigret> Subject: [PYTHON MATRIX-SIG] Display of 1 & 2D matrices? To: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Sat, 9 Dec 1995 16:12:04 -0500 (EST) Cc: guido@CNRI.Reston.VA.US, matrix-sig@python.org In-Reply-To: <199510021627.MAA14222@cyclone.ERE.UMontreal.CA> from "Hinsen Konrad" at Oct 2, 95 12:27:22 pm X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 196 Sender: owner-matrix-sig@python.org Precedence: bulk Pardon this digression please: Has anyone started working on display modules for 1 and 2D matrices using any of the gui systems available for Python (Tk-based, X-based, whatever-based)? --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Dec 11 08:40:27 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id IAA09173 for matrix-sig-people; Mon, 11 Dec 1995 08:40:27 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id IAA09169 for ; Mon, 11 Dec 1995 08:40:22 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id NAA26539; Mon, 11 Dec 1995 13:39:44 GMT From: "Jim Fulton, U.S. Geological Survey" Message-Id: <9512110839.ZM26537@dsdbqvarsa.er.usgs.gov> Date: Mon, 11 Dec 1995 08:39:43 -0500 In-Reply-To: "Paul. Dubois" "Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available" (Dec 8, 4:05pm) References: <199512082316.SAA05920@cyclone.ERE.UMontreal.CA> <30C8D2E2.6190@llnl.gov> X-Phone: (703) 648-5622 X-Face: (;vf"lccr&V=y, Hinsen Konrad Subject: Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available Cc: chris.chase@jhuapl.edu, matrix-sig@python.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-matrix-sig@python.org Precedence: bulk One minor note on these naming discussions. It is necessary for names to be unique without regard to case. That is, although Python is case sensitive, some of the operating systems it functions in are not, and since module names are found in the local file systems, the names like array and Array may conflict. Jim On Dec 8, 4:05pm, Paul. Dubois wrote: > Subject: Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is availabl > A followup to the remarks about precision. > In fact, Fortran 90 goes to considerable lengths to try to improve this > situation but it isn't REAL*8 that is the solution. Fortran 90 uses > something called a Kind specifier, and Kind specifiers are calculable > from desired characteristics at compile time. Actually knowing that > something is a certain number of bits doesn't tell you what you need to > know, as there is a tradeoff between precision and dynamic range. > Anyway, you end up saying REAL(kind) x, y, z. > > I believe Jim has made fundamentally the right decision in deciding that > the abstraction he is after is "things that store a homogenous bunch of > stuff in contiguous memory". This would be a parameterized type in other > languages, such as Eiffel's ARRAY[T], where one actually instantiates > ARRAY[DOUBLE], ARRAY[INTEGER], ARRAY[CHARACTER], ARRAY[TELEPHONE_BOOK], > etc. Since a large use of this abstraction is connection to C and > Fortran libraries, one has to be precise about it in a cross-language > way and I think the present proposal is correct. The question of what > language must be used to express this at the Python level is an > interesting one. While I favor names like IntegerMatrix, ..., reserving > Matrix to be a "smart" operator that decides on the type from the > arguments, there is a place for a version with a calculable type or type > code, so that one can get different things on different platforms with > the same source by doing tests on the system type. > > I think Array is a better name than Matrix and if 'array' weren't > already taken would have no hesitation in going ahead. If we picked > Array as the name that is now Matrix, what would we call that which is > now 'matrix'? ArrayFromSequence ? > > -- > Paul F. Dubois, L-472 (510)-422-5426 > Lawrence Livermore National Laboratory FAX (510)-423-9969 > Livermore, CA 94550 dubois1@llnl.gov > Consulting: PaulDubois@aol.com > > ================= > MATRIX-SIG - SIG on Matrix Math for Python > > send messages to: matrix-sig@python.org > administrivia to: matrix-sig-request@python.org > ================= >-- End of excerpt from Paul. Dubois ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Dec 11 09:34:09 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA09454 for matrix-sig-people; Mon, 11 Dec 1995 09:34:09 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id JAA09450 for ; Mon, 11 Dec 1995 09:34:07 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id JAA05674 (8.6.11/IDA-1.6); Mon, 11 Dec 1995 09:32:08 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA24365; Mon, 11 Dec 1995 09:32:06 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA26952; Mon, 11 Dec 1995 09:32:05 -0500 Date: Mon, 11 Dec 1995 09:32:05 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199512111432.JAA26952@cyclone.ERE.UMontreal.CA> To: jfulton@usgs.gov CC: dubois1@llnl.gov, chris.chase@jhuapl.edu, matrix-sig@python.org In-reply-to: <9512110839.ZM26537@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available Sender: owner-matrix-sig@python.org Precedence: bulk It is necessary for names to be unique without regard to case. That is, although Python is case sensitive, some of the operating systems it functions in are not, and since module names are found in the local file systems, the names like array and Array may conflict. Does that mean that Python doesn't do any clever filename mapping on sytems that do not support case distinction and/or long file names? I never checked, but I always assumed that it does something like the GNU library that maps Unix filenames to short DOS-style filenames (i.e. provide a mapping that is as unique as possible, even if that produces strange-looking filenames). So, in effect, Python modules names are restricted to 8 lower-case characters? That would create a few problems with the Python code I have produced so far... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Dec 11 09:36:23 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA09478 for matrix-sig-people; Mon, 11 Dec 1995 09:36:23 -0500 Received: from muecke.appl-math.tu-muenchen.de (muecke.appl-math.tu-muenchen.de [129.187.242.1]) by python.org (8.6.12/8.6.12) with SMTP id JAA09421 for ; Mon, 11 Dec 1995 09:23:52 -0500 Received: from piranha.appl-math.tu-muenchen.de by muecke.appl-math.tu-muenchen.de via SMTP (931110.SGI/911001.SGI) for matrix-sig@python.org id AA11863; Mon, 11 Dec 95 15:11:57 +0100 Received: by piranha.appl-math.tu-muenchen.de (940816.SGI.8.6.9/911001.SGI) for matrix-sig@python.org id PAA04025; Mon, 11 Dec 1995 15:11:52 +0100 From: "Thomas Schwaller" Message-Id: <9512111511.ZM4023@piranha.appl-math.tu-muenchen.de> Date: Mon, 11 Dec 1995 15:11:45 +0100 X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] plmodule.c, openglmodule.c Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-matrix-sig@python.org Precedence: bulk Hi Matrix SIG'ers, 1) First a comment on the Array, array, matrix discussion... What about the name "multiarray" for Array. When I see uppercase letters I always think I can derive from that class, but for Array this would not be the case. Just an idea, not really important. 2) I found a bug in my plmodule.c. When you use just matrices everything is ok but when tuple or list input is transformed to matrices these are dereferenced too early. Fixed that bug. ftp://ftp.appl-math.tu-muenchen.de/pub/et/plmodule.c.0.3.gz Will "try" to put it on our matrix location. Connections are nearly impossible for me here! :-( 3) Completely reworked my OpenGl module. This was quite old, used the old naming convention and the cgen script for producing the glmodule. Now everything works with matrices as input, not just tuples or lists. The module is not completely finished yet, but you can already do a whole bunch of things with it (257 procedures up to now in the module). For GLUT or OpenGL widgets and general information about this openGL module check also the Info collected by David Ascher at http://maigret.cog.brown.edu:80/python/opengl/ If you want to have a look at this preversion check: ftp://ftp.appl-math.tu-muenchen.de/pub/et/openglmodule.c.0.1.gz If there's enough interest in that I will certainly contribute a lot of examples. 4) Heard about the visual toolkit C++ classes and the Tcl binding for it. Coolest thing for OpenGl graphics at the moment (at least for me! :-)) Ken Martin (one of the authors) agreed to to a python binding with me. The Tcl binding works well (I already did some wrappers for Python `a la Tkinter, but we will do a native Python binding) and at the moment he is doing a Java binding. To map the quite complex data hierarchy to python I will certainly need advice from people having done this before. Unfortunately startup time is quite high and comiling the classes and working with them shows the limits of C++ compiler technology (at least the one I have, g++ and SGI CC). Using the code in the classes directly in a Python module would be great but why developping C++ classes you will ask. Well, ... good question ... >>THIS WAS JUST FOR YOUR INFORMATION<< Hope it's not boring you to death! ;_) 5) How was the feedback at the python workshop concernig the matrixmodule. Would be nice to hear a word or two. 6) Read about the sprse matrix stuff in the paper for the workshop. Is this making any progress or is it a long term story not been started yet. Sorry I'm just curious! Tom ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Dec 11 10:15:00 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA09645 for matrix-sig-people; Mon, 11 Dec 1995 10:15:00 -0500 Received: from catch.ivab.se (catch.ivab.se [193.180.173.22]) by python.org (8.6.12/8.6.12) with ESMTP id KAA09640 for ; Mon, 11 Dec 1995 10:14:57 -0500 Received: from arnold.image.ivab.se (arnold.image.ivab.se [193.180.173.119]) by catch.ivab.se (8.6.12/8.6.12) with SMTP id QAA23823 for ; Mon, 11 Dec 1995 16:14:21 +0100 Received: from kalle.image.ivab.se by arnold.image.ivab.se; (5.65v3.2/1.1.8.2/16Nov95-0104AM) id AA01132; Mon, 11 Dec 1995 16:14:17 +0100 Received: by kalle.image.ivab.se; (5.65/1.1.8.2/27Jul95-0635PM) id AA28909; Mon, 11 Dec 1995 16:14:15 +0100 Date: Mon, 11 Dec 1995 16:14:15 +0100 Message-Id: <9512111514.AA28909@kalle.image.ivab.se> From: Fredrik Lundh To: matrix-sig@python.org In-Reply-To: <199512111432.JAA26952@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Subject: Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available Reply-To: fredrik_lundh@ivab.se Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk > Does that mean that Python doesn't do any clever filename mapping > on sytems that do not support case distinction and/or long file names? As with most other things, Python leaves it to the underlying system to handle this. Would think that MS-DOS and Windows 3.1 is the worst case, where filenames are _silently truncated_ to fit into the 8.3 scheme. So "import ContainerIO" will in look for "containe.py". Since my editor does the same, I have no problems when writing modules, as long as I make sure that the 8 first characters are unique. This is no problem under Windows 95/NT, even on FAT file systems, nor on Macs. Don't know about Amigas. (Generally, I think its a bad idea to let lowercase and capitalized versions of the same name mean different things.) /F ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Dec 11 10:17:10 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA09659 for matrix-sig-people; Mon, 11 Dec 1995 10:17:10 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id KAA09655 for ; Mon, 11 Dec 1995 10:17:07 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id PAA27296; Mon, 11 Dec 1995 15:16:21 GMT From: "Jim Fulton, U.S. Geological Survey" Message-Id: <9512111016.ZM27294@dsdbqvarsa.er.usgs.gov> Date: Mon, 11 Dec 1995 10:16:20 -0500 In-Reply-To: hinsenk@ERE.UMontreal.CA (Hinsen Konrad) "Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available" (Dec 11, 9:32am) References: <199512111432.JAA26952@cyclone.ERE.UMontreal.CA> X-Phone: (703) 648-5622 X-Face: (;vf"lccr&V=y Subject: Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is availabl > > It is necessary for names to be unique without regard to case. That is, > although Python is case sensitive, some of the operating systems it functions > in are not, and since module names are found in the local file systems, the > names like array and Array may conflict. > > Does that mean that Python doesn't do any clever filename mapping > on sytems that do not support case distinction and/or long file names? > I never checked, but I always assumed that it does something like > the GNU library that maps Unix filenames to short DOS-style filenames > (i.e. provide a mapping that is as unique as possible, even if that > produces strange-looking filenames). > > So, in effect, Python modules names are restricted to 8 lower-case > characters? That would create a few problems with the Python code I > have produced so far... Actually, the 8-character limit is not as serious, since: 1. For most dos/windows-3.1x programs, including python, long file names are automatically truncated, and 2. Windows NT and Windows-95 do not have the 8-character limitation. Still, Windows-NT and Windows 95 are both case-insensitive, and when Python is told to import "Spam", it will happily import from Spam.py, spam.py. SPAM.PY, ... Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Dec 11 13:01:40 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA10522 for matrix-sig-people; Mon, 11 Dec 1995 13:01:40 -0500 Received: from eeel.nist.gov (sparky.eeel.nist.gov [129.6.64.163]) by python.org (8.6.12/8.6.12) with SMTP id NAA10518 for ; Mon, 11 Dec 1995 13:01:37 -0500 Received: from acdc.eeel.nist.gov by eeel.nist.gov (4.1/SMI-3.2-del.6) id AA17001; Mon, 11 Dec 95 13:01:17 EST Received: by acdc.eeel.nist.gov (4.1/SMI-3.2-del.5) id AA00826; Mon, 11 Dec 95 12:57:22 EST Date: Mon, 11 Dec 95 12:57:22 EST Message-Id: <9512111757.AA00826@acdc.eeel.nist.gov> From: Michael McLay To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available In-Reply-To: <199512111432.JAA26952@cyclone.ERE.UMontreal.CA> References: <9512110839.ZM26537@dsdbqvarsa.er.usgs.gov> <199512111432.JAA26952@cyclone.ERE.UMontreal.CA> Sender: owner-matrix-sig@python.org Precedence: bulk Hinsen Konrad writes: > > It is necessary for names to be unique without regard to case. That is, > although Python is case sensitive, some of the operating systems it functions > in are not, and since module names are found in the local file systems, the > names like array and Array may conflict. > > Does that mean that Python doesn't do any clever filename mapping > on sytems that do not support case distinction and/or long file names? > I never checked, but I always assumed that it does something like > the GNU library that maps Unix filenames to short DOS-style filenames > (i.e. provide a mapping that is as unique as possible, even if that > produces strange-looking filenames). > > So, in effect, Python modules names are restricted to 8 lower-case > characters? That would create a few problems with the Python code I > have produced so far... I don't think a universally agreed upon solution has ever been reached on the filename mapping issue. It wouldn't be to hard to implement a platform independent mechanism that allowed case sensitive names. It could all be done by redefining the import statement so that filenames are looked up at run time on some platforms. The filesystem on CD-ROMs uses this technique for allowing "portable" file names. Python might even use the same encoding rules. This will have to be done when the PSA develops a multi-platform CD-ROM distribution of Python. Michael ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Dec 11 18:29:22 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA12273 for matrix-sig-people; Mon, 11 Dec 1995 18:29:22 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id SAA12268 for ; Mon, 11 Dec 1995 18:29:10 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id SAA05351 (8.6.11/IDA-1.6); Mon, 11 Dec 1995 18:27:46 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id SAA29272; Mon, 11 Dec 1995 18:27:46 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id SAA05548; Mon, 11 Dec 1995 18:27:45 -0500 Date: Mon, 11 Dec 1995 18:27:45 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199512112327.SAA05548@cyclone.ERE.UMontreal.CA> To: jfulton@usgs.gov CC: matrix-sig@python.org In-reply-to: <9512111016.ZM27294@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available Sender: owner-matrix-sig@python.org Precedence: bulk Actually, the 8-character limit is not as serious, since: 1. For most dos/windows-3.1x programs, including python, long file names are automatically truncated, and Bad enough. I have lots of module names that do not differ in the first eight characters. Not much of a problem for me, because I am not going to use 8-character systems any more unless forced with a gun, but a problem for Python portability in general. Still, Windows-NT and Windows 95 are both case-insensitive, and when Python is told to import "Spam", it will happily import from Spam.py, spam.py. SPAM.PY, ... Another portability problem. Although in general I wouldn't use upper- and lowercase variants of the same name together, this can happen by accident, and then I have code that works only on some platforms. Since Pyton uses case-sensitive names everywhere, it should try to enforce this as much as possible also for module names. Michael McLay on the same topic: I don't think a universally agreed upon solution has ever been reached on the filename mapping issue. It wouldn't be to hard to implement a platform independent mechanism that allowed case sensitive names. It could all be done by redefining the import statement so that filenames are looked up at run time on some platforms. The filesystem on That would be a good solution. Is there any SIG we can make responsible for this? ;-) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Dec 12 18:39:21 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA17004 for matrix-sig-people; Tue, 12 Dec 1995 18:39:21 -0500 Received: from qvarsx.er.usgs.GOV (qvarsx.er.usgs.gov [130.11.51.82]) by python.org (8.6.12/8.6.12) with ESMTP id SAA16999 for ; Tue, 12 Dec 1995 18:39:18 -0500 Received: from localhost (disqvarsa.er.usgs.gov [130.11.50.157]) by qvarsx.er.usgs.GOV (EMAIL 1.2.1) with ESMTP id XAA14048; Tue, 12 Dec 1995 23:38:23 GMT Message-Id: <199512122338.XAA14048@qvarsx.er.usgs.GOV> X-Face: (;vf"lccr&V=y Subject: [PYTHON MATRIX-SIG] [comp.lang.python] Proposal for sharing C interfaces between extension modules Date: Tue, 12 Dec 1995 18:38:35 -0500 From: "Jim Fulton, U.S. Geological Survey" Sender: owner-matrix-sig@python.org Precedence: bulk The following message was posted on the Python list. One of the driving reasons for this proposal is to simplify the configuration of the matrix object implementation and the many extension modules that will use it. My idea is that the matrix object can be included in a module and that the C interface to the matrix object type can be exported as a collection of CObjects or through a single CObject that is an array or struct (or other data structure) of C function pointers. Then extension modules that want to use the C interface to the matrix object can import the matrix module and use PyObject_GetAttrString to get access to the C interface. In this way, the matrix object need not be munged into the standard source tree. -- Jim ------- Forwarded Message From: jfulton@disqvarsa.er.usgs.GOV (Jim Fulton ) Newsgroups: comp.lang.python Subject: Proposal for sharing C interfaces between extension modules Date: 12 Dec 1995 19:36:04 GMT Organization: U.S. Geological Survey NNTP-Posting-Host: disqvarsa.er.usgs.gov Problem: It is sometimes the case that an extension module needs to export a C interface to other extension modules. This is especially true for modules that define useful built-in types. If modules are dynamically linked, sharing C interfaces between modules can get someone complicated, because the extension modules need to get linked together and get linked with Python. How this is done can vary significantly from system to system. Python already provides a dynamic linking facility, via import. An extension module can import other modules, including other extension modules. So, for example, a Matrix module could import a complex number module and use PyObject_GetAttrString to get any objects (classes, functions, etc) exported by the complex module. This mechanism is nice because it doesn't depend in any way how the module being imported was linked. Python handles the linking for you. Unfortunately, this currently only works with Python objects exported by a module. (Fortunately, if only a Python interface to a module is needed, this mechanism allows an extension module to "link" to another module regardless of whether the other module is a Python module or an extension module. Cool. :) I propose adding a very small new type to the core language, called CObject, that is a Python wrapper for C objects. On my system, this new type adds a total of 204 bytes to the text portion of the interpreter. :-) If an extension module wishes to export a C object (such as a C function, or an array of C functions) to other extension modules, the extension module would use the function CObject_FromVoidPtr to create a Python object that wraps a pointer to the C object. The module would then insert this object in the module's dictionary during module initialization. Here is the declaration for CObject_FromVoidPtr: /* Create a CObject from a pointer to a C object and an optional destrutor function. If the second argument is non-null, then it will be called with the first argument if and when the CObject is destroyed. */ extern PyObject * CObject_FromVoidPtr Py_PROTO((void *cobj, void (*destruct)(void*))); An extension module that wishes to use a C object exported by another extension module, would: - Import the other module, - Use PyObject_GetAttrString to get the CObject, using a published name, and - Use CObject_AsVoidPtr to obtain a pointer to the C object, Here is the declaration for CObject_AsVoidPtr: /* Retrieve a pointer to a C object from a CObject. */ extern void * CObject_AsVoidPtr Py_PROTO((PyObject *)); Of course, CObjects exported by a module can be accessed from Python, but they provide no useful behavior that can be accessed directly from Python scripts. They can be assigned to Python variables and passed to extension functions or extension type methods. This small extension to the language should significantly ease the configuration of modules that need or want to interface with each other via C. -- -- Jim Fulton jfulton@usgs.gov (703) 648-5622 U.S. Geological Survey, Reston VA 22092 This message is being posted to obtain or provide technical information relating to my duties at the U.S. Geological Survey. ------- End of Forwarded Message ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Dec 13 12:51:32 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA20383 for matrix-sig-people; Wed, 13 Dec 1995 12:51:32 -0500 Received: from mama-bear (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id MAA20379 for ; Wed, 13 Dec 1995 12:51:27 -0500 Received: from ling-ling.lcs.mit.edu.LCS.MIT.EDU by mama-bear (4.1/SLS-4.1.1) id AA04654; Wed, 13 Dec 95 12:50:21 EST Received: by ling-ling.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA09345; Wed, 13 Dec 95 12:51:33 -0500 Message-Id: <9512131751.AA09345@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Wed, 13 Dec 95 12:51:32 -0500 To: "Paul. Dubois" Subject: [PYTHON MATRIX-SIG] Re: Version 2.0 Cc: matrix-sig@python.org References: <9512011859.AA10898@kristen.llnl.gov> <9512011916.AA04586@ling-ling.lcs.mit.edu.LCS.MIT.EDU> <30BF59D4.1E33@llnl.gov> <9512071830.AA07326@ling-ling.lcs.mit.edu.LCS.MIT.EDU> <30C8AAE4.616E@llnl.gov> Sender: owner-matrix-sig@python.org Precedence: bulk Paul sent me this as private comments and I'm going to violate all rules of Netiquette by replying to the entire group. I think that there are some important things in here to comment on. > a. It seems like if I don't do the import of omath then 2.*x doesn't > work. After a from omath import * it does. (?) This is true. You have to import omath somehow, or matrices do not have any numeric behavior (notice that if you use import Matrix, or from Matrix import *. this gets done for you). Here are the reasons... 1) This makes the matrixobject code a lot smaller, and therefore likelier to be included in the base python distribution (fingers crossed) 2) This makes add.reduce(m) possible. 3) This makes it possible to have a vectorized_ofuncmodule (or something like that) that defines high performance versions of these operators for a particular machine, and then have them be used when you say "a+b". > b. Matrix(1,2,3) produces a Matrix_l, or at least that is what it > claims it is. Should be a Matrix_i? This will be seen as a BIG issue, and I'll write more about it later (I've a few other demands on my time right now). It should either be Matrix_b (a matrix of bytes, because 1,2, and 3 can all be stored in a single byte) or it should be Matrix_l (a matrix of longs because a python integer technically holds a C long). > c. I preferred the [1.,2.,3.] rather than the Matrix_d(...). A lot. A > whole lot. Provide a query function for those who really want to know > the flavor of matrix. The printing of matrices can be GREATLY improved. I changed the matrix_print function so that it goes out and calls a python routine to do the actual printing. Check out the PrintMatrix function in Matrix.py. Currently this function just returns the string representation of the object (for which Matrix_d is a good choice modulo naming conventions). Anybody who has good ideas about how to print a matrix can go in and modify this python function to get whatever print behavior they like. If you come up with a good print function, send it to me, and I'll include the best one in the next release. > d. I really like the way Matrix decides on a type. Thanks. > e. I like the idea that others have suggested of IntegerMatrix(...) > instead of a constructor that takes a character to decide type. However, > there are times when the latter is useful because you are COMPUTING the > type (like choose a precision based on the system, for example) so I > wouldn't get rid of the character method. > The fact that Matrix() does the right thing and that everything > coerces mitigates most of this to the margins, however. And here I decided to respond to your message in order to stay out of the naming issue. I like both approaches, and I'm waiting to see what things look like after the dust has settled. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Dec 13 14:06:08 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA20935 for matrix-sig-people; Wed, 13 Dec 1995 14:06:08 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id OAA20931 for ; Wed, 13 Dec 1995 14:06:03 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id OAA29336 (8.6.11/IDA-1.6); Wed, 13 Dec 1995 14:04:11 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA01482; Wed, 13 Dec 1995 14:04:11 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA20843; Wed, 13 Dec 1995 14:04:10 -0500 Date: Wed, 13 Dec 1995 14:04:10 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199512131904.OAA20843@cyclone.ERE.UMontreal.CA> To: jjh@mama-bear.lcs.mit.edu CC: dubois1@llnl.gov, matrix-sig@python.org In-reply-to: <9512131751.AA09345@ling-ling.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Wed, 13 Dec 95 12:51:32 -0500) Subject: Re: [PYTHON MATRIX-SIG] Re: Version 2.0 Sender: owner-matrix-sig@python.org Precedence: bulk This is true. You have to import omath somehow, or matrices do not have any numeric behavior (notice that if you use import Matrix, or from Matrix import *. this gets done for you). Here are the reasons... If omath is essential to the module "matrix", then this module should do the import. All it takes is PyImport_ImportModule("omath"); somewhere in the module initialization. > b. Matrix(1,2,3) produces a Matrix_l, or at least that is what it > claims it is. Should be a Matrix_i? This will be seen as a BIG issue, and I'll write more about it later (I've a few other demands on my time right now). It should either be Matrix_b (a matrix of bytes, because 1,2, and 3 can all be stored in a single byte) or it should be Matrix_l (a matrix of longs because a python integer technically holds a C long). I'd vote for "long" ("IntegerMatrix"). The idea is that the existence of all the types that Python normally doesn't have should be hidden as far as possible; those who need the smaller types to save memory should be the only ones who have to know about it. This guarantees that code like x = Matrix(a,b) x[0] = c will always work correctly as long as a, b, and c are of the same Python type. If the "smallest possible" rule were used, then this would show an unexpected behaviour if a=1, b=2 and c=10000. > c. I preferred the [1.,2.,3.] rather than the Matrix_d(...). A lot. A > whole lot. Provide a query function for those who really want to know > the flavor of matrix. I agree! Anybody who has good ideas about how to print a matrix can go in and modify this python function to get whatever print behavior they like. If you come up with a good print function, send it to me, and I'll include the best one in the next release. The first contest in this SIG! ;-) But before everyone starts coding essentially the same stuff, let's first discuss what the output format should look like and then search a volunteer to do the implementation. The two esential problems to be solved are: 1) How to print arrays of rank > 2. 2) How to deal with large arrays that don't fit on the screen. Personally I have a certain bias towards the APL output format, which is based on the following principles: - all elements of an array are printed using the same format, to make them line up nicely - there is no border decoration whatsoever, just elements separated by spaces - one-dimensional arays are printed in a line, two-dimensional arrays as a sequence of lines, N-dimensional arrays as sequences of (N-1)-dimensional arrays separated by (N-2) blank lines - lines longer than the terminal width (an adjustable parameter) are wrapped to the following line with an indentation of six characters This format is space-saving and very readable. The only problem is that there is no visible difference between a scalar and a higher-dimensional array with exactly one element, but this has not turned out to be a problem in practice. And here I decided to respond to your message in order to stay out of the naming issue. I like both approaches, and I'm waiting to see what things look like after the dust has settled. My suggestion for constructors: Array(sequence, typecode=None) transforms the sequence into an array of the given typecode; if the typecode is None, it determines the type by itself. Typecodes can be letters or type objects (as returned by type()). IntegerArray(sequence) transforms the given sequence into an array of integers. ShortIntegerArray(sequence) transforms the given sequence into an array of C-integers. and so on for the other specific types. Output of repr() should be IntegerArray() etc., output of str() should be the "pretty" format to be discussed separately. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Dec 13 14:20:16 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA21064 for matrix-sig-people; Wed, 13 Dec 1995 14:20:16 -0500 Received: from mama-bear (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id OAA21060 for ; Wed, 13 Dec 1995 14:20:13 -0500 Received: from ling-ling.lcs.mit.edu.LCS.MIT.EDU by mama-bear (4.1/SLS-4.1.1) id AA05536; Wed, 13 Dec 95 14:19:05 EST Received: by ling-ling.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA09380; Wed, 13 Dec 95 14:20:12 -0500 Message-Id: <9512131920.AA09380@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Wed, 13 Dec 95 14:20:11 -0500 To: "Thomas Schwaller" Subject: Re: [PYTHON MATRIX-SIG] plmodule.c, openglmodule.c Cc: matrix-sig@python.org References: <9512111511.ZM4023@piranha.appl-math.tu-muenchen.de> Sender: owner-matrix-sig@python.org Precedence: bulk It's great to see these modules! This object is primarily intended as a vehicle to make it easier to extend python with these sorts of numerically intensive libraries. The fact that writing them has gone well so far is a great sign. > 5) How was the feedback at the python workshop concernig the matrixmodule. > Would be nice to hear a word or two. Everybody seemed very positive about it. They loved the surface plot you sent me. Many people (including Guido) seemed rather excited when I mentioned that you were also doing an openglmodule "right" using matrices as inputs. A few people seemed skeptical about the speed claims, but hopefully those of you who have been using the object believe me. Check out the main list for a discussion of complex numbers that was motivated by the matrix discussion at the meeting. > 6) Read about the sprse matrix stuff in the paper for the workshop. Is this > making any progress or is it a long term story not been started yet. > Sorry I'm just curious! I'm writing a sparse matrix object in python on top of the matrix object. This is partially a proof of concept, but it is also because my speech recognition code desperately needs sparse matrices. Currently it's good enough for my applications. I'll try and get a rough version of this assembled for public consumption by the 0.21 release of the matrix object. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Dec 13 14:23:42 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA21064 for matrix-sig-people; Wed, 13 Dec 1995 14:20:16 -0500 Received: from mama-bear (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id OAA21060 for ; Wed, 13 Dec 1995 14:20:13 -0500 Received: from ling-ling.lcs.mit.edu.LCS.MIT.EDU by mama-bear (4.1/SLS-4.1.1) id AA05536; Wed, 13 Dec 95 14:19:05 EST Received: by ling-ling.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA09380; Wed, 13 Dec 95 14:20:12 -0500 Message-Id: <9512131920.AA09380@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Wed, 13 Dec 95 14:20:11 -0500 To: "Thomas Schwaller" Subject: Re: [PYTHON MATRIX-SIG] plmodule.c, openglmodule.c Cc: matrix-sig@python.org References: <9512111511.ZM4023@piranha.appl-math.tu-muenchen.de> Sender: owner-matrix-sig@python.org Precedence: bulk It's great to see these modules! This object is primarily intended as a vehicle to make it easier to extend python with these sorts of numerically intensive libraries. The fact that writing them has gone well so far is a great sign. > 5) How was the feedback at the python workshop concernig the matrixmodule. > Would be nice to hear a word or two. Everybody seemed very positive about it. They loved the surface plot you sent me. Many people (including Guido) seemed rather excited when I mentioned that you were also doing an openglmodule "right" using matrices as inputs. A few people seemed skeptical about the speed claims, but hopefully those of you who have been using the object believe me. Check out the main list for a discussion of complex numbers that was motivated by the matrix discussion at the meeting. > 6) Read about the sprse matrix stuff in the paper for the workshop. Is this > making any progress or is it a long term story not been started yet. > Sorry I'm just curious! I'm writing a sparse matrix object in python on top of the matrix object. This is partially a proof of concept, but it is also because my speech recognition code desperately needs sparse matrices. Currently it's good enough for my applications. I'll try and get a rough version of this assembled for public consumption by the 0.21 release of the matrix object. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Dec 13 18:24:30 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA00730 for matrix-sig-people; Wed, 13 Dec 1995 18:24:30 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id SAA00726 for ; Wed, 13 Dec 1995 18:24:25 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id SAA08821 (8.6.11/IDA-1.6 for ); Wed, 13 Dec 1995 18:23:49 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id SAA11638; Wed, 13 Dec 1995 18:23:47 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id SAA07693; Wed, 13 Dec 1995 18:23:46 -0500 Date: Wed, 13 Dec 1995 18:23:46 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199512132323.SAA07693@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Another naming issue Sender: owner-matrix-sig@python.org Precedence: bulk While we are discussing names, let me point out another one that is not perfect: omath. From the source code I know that "o" is supposed to mean "optimized", which may well be true, but the characterising feature from a user's point of view is generality (compared to "math"). So how about "gmath" (general math) or "umath" (universal math)? As a motivation for a quick decision, I promise to make two of my Python modules that use Xmath (no, *not* Xmas!) available as soon as there is a definite name. One implements 3D-Vectors (in the sense of geometrical objects, not vectors in the sense of linear algebra), the other automatic differentiation of arbitrary functions. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Dec 13 18:24:37 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA00736 for matrix-sig-people; Wed, 13 Dec 1995 18:24:37 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id SAA00732 for ; Wed, 13 Dec 1995 18:24:34 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id XAA11094; Wed, 13 Dec 1995 23:24:32 GMT From: "Jim Fulton, U.S. Geological Survey" Message-Id: <9512131824.ZM11092@dsdbqvarsa.er.usgs.gov> Date: Wed, 13 Dec 1995 18:24:31 -0500 In-Reply-To: James Hugunin "[PYTHON MATRIX-SIG] Re: Version 2.0" (Dec 13, 12:51pm) References: <9512011859.AA10898@kristen.llnl.gov> <9512011916.AA04586@ling-ling.lcs.mit.edu.LCS.MIT.EDU> <30BF59D4.1E33@llnl.gov> <9512071830.AA07326@ling-ling.lcs.mit.edu.LCS.MIT.EDU> <30C8AAE4.616E@llnl.gov> <9512131751.AA09345@ling-ling.lcs.mit.edu.LCS.MIT.EDU> X-Phone: (703) 648-5622 X-Face: (;vf"lccr&V=y, "Paul. Dubois" Subject: Re: [PYTHON MATRIX-SIG] Re: Version 2.0 Cc: matrix-sig@python.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-matrix-sig@python.org Precedence: bulk On Dec 13, 12:51pm, James Hugunin wrote: > Subject: [PYTHON MATRIX-SIG] Re: Version 2.0 > > Paul sent me this as private comments and I'm going to violate all rules of > Netiquette by replying to the entire group. I think that there are some > important things in here to comment on. > > > a. It seems like if I don't do the import of omath then 2.*x doesn't > > work. After a from omath import * it does. (?) > > This is true. You have to import omath somehow, or matrices do not have > any numeric behavior (notice that if you use import Matrix, or from Matrix > import *. this gets done for you). Here are the reasons... > > 1) This makes the matrixobject code a lot smaller, and therefore likelier > to be included in the base python distribution (fingers crossed) I don't see that this makes any difference. The matrix module can and should be in the standard "distribution", but it need not be linked statically into the interpreter on systems that support dynamic linking, which includes most or all of the interesting ones. If it is not linked statically into the interpreter, but is available as a dynamically linked module, then the size is not so critical. I don't think the space issue justifies the weird side-effect dependencies indicated above. > > 2) This makes add.reduce(m) possible. This doesn't make any sense to me. But I haven't kept up with this list either. How does importing or not importing a module have any impact on the amove expression or it's feasabity? > 3) This makes it possible to have a vectorized_ofuncmodule (or something > like that) that defines high performance versions of these operators for a > particular machine, and then have them be used when you say "a+b". I don't get this either. Could you elaborate. > > b. Matrix(1,2,3) produces a Matrix_l, or at least that is what it > > claims it is. Should be a Matrix_i? > > This will be seen as a BIG issue, and I'll write more about it later (I've > a few other demands on my time right now). > > It should either be Matrix_b (a matrix of bytes, because 1,2, and 3 can all > be stored in a single byte) or it should be Matrix_l (a matrix of longs > because a python integer technically holds a C long). > > > c. I preferred the [1.,2.,3.] rather than the Matrix_d(...). A lot. A > > whole lot. Provide a query function for those who really want to know > > the flavor of matrix. > > The printing of matrices can be GREATLY improved. I changed the > matrix_print function so that it goes out and calls a python routine to do > the actual printing. Check out the PrintMatrix function in Matrix.py. > Currently this function just returns the string representation of the object > (for which Matrix_d is a good choice modulo naming conventions). > > Anybody who has good ideas about how to print a matrix can go in and modify > this python function to get whatever print behavior they like. If you come > up with a good print function, send it to me, and I'll include the best one > in the next release. > > > d. I really like the way Matrix decides on a type. > > Thanks. > > > e. I like the idea that others have suggested of IntegerMatrix(...) > > instead of a constructor that takes a character to decide type. However, > > there are times when the latter is useful because you are COMPUTING the > > type (like choose a precision based on the system, for example) so I > > wouldn't get rid of the character method. I agree. > > The fact that Matrix() does the right thing and that everything > > coerces mitigates most of this to the margins, however. > > And here I decided to respond to your message in order to stay out of the > naming issue. I like both approaches, and I'm waiting to see what things > look like after the dust has settled. I think both approaches should be provided. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Dec 14 13:42:07 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA05472 for matrix-sig-people; Thu, 14 Dec 1995 13:42:07 -0500 Received: from mama-bear (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id NAA05468 for ; Thu, 14 Dec 1995 13:42:04 -0500 Received: from ling-ling.lcs.mit.edu.LCS.MIT.EDU by mama-bear (4.1/SLS-4.1.1) id AA17833; Thu, 14 Dec 95 13:41:50 EST Received: by ling-ling.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA09889; Thu, 14 Dec 95 13:43:09 -0500 Message-Id: <9512141843.AA09889@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Thu, 14 Dec 95 13:43:08 -0500 To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath References: <199512132323.SAA07693@cyclone.ERE.UMontreal.CA> Sender: owner-matrix-sig@python.org Precedence: bulk This relates to both Konrad's and Jim's recent posts. Also, this is a bit of a rambling set of comments rather than a coherent argument. I just want people to understand where this weird ofunc thing came from, and what it does now. 1) What is an Optimized FUNCtion? An ofunc is a python object that contains C code to execute a function on arrays of raw C numbers (or on raw python objects, but that's another story) at close to compiled speed. Each function takes an arbitrary number of arguments of given types, and produces an arbitrary number of results of given types. For every desired combination of types that the function should operate on, there is another bit of C code to do the actual computation. 2) Why not bundle it into the matrix object? For one thing, I'd like to eventually clean things up and document them well enough that anybody can create an ofunc. They're really useful little critters for doing very fast computations on arrays of numbers. Also, there are things like generalized reductions that need to have something to grab onto as the function to reduce over. Without ofuncs as real python objects, I don't have any good way to express something like: add.reduce(m) (equivalent to but much faster than reduce(lambda x, y: x+y, m) ). There are also many of these creatures that don't really belong with the matrixobject, but which should behave in exactly the same way as "built-in" functions like add. These include things like hypot. 3) Why the current weird dependency on import omath? I wanted to have a module which defined add and subtract as ofuncs so that they could be used in things like add.reduce. I also wanted to be able to play with different implementations of the ofuncs for different purposes. For example, it should be possible to do the usual numeric tricks of special casing the currently very general inner loops to gain speed improvements. Using the current setup, this can be done by creating a new module, and using the ofuncs for things like add and subtract defined in this new module in the matrix object. Also, I noticed that the matrix object was quite useful without any math functions at all, and I thought that it was a nice coincidence that this arrangement made it possible to have a "stripped-down" matrixobject with no numerical capabilites if you so desired. 4) How this turned into a generalized math package I started to notice a certain duplication of code going on. For example, mathmodule now had a function to take the sin of a double, cmathmodule had one to take the sin of a complex number, and omathmodule (for these ofuncs I'd created) had one to take the sin of a matrix of doubles, complex, or even of generic python objects (that had a sin method). This made it a very small jump to allow a single double or complex to be treated as a 0d matrix, and completely replace the code in mathmodule and cmathmodule with omathmodule. This is what turned into the generic math module that now exists. I have no arguments with giving it a name more fitting of its new role. If nobody has any objections, or better ideas, I'll change the name to "umath" (gxxxx always makes me think of gnu code) for the upcoming bug-fix release. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Dec 14 15:03:28 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA05880 for matrix-sig-people; Thu, 14 Dec 1995 15:03:28 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id PAA05876 for ; Thu, 14 Dec 1995 15:03:23 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id UAA11808; Thu, 14 Dec 1995 20:03:14 GMT From: "Jim Fulton, U.S. Geological Survey" Message-Id: <9512141503.ZM11806@dsdbqvarsa.er.usgs.gov> Date: Thu, 14 Dec 1995 15:03:13 -0500 In-Reply-To: James Hugunin "[PYTHON MATRIX-SIG] A ramble on ofunc's and omath" (Dec 14, 1:43pm) References: <199512132323.SAA07693@cyclone.ERE.UMontreal.CA> <9512141843.AA09889@ling-ling.lcs.mit.edu.LCS.MIT.EDU> X-Phone: (703) 648-5622 X-Face: (;vf"lccr&V=y, matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-matrix-sig@python.org Precedence: bulk Hm. This made be reread: > > a. It seems like if I don't do the import of omath then 2.*x doesn't > > work. After a from omath import * it does. (?) > This is true. You have to import omath somehow, or matrices do not have > any numeric behavior (notice that if you use import Matrix, or from Matrix > import *. this gets done for you). Here are the reasons... Let me guess: x is an object returned from a module other than the matrix module that creates matrix objects, right? In the above example, Matrix has not been imported yet? Then the initialization code for Matrix causes the ofunc stuff to get wired into the matrix type? I'm more and more inclined to think that extension types (types not built into the language) should always be hosted and exported by an extension module and accessed from other modules via the import mechanism. If this were the case, then any extension module that created matrices would have to import the Matrix module to get access to a matrix constructor. The Matrix module would then import other needed modules, like omath. (See additional comment below.) I intend to rewrite FIDL so that modules that it creates import Matrix and use PyObject_GetAttrString to obtain matrix constructors. So if you get a matrix from a FIDL-generated module, Matrix and therefore omath, will have been imported and matrices will have numeric behavior. On Dec 14, 1:43pm, James Hugunin wrote: > Subject: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath > > This relates to both Konrad's and Jim's recent posts. Also, this is a bit > of a rambling set of comments rather than a coherent argument. I just want > people to understand where this weird ofunc thing came from, and what it > does now. > > 1) What is an Optimized FUNCtion? > > An ofunc is a python object that contains C code to execute a function on > arrays of raw C numbers (or on raw python objects, but that's another story) > at close to compiled speed. Each function takes an arbitrary number of > arguments of given types, and produces an arbitrary number of results of > given types. For every desired combination of types that the function > should operate on, there is another bit of C code to do the actual > computation. > > > 2) Why not bundle it into the matrix object? > > For one thing, I'd like to eventually clean things up and document them > well enough that anybody can create an ofunc. They're really useful little > critters for doing very fast computations on arrays of numbers. > > Also, there are things like generalized reductions that need to have > something to grab onto as the function to reduce over. Without ofuncs as > real python objects, I don't have any good way to express something like: > > add.reduce(m) > (equivalent to but much faster than reduce(lambda x, y: x+y, m) ). > > There are also many of these creatures that don't really belong with the > matrixobject, but which should behave in exactly the same way as "built-in" > functions like add. These include things like hypot. > > > 3) Why the current weird dependency on import omath? > > I wanted to have a module which defined add and subtract as ofuncs so that > they could be used in things like add.reduce. > > I also wanted to be able to play with different implementations of the > ofuncs for different purposes. For example, it should be possible to do the > usual numeric tricks of special casing the currently very general inner > loops to gain speed improvements. Using the current setup, this can be done > by creating a new module, and using the ofuncs for things like add and > subtract defined in this new module in the matrix object. > > Also, I noticed that the matrix object was quite useful without any math > functions at all, and I thought that it was a nice coincidence that this > arrangement made it possible to have a "stripped-down" matrixobject with no > numerical capabilites if you so desired. But you now have a weird interface to the matrix module. Assuming that on most systems, the matrix module will be dynamically linked anyway, I wouldn't worry about size. Just let the matrix module import omath so that the user who doesn't need omath can still use matrix math without having to be aware of your implementation decision to use omath. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Dec 14 18:49:23 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA00124 for matrix-sig-people; Thu, 14 Dec 1995 18:49:23 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id SAA00120 for ; Thu, 14 Dec 1995 18:49:18 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id XAA12632; Thu, 14 Dec 1995 23:07:58 GMT From: "Jim Fulton, U.S. Geological Survey" Message-Id: <9512141807.ZM12630@dsdbqvarsa.er.usgs.gov> Date: Thu, 14 Dec 1995 18:07:57 -0500 In-Reply-To: hinsenk@ERE.UMontreal.CA (Hinsen Konrad) "Re: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath" (Dec 14, 5:33pm) References: <199512142233.RAA15710@cyclone.ERE.UMontreal.CA> X-Phone: (703) 648-5622 X-Face: (;vf"lccr&V=y Subject: Re: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath > > I'm more and more inclined to think that extension types (types not > built into the language) should always be hosted and exported by an > extension module and accessed from other modules via the import > mechanism. If this were the case, then any extension module that > > That is certainly preferable. However, I am not sure whether it > is possible in all cases with the current implementation. Clearly > there is not much provision for C modules that depend on each other. > Your proposed CObjects solve one problem, but there may be more. There may be, but I haven't though of any yet. If you come up with any, I'd be happy to try to come up with a solution to them too. :-] Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Dec 14 19:18:00 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id TAA01819 for matrix-sig-people; Thu, 14 Dec 1995 19:18:00 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id TAA01815 for ; Thu, 14 Dec 1995 19:17:52 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id RAA09729 (8.6.11/IDA-1.6); Thu, 14 Dec 1995 17:33:38 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA18882; Thu, 14 Dec 1995 17:33:37 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA15710; Thu, 14 Dec 1995 17:33:36 -0500 Date: Thu, 14 Dec 1995 17:33:36 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199512142233.RAA15710@cyclone.ERE.UMontreal.CA> To: jfulton@usgs.gov CC: jjh@mama-bear.lcs.mit.edu, matrix-sig@python.org In-reply-to: <9512141503.ZM11806@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath Sender: owner-matrix-sig@python.org Precedence: bulk I'm more and more inclined to think that extension types (types not built into the language) should always be hosted and exported by an extension module and accessed from other modules via the import mechanism. If this were the case, then any extension module that That is certainly preferable. However, I am not sure whether it is possible in all cases with the current implementation. Clearly there is not much provision for C modules that depend on each other. Your proposed CObjects solve one problem, but there may be more. But you now have a weird interface to the matrix module. Assuming that on most systems, the matrix module will be dynamically linked anyway, I wouldn't worry about size. Just let the matrix module import omath so that the user who doesn't need omath can still use matrix math without having to be aware of your implementation decision to use omath. I agree. omath should be usable without matrix, but the reverse is not so important. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Dec 15 09:06:05 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA05010 for matrix-sig-people; Fri, 15 Dec 1995 09:06:05 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id JAA05006 for ; Fri, 15 Dec 1995 09:06:02 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id JAA23967 (8.6.11/IDA-1.6); Fri, 15 Dec 1995 09:05:17 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA11538; Fri, 15 Dec 1995 09:05:16 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA23256; Fri, 15 Dec 1995 09:05:15 -0500 Date: Fri, 15 Dec 1995 09:05:15 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199512151405.JAA23256@cyclone.ERE.UMontreal.CA> To: jfulton@usgs.gov CC: matrix-sig@python.org In-reply-to: <9512141807.ZM12630@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath Sender: owner-matrix-sig@python.org Precedence: bulk > That is certainly preferable. However, I am not sure whether it > is possible in all cases with the current implementation. Clearly > there is not much provision for C modules that depend on each other. > Your proposed CObjects solve one problem, but there may be more. There may be, but I haven't though of any yet. If you come up with any, I'd be happy to try to come up with a solution to them too. :-] It seems that you can do everything necessary with these CObjects. After all, you can make all non-static variables and functions known in this way, and then you have published everything that other statically linked C modules have access to. A potential problem is that another module can do whatever it wants with the CObjects, including something wrong (like calling a variable). This can be avoided (except for intentional misuse) by providing a header file that decribes the public features of a module. One could produce a tool (in Python, of course!) that produces such a header file with the prototypes of the original module code. It could be made so easy to use that except for an additional import call at the beginning, another module could use all functions and variables exactly as if they were statically linked. And then one could pack the whole interface for a module into a single CObject, which helps keeping a module's name space clean. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Dec 15 09:40:33 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA05171 for matrix-sig-people; Fri, 15 Dec 1995 09:40:33 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id JAA05166 for ; Fri, 15 Dec 1995 09:40:30 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id OAA14207; Fri, 15 Dec 1995 14:40:17 GMT From: "Jim Fulton, U.S. Geological Survey" Message-Id: <9512150940.ZM14205@dsdbqvarsa.er.usgs.gov> Date: Fri, 15 Dec 1995 09:40:16 -0500 In-Reply-To: hinsenk@ERE.UMontreal.CA (Hinsen Konrad) "Re: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath" (Dec 15, 9:05am) References: <199512151405.JAA23256@cyclone.ERE.UMontreal.CA> X-Phone: (703) 648-5622 X-Face: (;vf"lccr&V=y Subject: Re: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath > > > That is certainly preferable. However, I am not sure whether it > > is possible in all cases with the current implementation. Clearly > > there is not much provision for C modules that depend on each other. > > Your proposed CObjects solve one problem, but there may be more. > > There may be, but I haven't though of any yet. If you come up with any, I'd be > happy to try to come up with a solution to them too. :-] > > It seems that you can do everything necessary with these CObjects. After > all, you can make all non-static variables and functions known in this > way, Actually, with CObjects, all of the exported C objects can be static. So essentially, the only non-static variable in a module is the module initialization function. This is a plus when the module is statically linked, because you don't have to worry about global name colisions. You get the benefits of Python's namespace mechanisms even for your C code. :-) > and then you have published everything that other statically > linked C modules have access to. > > A potential problem is that another module can do whatever it wants > with the CObjects, including something wrong (like calling a variable). This is a problem with C in general. Of course ... > This can be avoided (except for intentional misuse) by providing > a header file that decribes the public features of a module. Absolutely. > One could produce a tool (in Python, of course!) that produces such > a header file with the prototypes of the original module code. > It could be made so easy to use that except for an additional import > call at the beginning, another module could use all functions and > variables exactly as if they were statically linked. And then one > could pack the whole interface for a module into a single CObject, > which helps keeping a module's name space clean. Yup, there are plenty of opportunities to build on and improve the basic mechanism. Alot of this depends on how complex the C interface being exported is. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Dec 15 14:45:56 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA01107 for matrix-sig-people; Fri, 15 Dec 1995 14:45:56 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id OAA01103 for ; Fri, 15 Dec 1995 14:45:45 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA20005; Fri, 15 Dec 95 11:45:39 PST Received: from kristen (localhost) by kristen.llnl.gov (5.x/SMI-SVR4) id AA22403; Fri, 15 Dec 1995 11:45:37 -0800 Message-Id: <30D1D061.5045@llnl.gov> Date: Fri, 15 Dec 1995 11:45:37 -0800 From: "Paul. Dubois" Organization: Lawrence Livermore National Laboratory X-Mailer: Mozilla 2.0b2 (X11; I; SunOS 5.4 sun4m) Mime-Version: 1.0 To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Re: Version 2.0 References: <199512131904.OAA20843@cyclone.ERE.UMontreal.CA> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk In a previous message I asked (stupidly) > b. Matrix(1,2,3) produces a Matrix_l, or at least that is what it > claims it is. Should be a Matrix_i? This question provoked a long response from Jim which taught me that I had simply made a mistake; I didn't know that a Python integer corresponded to a C long. If it does, Matrix_l is correct. Sorry. -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Dec 16 11:10:35 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA06319 for matrix-sig-people; Sat, 16 Dec 1995 11:10:35 -0500 Received: from harfang.CC.UMontreal.CA ([132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id LAA06314 for ; Sat, 16 Dec 1995 11:10:32 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id LAA22084 (8.6.11/IDA-1.6); Sat, 16 Dec 1995 11:09:40 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id LAA18202; Sat, 16 Dec 1995 11:09:39 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id LAA02631; Sat, 16 Dec 1995 11:09:38 -0500 Date: Sat, 16 Dec 1995 11:09:38 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199512161609.LAA02631@cyclone.ERE.UMontreal.CA> To: jfulton@usgs.gov CC: matrix-sig@python.org In-reply-to: <9512150940.ZM14205@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath Sender: owner-matrix-sig@python.org Precedence: bulk Yup, there are plenty of opportunities to build on and improve the basic mechanism. Alot of this depends on how complex the C interface being exported is. Variables and functions are all there is to export; anything else (typedefs, macros etc.) would have to be exported via header files even in the case of statically linked C programs. If something like this is to become an official feature of Python it would be nice to add a minimum of support to CObjects, even if that increases the code size too 300 bytes ;-) The aim should be to make this feature as easy to use as possible and reduce potential problems to a minimum. For example, one could have a standard typedef for a structure that contains the complete C interface to a module. Plus a standard special name for the variable that holds the C interface (perhaps "__C__"), and an access function that looks for this variable and retrieves the pointer. Then add conventions for header files, and the only difference between this mechanism and acess to a statically linked module is two lines of code: one for the import, and one to retrieve the C interface. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Dec 17 17:45:55 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA11257 for matrix-sig-people; Sun, 17 Dec 1995 17:45:55 -0500 Received: from ibp.ibp.fr (ibp.ibp.fr [132.227.60.30]) by python.org (8.6.12/8.6.12) with ESMTP id RAA11253 for ; Sun, 17 Dec 1995 17:45:51 -0500 Received: from laforia.ibp.fr (laforia.ibp.fr [132.227.60.10]) by ibp.ibp.fr (8.6.12/jtpda-5.0) with ESMTP id XAA12673 for ; Sun, 17 Dec 1995 23:45:21 +0100 Received: from lpia2.ibp.fr (singha.ibp.fr [132.227.201.18]) by laforia.ibp.fr (8.6.10/jtpda-5.0) with ESMTP id XAA14006 for ; Sun, 17 Dec 1995 23:45:19 +0100 From: Emmanuel.Viennet@laforia.ibp.fr (VIENNET Emmanuel 48.06.27.97 Equipe Gallinari) Received: from (viennet@localhost) by lpia2.ibp.fr (8.6.10/jtpda-5.0) id XAA15748 ; Sun, 17 Dec 1995 23:45:18 +0100 Date: Sun, 17 Dec 1995 23:45:18 +0100 Message-Id: <199512172245.XAA15748@lpia2.ibp.fr> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] bug: assignment from string Sender: owner-matrix-sig@python.org Precedence: bulk While playing with some matrices of char: ----------------------------------------------------------------------------- [~/softs/Python-1.3/Modules/]% python Python 1.3 (Dec 17 1995) [GCC 2.5.8] Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> import Matrix >>> >>> m1 = Matrix.fromString( '\001\002\003\004', 4, 'c' ) >>> >>> m2 = m1.reshape( 2, 2 ) >>> >>> m2[1,1] '\004' >>> >>> m2[1,1] = 'e' Segmentation fault (core dumped) [~/softs/Python-1.3/Modules/]% gdb ../python core GDB 4.13 (i486-slackware-linux), Copyright 1994 Free Software Foundation, Inc... Core was generated by `python'. #0 0x64fc4 in optimize_slices (dest_strides=0xbffff37c, dest_dimensions=0xbffff378, dest_nd=0xbffff374, src_strides=0xbffff370, src_dimensions=0xbffff36c, src_nd=0xbffff368, elsize=0xbffff364, copies=0xbffff360) at matrixobject.c:127 127 if (((*dest_strides)[*dest_nd-1] == *elsize) && ((*src_strides)[*src_nd-1] == *elsize)) { (gdb) where #0 0x64fc4 in optimize_slices (dest_strides=0xbffff37c, dest_dimensions=0xbffff378, dest_nd=0xbffff374, src_strides=0xbffff370, src_dimensions=0xbffff36c, src_nd=0xbffff368, elsize=0xbffff364, copies=0xbffff360) at matrixobject.c:127 #1 0x68949 in PyMatrix_CopyMatrix (dest=0x164080, src=0x164200) at matrixobject.c:262 #2 0x68a21 in PyMatrix_CopyObject (dest=0x164080, src_object=0x187b20) at matrixobject.c:278 #3 0x6a0a1 in matrix_ass_sub (self=0x164140, index=0x174600, op=0x187b20) at matrixobject.c:673 #4 0x3531f in assign_subscript (w=0x164140, key=0x174600, v=0x187b20) at ceval.c:2600 ----------------------------------------------------------------------------- The problem is with the string in the rhs: m2[1,1] = 'e' (m1[1,1] = 66 works). I suppose the bug is in matrix_ass_sub(), but I can't figure where. Also, I wonder why can't we specify all the dimensions as arguments to fromString(), avoiding to call reshape() ? Emmanuel ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Dec 18 15:55:05 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA01159 for matrix-sig-people; Mon, 18 Dec 1995 15:55:05 -0500 Received: from mama-bear (mama-bear.lcs.mit.edu [18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id PAA01154 for ; Mon, 18 Dec 1995 15:55:00 -0500 Received: from ling-ling.lcs.mit.edu.LCS.MIT.EDU by mama-bear (4.1/SLS-4.1.1) id AA02529; Mon, 18 Dec 95 15:54:50 EST Received: by ling-ling.lcs.mit.edu.LCS.MIT.EDU (NX5.67e/SMI-4.1) id AA12077; Mon, 18 Dec 95 15:56:36 -0500 Message-Id: <9512182056.AA12077@ling-ling.lcs.mit.edu.LCS.MIT.EDU> Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Mon, 18 Dec 95 15:56:36 -0500 To: Emmanuel.Viennet@laforia.ibp.fr (VIENNET Emmanuel 48.06.27.97 Equipe Gallinari) Subject: Re: [PYTHON MATRIX-SIG] bug: assignment from string Cc: matrix-sig@python.org References: <199512172245.XAA15748@lpia2.ibp.fr> Sender: owner-matrix-sig@python.org Precedence: bulk > (Core dump data ommitted) As a small point, these detailed listings of the results of a core dump are probably not at all useful to any member of this list except myself (to whom the are EXTREMELY helpful) so it might be a better idea to just send them directly to me and I'll fix them ASAP. However, I don't want to say anything that will discourage people from sending in bug reports. The other comments are great for the list. > The problem is with the string in the rhs: m2[1,1] = 'e' > (m1[1,1] = 66 works). > > I suppose the bug is in matrix_ass_sub(), but I can't figure where. There's actually something rather strange going on here, and while it shouldn't dump core, my current interpretation of strings says that this should raise an exception. Here's what's going on: 'hello' gets converted to a 5-length vector ['h', 'e', 'l', 'l', 'o'] 'e' gets converted to a 1-length vector ['e'] 66 is a 0-dimensional scalar So, m[1,1] = 'e' is trying to assign a 1-length vector to a 0-dimensional scalar, which is an exception. I can fix this by treating 1-length strings as 0-dimensional scalars if you think this is the appropriate choice. Note: I NEVER use arrays of strings personally, so I'm depending on the other people on this list to tell me how you want them to behave. I can't imagine a use for arrays of strings except for interfacing to existing libraries. > Also, I wonder why can't we specify all the dimensions as arguments to > fromString(), avoiding to call reshape() ? This is a simple oversight on my part. I'm not convinced that the current form of "fromString" is exactly the way it should finally be, and I'm reluctant to churn out C-code for an interface that might change. Nevertheless, I'll add a function to Matrix.py to fix this in the next release. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Dec 19 10:51:08 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA00197 for matrix-sig-people; Tue, 19 Dec 1995 10:51:08 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id KAA00183 for ; Tue, 19 Dec 1995 10:49:30 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id OAA19949; Tue, 19 Dec 1995 14:50:29 GMT From: "Jim Fulton, U.S. Geological Survey" Message-Id: <9512190950.ZM19947@dsdbqvarsa.er.usgs.gov> Date: Tue, 19 Dec 1995 09:50:26 -0500 In-Reply-To: James Hugunin "Re: [PYTHON MATRIX-SIG] bug: assignment from string" (Dec 18, 3:56pm) References: <199512172245.XAA15748@lpia2.ibp.fr> <9512182056.AA12077@ling-ling.lcs.mit.edu.LCS.MIT.EDU> X-Phone: (703) 648-5622 X-Face: (;vf"lccr&V=y, Emmanuel.Viennet@laforia.ibp.fr (VIENNET Emmanuel 48.06.27.97 Equipe Gallinari) Subject: Re: [PYTHON MATRIX-SIG] bug: assignment from string Cc: matrix-sig@python.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-matrix-sig@python.org Precedence: bulk On Dec 18, 3:56pm, James Hugunin wrote: > Subject: Re: [PYTHON MATRIX-SIG] bug: assignment from string > (snip) > The other comments are great for the list. > > > The problem is with the string in the rhs: m2[1,1] = 'e' > > (m1[1,1] = 66 works). > > > > I suppose the bug is in matrix_ass_sub(), but I can't figure where. > > There's actually something rather strange going on here, and while it > shouldn't dump core, my current interpretation of strings says that this > should raise an exception. >> > Here's what's going on: > > 'hello' gets converted to a 5-length vector ['h', 'e', 'l', 'l', 'o'] > 'e' gets converted to a 1-length vector ['e'] > 66 is a 0-dimensional scalar > > So, m[1,1] = 'e' is trying to assign a 1-length vector to a 0-dimensional > scalar, which is an exception. > > I can fix this by treating 1-length strings as 0-dimensional scalars if you > think this is the appropriate choice. Yes. This is necessary because Python does not have a character type. 1-length strings must be used as "characters". Witness the fact that 'hello'[1] wants to return a character, but returns a string instead. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Dec 20 03:34:14 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id DAA02972 for matrix-sig-people; Wed, 20 Dec 1995 03:34:14 -0500 Received: from atr-sw.atr-sw.atr.co.jp ([133.186.20.117]) by python.org (8.6.12/8.6.12) with ESMTP id DAA02968 for ; Wed, 20 Dec 1995 03:34:00 -0500 Received: from ciris21.atr-sw.atr.co.jp (ciris21.atr-sw.atr.co.jp [133.186.20.41]) by atr-sw.atr-sw.atr.co.jp (8.6.11+2.5Wb2/3.4Wbeta3/95031616ATR-SW) with ESMTP id RAA09297 for ; Wed, 20 Dec 1995 17:32:42 +0900 Received: from atr-sw by ciris21.atr-sw.atr.co.jp (940816.SGI.8.6.9/6.4J.6) id RAA18484; Wed, 20 Dec 1995 17:32:40 +0900 Message-Id: <199512200832.RAA18484@ciris21.atr-sw.atr.co.jp> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] __doc__ string problems? Date: Wed, 20 Dec 1995 17:32:39 +0900 From: "Perry A. Stoll" Sender: owner-matrix-sig@python.org Precedence: bulk Matrix-sig'ers, Sorry to disturb everyone. Is anyone else having a problem using __ doc__ strings since installing the Matrix module? Before rebuilding, I thought I'd check with others on this list. Could it be due to some of the changes in the distribution? As a short example, the following does nothing in my python with the Matrix library but works fine on the Mac: def foo(x): """Some documentation string.""" return x foo.__doc__ Ideas? -Perry Stoll Perry A. Stoll Research Associate, CSRL Advanced Telecommunications Research 2-2 Hikaridai, Seika-cho Soraku-gun, Kyoto 619-02 JAPAN VOICE: +81-774-95-1217 FINGER: stoll@atrwide.atr.co.jp FAX : +81-774 95-1208 EMAIL: stoll@atr-sw.atr.co.jp PGP 2.6 Key fingerprint = AF 56 5C D8 5F 78 BA FD 21 6E 2A 68 C4 55 9E B0 ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Dec 21 10:15:17 1995 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA10630 for matrix-sig-people; Thu, 21 Dec 1995 10:15:17 -0500 Received: from acds21 (acds21.physik.rwth-aachen.de [137.226.30.31]) by python.org (8.6.12/8.6.12) with SMTP id KAA10615 for ; Thu, 21 Dec 1995 10:11:27 -0500 Received: by acds21; (5.65/1.1.8.2/01Dec95-8.2MPM) id AA06882; Thu, 21 Dec 1995 16:02:53 +0100 Date: Thu, 21 Dec 1995 16:02:53 +0100 From: Konrad Hinsen Message-Id: <9512211502.AA06882@acds21> To: stoll@atr-sw.atr.co.jp Cc: matrix-sig@python.org In-Reply-To: <199512200832.RAA18484@ciris21.atr-sw.atr.co.jp> (stoll@atr-sw.atr.co.jp) Subject: Re: [PYTHON MATRIX-SIG] __doc__ string problems? Sender: owner-matrix-sig@python.org Precedence: bulk Sorry to disturb everyone. Is anyone else having a problem using __ doc__ strings since installing the Matrix module? Before rebuilding, I My fault! It has to do with the power operator patch. I have already sent the fix to Guido, but obviously forgotten that other people on the Matrix SIG might need it. You have to add one line in the function get_docstring() in Python/compile.c; I'll include the full function: static object * get_docstring(n) node *n; { int i; switch (TYPE(n)) { case suite: if (NCH(n) == 1) return get_docstring(CHILD(n, 0)); else { for (i = 0; i < NCH(n); i++) { node *ch = CHILD(n, i); if (TYPE(ch) == stmt) return get_docstring(ch); } } break; case file_input: for (i = 0; i < NCH(n); i++) { node *ch = CHILD(n, i); if (TYPE(ch) == stmt) return get_docstring(ch); } break; case stmt: case simple_stmt: case small_stmt: return get_docstring(CHILD(n, 0)); case expr_stmt: case testlist: case test: case and_test: case not_test: case comparison: case expr: case xor_expr: case and_expr: case shift_expr: case arith_expr: case term: case factor: case power: if (NCH(n) == 1) return get_docstring(CHILD(n, 0)); break; case atom: if (TYPE(CHILD(n, 0)) == STRING) return parsestrplus(n); break; } return NULL; } The new line is "case power:". Briefly, I had to introduce a new node type to give the power operator a higher priority than multiplication, and get_docstring() needs a complete list of node types in its case statements in order to work correctly. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de Chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 9 21:15:13 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id VAA00491 for matrix-sig-people; Tue, 9 Jan 1996 21:15:13 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id VAA00485 for ; Tue, 9 Jan 1996 21:15:11 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA00365; Tue, 9 Jan 96 11:40:39 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) id LAA22547; Tue, 9 Jan 1996 11:50:12 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199601091650.LAA22547@maigret> Subject: [PYTHON MATRIX-SIG] hello? To: matrix-sig@python.org Date: Tue, 9 Jan 1996 11:50:12 -0500 (EST) Cc: guido@CNRI.Reston.VA.US (Guido van Rossum) X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 163 Sender: owner-matrix-sig@python.org Precedence: bulk I got a bounce from the list the last time I tried. I'll try again. Content part of mesage: How is the doc for the matrix module progressing? Anyone? --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 12 17:49:44 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA14676 for matrix-sig-people; Fri, 12 Jan 1996 17:49:44 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id RAA14669 for ; Fri, 12 Jan 1996 17:49:31 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id RAA10558 (8.6.11/IDA-1.6 for ); Fri, 12 Jan 1996 17:47:44 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA11333; Fri, 12 Jan 1996 17:47:44 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA02079; Fri, 12 Jan 1996 17:47:43 -0500 Date: Fri, 12 Jan 1996 17:47:43 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601122247.RAA02079@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Fast evaluation of numerical expressions Sender: owner-matrix-sig@python.org Precedence: bulk The following is not directly related to matrices, but I suppose most of us are also interested in more general numerical applications of Python. A frequent problem with Python code for numerical applications is speed, and the matrix module will help only in those cases where an algorithm can be expressed in terms of matrix operations on sufficiently large matrices. Many applications also require the evaluation of many "simple" expressions, i.e. functions of scalar values or small arrays. I have thought about possibilities to speed up such calculations without writing a special C extension module. One idea that looks promising to me is to write a special compiler for numerical expressions. This compiler would generate code for a simple (and fast) stack-based expression evaluator that would be written in C; the compiler itself could be written in Python. The expression evaluator would not have to deal with type checks etc., and would know the standard mathematical functions. It could therefore be a lot faster than the general expression evaluator in Python. In addition, the expression compiler could be followed by an optimizer that takes care of constant subexpressions and other things. The compiler could most easily be constructed by defining a new type "compiled expression" and define the usual functions on that type. Then a user could write something like x = variable(types.FloatType, 0) n = variable(types.IntType, 1) y = variable(types.FloatType, 2) compiled_function = sin(x**n+y) print compiled_function(2.5, 3, 0.3) The function "variable" would return an object of type "compiled expression" that pushes the n-th argument to the evaluation stack. All the other operations would just append their code to whatever code they get in their arguments. Variable types could be restricted to integers, floats, complex numbers, and arrays of these types. To compensate the lack of control structures it would probably be necessary to introduce a conditional function; loops would be replaced by array operations. Does this seem like a reasonable idea? And would anyone be willing to collaborate on an implementation? Or does anyone have a better idea to achieve fast expression evaluation? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 12 18:55:08 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA15092 for matrix-sig-people; Fri, 12 Jan 1996 18:55:08 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id SAA15088 for ; Fri, 12 Jan 1996 18:55:04 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa13934; 12 Jan 96 18:46 EST Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id SAA23753; Fri, 12 Jan 1996 18:45:32 -0500 Message-Id: <199601122345.SAA23753@monty> To: Hinsen Konrad cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Fast evaluation of numerical expressions In-reply-to: Your message of "Fri, 12 Jan 1996 17:47:43 EST." <199601122247.RAA02079@cyclone.ERE.UMontreal.CA> References: <199601122247.RAA02079@cyclone.ERE.UMontreal.CA> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Fri, 12 Jan 1996 18:45:32 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk This sounds like a useful idea. Please proceed. (I particularly like the idea of using a 'variable' data type which can be subjected to normal operations, yielding a data structure representing the expression structure. This could be used in a symbolic algebra package, as well...) --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Jan 13 09:35:39 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA17050 for matrix-sig-people; Sat, 13 Jan 1996 09:35:39 -0500 Received: from harfang.CC.UMontreal.CA ([132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id JAA17046 for ; Sat, 13 Jan 1996 09:35:35 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id JAA19441 (8.6.11/IDA-1.6); Sat, 13 Jan 1996 09:32:57 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA27867; Sat, 13 Jan 1996 09:32:57 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA07188; Sat, 13 Jan 1996 09:32:56 -0500 Date: Sat, 13 Jan 1996 09:32:56 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601131432.JAA07188@cyclone.ERE.UMontreal.CA> To: guido@CNRI.Reston.VA.US CC: matrix-sig@python.org In-reply-to: <199601122345.SAA23753@monty> (message from Guido van Rossum on Fri, 12 Jan 1996 18:45:32 -0500) Subject: Re: [PYTHON MATRIX-SIG] Fast evaluation of numerical expressions Sender: owner-matrix-sig@python.org Precedence: bulk This sounds like a useful idea. Please proceed. As soon as I find some spare time... (I particularly like the idea of using a 'variable' data type which can be subjected to normal operations, yielding a data structure representing the expression structure. This could be used in a symbolic algebra package, as well...) Indeed. Symbolic algebra in Python would be neat... A nice feature of this approach is that existing code can be compiled without modification. Any Python function foo(x) that consists of a linear sequence of statements (no conditionals, no loops) can be compiled as simply as compiled_foo = foo(variable(types.FloatType)) The tricky part will be the optimizer. I'd like to have at least extraction of common subexpressions, which occur rather often in numerical expressions. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 16 09:50:25 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA03572 for matrix-sig-people; Tue, 16 Jan 1996 09:50:25 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id JAA03565 for ; Tue, 16 Jan 1996 09:50:15 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA16343; Tue, 16 Jan 96 09:50:00 EST Date: Tue, 16 Jan 96 09:50:00 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601161450.AA16343@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging Sender: owner-matrix-sig@python.org Precedence: bulk Hi all. I just got back in town from an extended Christmas vacation (I also got married, so I had a good excuse for being gone so long). Now that I'm back in the office again, I've been thinking that it's about time that the matrix object made it out into beta release. There are a few naming conventions and packaging decisions to be made before this happens. Below are the conventions that I propose to use. Once this object is released in beta form I have no intention of changing the naming conventions without a REALLY good reason, so please let me know your opinions now. Every name in quotes should be considered a proposed name open for discussion. ----- The "NumericPython" package contains the following major pieces. 1) Konrad's numeric patches to the python core (incorporated into the working version of python by Guido) will be required and included with the distribution. These will be the only patches to the python core required. 2) One C module that must be statically linked called "multiarraymodule.c" Having this particular module statically linked will eliminate the need for getting the CObject proposal working before release. Use "PyArray_" as the name of the Matrix Object. This is a simple renaming of the existing "PyMatrix_". Use "array(sequence, typecode='d')" as the default constructor for this new C type. This PyArray C type will not implement automatic type-coercion (unlike the current implementation). The reason for this is that I have decided type-coercion is a major pain for arrays of floats (as opposed to doubles) and I happen to use arrays of floats for most of my work. If somebody can give me a good suggestion for how to keep automatic type-coercion and have Matrix_f(1,2,3)*1.2 == Matrix_f(1.2,2.4,3.6) then I might consider changing this decision. See later note on Array.py for an alternative. The include files "arrayobject.h", and "ofuncobject.h" will provide the needed C interface to the array and optimized function objects. 3) Two dynamically linkable modules called "umathmodule.c", and "ieee_umathmodule.c" These will both provide "universal" math support, providing the basic functions of mathmodule, plus things like "greater" and "booleanOr" for matrices, floats, complex, ints, and generic python objects. The basic "umath" will cause python exceptions in the event of an overflow or divide-by-zero, etc. (this means it will be slow). "ieee_umath" will not check the arguments, or the results of its computations, and this should result in standard IEEE overflow, and NaN values occuring in arrays as well as unpredictable modulo effects for integer overflows. (this will be the fast version). 4) Two python objects, "Array.py" and "Matrix.py" Array is essentially a python binding around the underlying C type, and this will also provide for automatic type-coercion and will generally assume that it is only working with arrays of type long, double, and complex double (the three types of arrays that are equivalent to python objects). In my initial tests of this object on LLNL's simple benchmark, I found that the performance was only 5% slower than using the C object directly. Matrix will inherit almost everything from Array, however it will be limited to 2 or fewer dimensions, and m1*m2 where m1 and m2 are matrices will perform matrix style multiplication. If the linear algebra people would like, other changes can be made (ie. ~m1 == m1.transpose(), ...). Based on the experiments with Array, the performance penalty for this approach should be minimal. In order to support these python objects (and others like them), two special data members will be added, "__array__", and "__object__". If an object has the member "__array__", then the C functions that handle matrices will attempt to retrieve the matrix from this member when passed in a python object. In addition, they will attempt to convert their result to an object of class "__object__" upon return. This means that umath.sin(Array([0, pi/2, pi])) == Array([0.,1.,0.]). Hopefully, this convention will allow these python objects to coexist well with any numeric libraries. 5) A standard library "Numeric.py" which will be the standard way of importing multiarray, Array, Matrix, umath, etc. It will include the inverted trig functions ("sec", "csc", "arcsec", ...) as well as most of the standard functions currently in Matrix.py 6) Great documentation and tutorials (hopefully written by Paul DuBois). 7) A standard test suite. 8) A new version of pickle.py which is aware of matrix objects. This will be suggested as a replacement for the existing pickle.py. Matrix objects are pickled using a binary format that is endian-aware, so pickling a matrix object is a very efficient and portable way of both storing them and sending them around the network. 9?) A "numericmodule.c" which contains reasonably efficient fft, matrix inversion, convolution, random numbers, eigenvalues and filtering functions stolen from existing C code so that the package can be viewed as a "complete" numerical computing system? Well, that's all I can think of for now, let me know your opinions before I start my final burst of coding and get this thing polished up and released. Planned schedule: Comments on this proposal until 1/22 Final alpha release 0.30 1/26 Massive use of release 0.30, and lots of good bug reports from users Bugfixed alpha release 0.31 2/2 More testing, and hopefully final forms of documentation and tutorials First beta release 1.0beta1 announced to general newsgroup 2/12 -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 16 10:22:22 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA03718 for matrix-sig-people; Tue, 16 Jan 1996 10:22:22 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id KAA03714 for ; Tue, 16 Jan 1996 10:22:19 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa09222; 16 Jan 96 10:20 EST Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id KAA05252; Tue, 16 Jan 1996 10:19:28 -0500 Message-Id: <199601161519.KAA05252@monty> To: James Hugunin cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging In-reply-to: Your message of "Tue, 16 Jan 1996 09:50:00 EST." <9601161450.AA16343@baribal.LCS.MIT.EDU.LCS.MIT.EDU> References: <9601161450.AA16343@baribal.LCS.MIT.EDU.LCS.MIT.EDU> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Tue, 16 Jan 1996 10:19:28 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk Looks good! No comments here, except: > 1) Konrad's numeric patches to the python core (incorporated into the > working version of python by Guido) will be required and included with > the distribution. These will be the only patches to the python core > required. I changed one thing in Konrad's final patches: I took out support for 'i' and 'I' to indicate imaginary constants -- you have to use 'j' or 'J'. I don't like to offer gratuitous choices (the lower/upper case equivalency is a C legacy that I keep for consistency with other Python numerical constants) and I expect that engineers will whine bitterly if I chose 'i' while mathematicians will accept 'j' without complaints... (Note that unless you never read code written by someone else, you must learn both alternatives if they are both present in the language. This is my reason for wanting to offer only one option -- not the savings in instruction size or cycles.) > 2) One C module that must be statically linked called "multiarraymodule.c" > > Having this particular module statically linked will eliminate the > need for getting the CObject proposal working before release. I do have a working version of CObject which you can use if you like. It's up to you. --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 16 11:32:49 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA04064 for matrix-sig-people; Tue, 16 Jan 1996 11:32:49 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id LAA04059 for ; Tue, 16 Jan 1996 11:32:16 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id QAA04107; Tue, 16 Jan 1996 16:31:58 GMT From: "Jim Fulton, U.S. Geological Survey" Message-Id: <9601161131.ZM4105@dsdbqvarsa.er.usgs.gov> Date: Tue, 16 Jan 1996 11:31:56 -0500 In-Reply-To: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) "[PYTHON MATRIX-SIG] Final matrix object renaming and packaging" (Jan 16, 9:50am) References: <9601161450.AA16343@baribal.LCS.MIT.EDU.LCS.MIT.EDU> X-Phone: (703) 648-5622 X-Face: (;vf"lccr&V=y Subject: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging > > Hi all. I just got back in town from an extended Christmas vacation > (I also got married, so I had a good excuse for being gone so long). Congradulations! > Now that I'm back in the office again, I've been thinking that it's > about time that the matrix object made it out into beta release. > > There are a few naming conventions and packaging decisions to be made > before this happens. Below are the conventions that I propose to use. > Once this object is released in beta form I have no intention of > changing the naming conventions without a REALLY good reason, so > please let me know your opinions now. Fair enough. > > Every name in quotes should be considered a proposed name open for > discussion. > > ----- > > The "NumericPython" package contains the following major pieces. > > 1) Konrad's numeric patches to the python core (incorporated into the > working version of python by Guido) will be required and included with > the distribution. These will be the only patches to the python core > required. > > > 2) One C module that must be statically linked called "multiarraymodule.c" By statically linked, I assume you mean statically linked into the interpreter. Right? > Having this particular module statically linked will eliminate the > need for getting the CObject proposal working before release. As Guido said, the CObject proposal is working. I'll send it to you in a separate note. I finished the (very small) CObject implementation very soon after the workshop because I feel it is important to use it for the Matrix (Numeric) module. I feel strongly that the Matrix module should export it's C interface using CObjects so that modules using matrices to not require that any of the matrix software be statically linked. In my distribution of Python, I statically link as little as possible to keep the interpreter and the interpreter start-up time small. I plan to modify FIDL to use the CObject-exported interface. I'd be happy to assist you with this if you wish. > > Use "PyArray_" as the name of the Matrix Object. This is a simple > renaming of the existing "PyMatrix_". > > Use "array(sequence, typecode='d')" as the default > constructor for this new C type. Is this a replacement for the existing array type? > This PyArray C type will not implement automatic type-coercion (unlike the > current implementation). The reason for this is that I have decided > type-coercion is a major pain for arrays of floats (as opposed to > doubles) and I happen to use arrays of floats for most of my work. If > somebody can give me a good suggestion for how to keep automatic > type-coercion and have Matrix_f(1,2,3)*1.2 == Matrix_f(1.2,2.4,3.6) > then I might consider changing this decision. See later note on > Array.py for an alternative. > > The include files "arrayobject.h", and "ofuncobject.h" will provide > the needed C interface to the array and optimized function objects. > > > 3) Two dynamically linkable modules called "umathmodule.c", and "ieee_umathmodule.c" > These will both provide "universal" math support, providing the basic > functions of mathmodule, plus things like "greater" and > "booleanOr" for matrices, floats, complex, ints, and generic python > objects. > > The basic "umath" will cause python exceptions in the event of an > overflow or divide-by-zero, etc. (this means it will be slow). > "ieee_umath" will not check the arguments, or the results of its > computations, and this should result in standard IEEE overflow, and > NaN values occuring in arrays as well as unpredictable modulo effects > for integer overflows. (this will be the fast version). > > > 4) Two python objects, "Array.py" and "Matrix.py" Are these imported by Numeric, or would the user be importing these? Is the current built-in array module going away? If not, then there is a name conflict on case-insensitive file systems. I like the idea of having the user import Numeric and then use the Numeric.Matrix_d(...) or Numeric.Array_f(...) rather than than importing Matrix and Array. Is there any reason for the user to import Array directly? I am strongly opposed to using "Array" unless the "array" module goes away. > > Array is essentially a python binding around the underlying C type, > and this will also provide for automatic type-coercion and will > generally assume that it is only working with arrays of type long, > double, and complex double (the three types of arrays that are > equivalent to python objects). In my initial tests of this object on > LLNL's simple benchmark, I found that the performance was only 5% > slower than using the C object directly. > > Matrix will inherit almost everything from Array, however it will be > limited to 2 or fewer dimensions, and m1*m2 where m1 and m2 are > matrices will perform matrix style multiplication. If the linear > algebra people would like, other changes can be made (ie. ~m1 == > m1.transpose(), ...). Based on the experiments with Array, the > performance penalty for this approach should be minimal. > > In order to support these python objects (and others like them), two > special data members will be added, "__array__", and "__object__". If > an object has the member "__array__", then the C functions that handle > matrices will attempt to retrieve the matrix from this member when > passed in a python object. Are we taking about python members or C structure members? Is the __array__ member supposed to be the C pointer to a block of memory? > In addition, they will attempt to convert > their result to an object of class "__object__" upon return. Class __object__? So __object__ is a pointer to a Python class object? Or is it a type code? How does the function manage to do this? Presumably the function needs to call some sort of constructor function. How does it do this? Is this explained in the header files? > This > means that umath.sin(Array([0, pi/2, pi])) == Array([0.,1.,0.]). OK. This makes sense > Hopefully, this convention will allow these python objects to coexist > well with any numeric libraries. Could you provide some additional details? > 5) A standard library "Numeric.py" which will be the standard way of > importing multiarray, Array, Matrix, umath, etc. It will include the > inverted trig functions ("sec", "csc", "arcsec", ...) as well as most > of the standard functions currently in Matrix.py So someone who wants to create a matrix object will do something like: import Numeric m=Numeric.Matrix_d(...) Right? I like it. :-) Did you keep a "Matrix" constructor that has a type code as an argument? > > > 6) Great documentation and tutorials (hopefully written by Paul > DuBois). Wat cool. Will we also get doc strings? > > 7) A standard test suite. > > > 8) A new version of pickle.py which is aware of matrix objects. This > will be suggested as a replacement for the existing pickle.py. Matrix > objects are pickled using a binary format that is endian-aware, so > pickling a matrix object is a very efficient and portable way of both > storing them and sending them around the network. > > > 9?) A "numericmodule.c" which contains reasonably efficient fft, > matrix inversion, convolution, random numbers, eigenvalues and > filtering functions stolen from existing C code so that the package > can be viewed as a "complete" numerical computing system? > > > Well, that's all I can think of for now, let me know your opinions > before I start my final burst of coding and get this thing polished up > and released. > > Planned schedule: > > Comments on this proposal until 1/22 > Final alpha release 0.30 1/26 > Massive use of release 0.30, and lots of good bug reports from users > Bugfixed alpha release 0.31 2/2 > More testing, and hopefully final forms of documentation and tutorials > First beta release 1.0beta1 announced to general newsgroup 2/12 Thanks for all the great work. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 16 12:12:24 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA04282 for matrix-sig-people; Tue, 16 Jan 1996 12:12:24 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id MAA04278 for ; Tue, 16 Jan 1996 12:12:21 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id MAA08810 (8.6.11/IDA-1.6); Tue, 16 Jan 1996 12:10:32 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA09649; Tue, 16 Jan 1996 12:10:30 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA07614; Tue, 16 Jan 1996 12:10:28 -0500 Date: Tue, 16 Jan 1996 12:10:28 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601161710.MAA07614@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: matrix-sig@python.org In-reply-to: <9601161450.AA16343@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging Sender: owner-matrix-sig@python.org Precedence: bulk Hi all. I just got back in town from an extended Christmas vacation (I also got married, so I had a good excuse for being gone so long). Congratulations! 1) Konrad's numeric patches to the python core (incorporated into the working version of python by Guido) will be required and included with the distribution. These will be the only patches to the python core required. There has been a slight modification in the meantime (there will be only 'j' for imaginary constants, not 'i') and two corrections that perhaps not everybody has received. I'll prepare a new set of patch files for the beta release. This PyArray C type will not implement automatic type-coercion (unlike the current implementation). The reason for this is that I have decided type-coercion is a major pain for arrays of floats (as opposed to doubles) and I happen to use arrays of floats for most of my work. If somebody can give me a good suggestion for how to keep automatic type-coercion and have Matrix_f(1,2,3)*1.2 == Matrix_f(1.2,2.4,3.6) then I might consider changing this decision. See later note on Array.py for an alternative. One way to solve your "float" problem would be to write Matrix_f(1,2,3)*Matrix_f(1.2), i.e. cast 1.2 explicitly to float. I don't see how you can avoid that cast anyway; without type coercion, Matrix_f(1,2,3)*1.2 would be an error. I'd prefer to have type coercion everywhere for consistency. If the "high-level" classes Array and Matrix are really fast enough (about which I have doubts, see below), then it doesn't matter that much, but in general I'd give top priority to consistency and put the burden of more complicated coding on those who want top speed (which of course must be possible). 3) Two dynamically linkable modules called "umathmodule.c", and "ieee_umathmodule.c" I don't think that using "ieee" in the name is a good idea. After all, there is no guarantee that this will really use IEEE arithmetic (a Vax, for example, doesn't have IEEE arithmetic hardware). Why not just "fast_umath"? 4) Two python objects, "Array.py" and "Matrix.py" Array is essentially a python binding around the underlying C type, and this will also provide for automatic type-coercion and will generally assume that it is only working with arrays of type long, double, and complex double (the three types of arrays that are equivalent to python objects). In my initial tests of this object on LLNL's simple benchmark, I found that the performance was only 5% slower than using the C object directly. Nevertheless the Python wrapper could cause a significant overhead for small arrays, which will also be used heavily. For example, I am planning to change my vector class (3d vectors for geometry etc.) to use an array of length 3 as its internal representation. Do you have any data on the speed penalty for such small arrays? Matrix will inherit almost everything from Array, however it will be limited to 2 or fewer dimensions, and m1*m2 where m1 and m2 are matrices will perform matrix style multiplication. If the linear algebra people would like, other changes can be made (ie. ~m1 == m1.transpose(), ...). Based on the experiments with Array, the performance penalty for this approach should be minimal. I suppose some discussion will be necessary about the details of this class, which until now didn't exist. Given that many people seem to prefer a distinction between row and column vectors, it would be a good idea to include separate constructors for them. 5) A standard library "Numeric.py" which will be the standard way of importing multiarray, Array, Matrix, umath, etc. It will include the inverted trig functions ("sec", "csc", "arcsec", ...) as well as most of the standard functions currently in Matrix.py I am not sure it is a good idea to have such an all-encompassing module. It would contain a rather arbitrary selection of things, i.e. those which exist at the time it is introduced. I expect that other numerical modules will appear later. So it would be better to group things according to functions or applications. 9?) A "numericmodule.c" which contains reasonably efficient fft, matrix inversion, convolution, random numbers, eigenvalues and filtering functions stolen from existing C code so that the package can be viewed as a "complete" numerical computing system? Again this should be several modules: linear algebra, random numbers, fft/convolution. We won't achieve anything "complete" anyway, so let's not pretend we do. Otherwise, your proposal sounds fine to me. Konrad. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 16 12:21:33 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA04348 for matrix-sig-people; Tue, 16 Jan 1996 12:21:33 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id MAA04327 for ; Tue, 16 Jan 1996 12:19:09 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id MAA09094 (8.6.11/IDA-1.6); Tue, 16 Jan 1996 12:17:45 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA11738; Tue, 16 Jan 1996 12:17:42 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA08000; Tue, 16 Jan 1996 12:17:41 -0500 Date: Tue, 16 Jan 1996 12:17:41 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601161717.MAA08000@cyclone.ERE.UMontreal.CA> To: jfulton@usgs.gov CC: jjh@Goldilocks.LCS.MIT.EDU, matrix-sig@python.org In-reply-to: <9601161131.ZM4105@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging Sender: owner-matrix-sig@python.org Precedence: bulk I feel strongly that the Matrix module should export it's C interface using CObjects so that modules using matrices to not require that any of the matrix software be statically linked. In my distribution of I agree in principle, but that doesn't have to happen in the first beta release. Are these imported by Numeric, or would the user be importing these? Is the current built-in array module going away? If not, then there is a name conflict on case-insensitive file systems. I like the idea Why? The current "array" in a C module, so "import Array" will never load it, independent of the file name conventions. Which means that "import Array" will always obtain an external module, and as long as ours is the only one, it hardly matters if it is called "Array.py" or "array.py". importing Matrix and Array. Is there any reason for the user to import Array directly? I am strongly opposed to using "Array" unless Certainly, e.g. if the user doesn't want anything else. Besides, as I pointed out in my last mail, a general "Numeric" module is not such a terrific idea. So someone who wants to create a matrix object will do something like: import Numeric m=Numeric.Matrix_d(...) Speaking about constructors... I still prefer (strongly) more meaningful names, like IntegerMatrix. Noone should be forced to learn these silly type codes for standard applications. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 16 12:39:34 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA04458 for matrix-sig-people; Tue, 16 Jan 1996 12:39:34 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id MAA04454 for ; Tue, 16 Jan 1996 12:39:30 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA19446; Tue, 16 Jan 96 12:39:17 EST Date: Tue, 16 Jan 96 12:39:17 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601161739.AA19446@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: jfulton@usgs.gov Cc: matrix-sig@python.org In-Reply-To: <9601161131.ZM4105@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging Sender: owner-matrix-sig@python.org Precedence: bulk From: "Jim Fulton, U.S. Geological Survey" > 2) One C module that must be statically linked called "multiarraymodule.c" By statically linked, I assume you mean statically linked into the interpreter. Right? > Having this particular module statically linked will eliminate the > need for getting the CObject proposal working before release. As Guido said, the CObject proposal is working. I'll send it to you in a separate note. I finished the (very small) CObject implementation very soon after the workshop because I feel it is important to use it for the Matrix (Numeric) module. I feel strongly that the Matrix module should export it's C interface using CObjects so that modules using matrices to not require that any of the matrix software be statically linked. In my distribution of Python, I statically link as little as possible to keep the interpreter and the interpreter start-up time small. I plan to modify FIDL to use the CObject-exported interface. I'd be happy to assist you with this if you wish. I guess we run on faster networks at MIT, I never bother with dynamic linking, and find that 4MB binaries launch as fast as I wish. I'd be more than happy to have somebody else implement the CObject interface, but I just don't see the time it would take (including getting myself up to speed on the vagaries of dynamic linking) is worthwhile. If you (Jim Fulton) want to try to add this interface, I'd be happy to help. > > Use "PyArray_" as the name of the Matrix Object. This is a simple > renaming of the existing "PyMatrix_". > > Use "array(sequence, typecode='d')" as the default > constructor for this new C type. Is this a replacement for the existing array type? I decided it was not worth the huge set of compromises that would have been necessary to make the matrix/array object truly compatible with the existing array object. Still, the right name for this object really is array. There's no problem with using PyArray as the C name for the object because the existing array object does not export an interface. Also, there's no problem with using the name "array" as a constructor because avoiding these sorts of naming conflicts is why python has a module system in the first place. No existing code that imports "arraymodule" will be broken, but hopefully people in the future will start using the new multiarraymodule for the same tasks. > 4) Two python objects, "Array.py" and "Matrix.py" Are these imported by Numeric, or would the user be importing these? I plan to have everything in the basic distribution imported into a flat name space under the Numeric module. However, there's nothing to stop people from importing these independently if they wish. Is the current built-in array module going away? If not, then there is a name conflict on case-insensitive file systems. The array module is not going away. I'm confused about the problem of case-insensitive file systems, what do they do with tkintermodule and TkInter? If this is in fact an issue, then "Array.py" can be changed to "UserArray.py" in the spirit of UserList, etc. I like the idea of having the user import Numeric and then use the Numeric.Matrix_d(...) or Numeric.Array_f(...) rather than than importing Matrix and Array. Is there any reason for the user to import Array directly? I am strongly opposed to using "Array" unless the "array" module goes away. I agree with all this (except the last line, which I'm willing to conceed after you explain my tkinter question). > In order to support these python objects (and others like them), two > special data members will be added, "__array__", and "__object__". If > an object has the member "__array__", then the C functions that handle > matrices will attempt to retrieve the matrix from this member when > passed in a python object. Are we taking about python members or C structure members? Is the __array__ member supposed to be the C pointer to a block of memory? The __array__ member is a member of a python object which is expected to contain a python object of type array (the type created in C that this whole thing is based on). > In addition, they will attempt to convert > their result to an object of class "__object__" upon return. Class __object__? So __object__ is a pointer to a Python class object? This is still a python member. In python what it would do is call m.__object__(new_array). I assume that a similar thing can be done in C (I haven't implemented this in C yet). > This > means that umath.sin(Array([0, pi/2, pi])) == Array([0.,1.,0.]). OK. This makes sense Remember that Array is a python object here, that's the trick I'm trying to make work out. > Hopefully, this convention will allow these python objects to coexist > well with any numeric libraries. Could you provide some additional details? Here's a bit of code for a unary function expecting a single PyArray argument of type "double" of two dimensions: PyObject *op; PyArrayObject *ap, *rp; TRY(PyArg_ParseTuple(args, "O", &op)); TRY(ap = PyArray_ContiguousFromObject(op, PyArray_DOUBLE, 2, 2)); // Do something with ap to get rp Py_DECREF(ap); return PyArray_Return(rp, op); With the exception of the second argument to PyArray_Return, this is the current way of writing such a chunk of code. PyArray_ContiguousFromObject will convert any python sequence type to an array of the appropriate type and dimensions if possible. If the argument is already an array of the appropriate type and dimensions, then that array will be increfed and returned (unless its data points to a discontiguous chunk of memory in which case it will be copied into a new array with contiguous memory). The new feature that I want to add to this function is that if its argument is a python object with the attribute "__array__", then this function wil return the PyArrayObject contained in that attribute (if this is indeed the case). PyArray_Return is used because some operations wind up producing a 0-dimensional array. These will be converted to the appropriate python scalars on return. The new feature that I want to add here is that if the second argument has a "__class__" attribute, then the constructor for that class will be used to return a new python object with the returned PyArrayObject in its "__array__" attribute. This is the simplest method I could come up with to get my "sin(Array())" example to work. > > 6) Great documentation and tutorials (hopefully written by Paul > DuBois). Wat cool. Will we also get doc strings? doc strings are already done (probably could use some polishing, but... -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 16 12:54:46 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA04588 for matrix-sig-people; Tue, 16 Jan 1996 12:54:46 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id MAA04584 for ; Tue, 16 Jan 1996 12:54:28 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA19495; Tue, 16 Jan 96 12:53:52 EST Date: Tue, 16 Jan 96 12:53:52 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601161753.AA19495@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: hinsenk@ere.umontreal.ca Cc: matrix-sig@python.org In-Reply-To: <199601161710.MAA07614@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA) Subject: Re: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging Sender: owner-matrix-sig@python.org Precedence: bulk Date: Tue, 16 Jan 1996 12:10:28 -0500 From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad) 1) Konrad's numeric patches to the python core (incorporated into the working version of python by Guido) will be required and included with the distribution. These will be the only patches to the python core required. There has been a slight modification in the meantime (there will be only 'j' for imaginary constants, not 'i') and two corrections that perhaps not everybody has received. I'll prepare a new set of patch files for the beta release. Great, just upload the patch files to "the site" when you're ready. This PyArray C type will not implement automatic type-coercion (unlike the current implementation). The reason for this is that I have decided type-coercion is a major pain for arrays of floats (as opposed to doubles) and I happen to use arrays of floats for most of my work. If somebody can give me a good suggestion for how to keep automatic type-coercion and have Matrix_f(1,2,3)*1.2 == Matrix_f(1.2,2.4,3.6) then I might consider changing this decision. See later note on Array.py for an alternative. One way to solve your "float" problem would be to write Matrix_f(1,2,3)*Matrix_f(1.2), i.e. cast 1.2 explicitly to float. I don't see how you can avoid that cast anyway; without type coercion, Matrix_f(1,2,3)*1.2 would be an error. I'm willing to be a little bit sloppy with some of the details of types, and force the type of a scalar argument to correspond to the type of the matrix. This is essentially what I'm doing anyway when I say Matrix_f(1.2) [Note, I'll have another post to come about constructor function]. I'd prefer to have type coercion everywhere for consistency. If the "high-level" classes Array and Matrix are really fast enough (about which I have doubts, see below), then it doesn't matter that much, but in general I'd give top priority to consistency and put the burden of more complicated coding on those who want top speed (which of course must be possible). We seem to just have a difference of opinion here, I'll wait and see if more opinions come in. 3) Two dynamically linkable modules called "umathmodule.c", and "ieee_umathmodule.c" I don't think that using "ieee" in the name is a good idea. After all, there is no guarantee that this will really use IEEE arithmetic (a Vax, for example, doesn't have IEEE arithmetic hardware). Why not just "fast_umath"? Good point, actually I was originally considering "slow_umath" and "fast_umath", but I knew you wouldn't like that. So, the new proposed name is "umath" and "fast_umath". 4) Two python objects, "Array.py" and "Matrix.py" Array is essentially a python binding around the underlying C type, and this will also provide for automatic type-coercion and will generally assume that it is only working with arrays of type long, double, and complex double (the three types of arrays that are equivalent to python objects). In my initial tests of this object on LLNL's simple benchmark, I found that the performance was only 5% slower than using the C object directly. Nevertheless the Python wrapper could cause a significant overhead for small arrays, which will also be used heavily. For example, I am planning to change my vector class (3d vectors for geometry etc.) to use an array of length 3 as its internal representation. Do you have any data on the speed penalty for such small arrays? No data yet on that, if you want to send me a nice simple "real" benchmark code for 3d vectors I'd be happy to add it to my bag of tricks. Right now everything is being optimized for LLNL style physics code because they sent me such a nice benchmark to run. Matrix will inherit almost everything from Array, however it will be limited to 2 or fewer dimensions, and m1*m2 where m1 and m2 are matrices will perform matrix style multiplication. If the linear algebra people would like, other changes can be made (ie. ~m1 == m1.transpose(), ...). Based on the experiments with Array, the performance penalty for this approach should be minimal. I suppose some discussion will be necessary about the details of this class, which until now didn't exist. Given that many people seem to prefer a distinction between row and column vectors, it would be a good idea to include separate constructors for them. Now that I've brought things out into python classes, I think I'll probably just let somebody who's into linear algebra code worry about those sorts of refinements. 5) A standard library "Numeric.py" which will be the standard way of importing multiarray, Array, Matrix, umath, etc. It will include the inverted trig functions ("sec", "csc", "arcsec", ...) as well as most of the standard functions currently in Matrix.py I am not sure it is a good idea to have such an all-encompassing module. It would contain a rather arbitrary selection of things, i.e. those which exist at the time it is introduced. I expect that other numerical modules will appear later. So it would be better to group things according to functions or applications. I disagree here. I find that there is a certain core of functions that are very nice to be able to grab with a single import statement. I would hate to have to start my typical script with import umath, UMathMore, Matrix, Array, ArrayConstructors, ArrayUtilities, ... 9?) A "numericmodule.c" which contains reasonably efficient fft, matrix inversion, convolution, random numbers, eigenvalues and filtering functions stolen from existing C code so that the package can be viewed as a "complete" numerical computing system? Again this should be several modules: linear algebra, random numbers, fft/convolution. We won't achieve anything "complete" anyway, so let's not pretend we do. You're obviously right here. Part of what I was hoping to do with this was to create a simple module so that people (even those without a FORTRAN compiler) could take the inverse of a matrix in a reasonable amount of time, and possibly to set up some sort of API, so that when somebody does write the ultimate linear algebra module they will be likely include a function called "inverse" so that I could then trivially plug in this new more powerful system. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 16 13:01:48 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA04635 for matrix-sig-people; Tue, 16 Jan 1996 13:01:48 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id NAA04631 for ; Tue, 16 Jan 1996 13:01:44 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id SAA04176; Tue, 16 Jan 1996 18:00:46 GMT From: "Jim Fulton, U.S. Geological Survey" Message-Id: <9601161300.ZM4174@dsdbqvarsa.er.usgs.gov> Date: Tue, 16 Jan 1996 13:00:45 -0500 In-Reply-To: hinsenk@ERE.UMontreal.CA (Hinsen Konrad) "Re: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging" (Jan 16, 12:17pm) References: <199601161717.MAA08000@cyclone.ERE.UMontreal.CA> X-Phone: (703) 648-5622 X-Face: (;vf"lccr&V=y Subject: Re: [PYTHON MATRIX-SIG] Final matrix object renaming and packagin > > I feel strongly that the Matrix module should export it's C interface > using CObjects so that modules using matrices to not require that any > of the matrix software be statically linked. In my distribution of > > I agree in principle, but that doesn't have to happen in the > first beta release. Perhaps not. I'll put this together myself when I get the next alpha release. > > Are these imported by Numeric, or would the user be importing these? > Is the current built-in array module going away? If not, then there > is a name conflict on case-insensitive file systems. I like the idea > > Why? The current "array" in a C module, so "import Array" will > never load it, independent of the file name conventions. Which > means that "import Array" will always obtain an external module, > and as long as ours is the only one, it hardly matters if it > is called "Array.py" or "array.py". Actually, at least in 1.2, if a C module and a Python module are in the same directory, the C module gets loaded first. But this will be most likely a function of what order libraries appear in sys.path. I just think that it would be best to avoid this problem altogther by using a different name. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 16 13:07:31 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA04718 for matrix-sig-people; Tue, 16 Jan 1996 13:07:31 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id NAA04713 for ; Tue, 16 Jan 1996 13:07:27 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA19538; Tue, 16 Jan 96 13:07:03 EST Date: Tue, 16 Jan 96 13:07:03 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601161807.AA19538@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: hinsenk@ere.umontreal.ca Cc: jfulton@usgs.gov, matrix-sig@python.org In-Reply-To: <199601161717.MAA08000@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA) Subject: Re: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging Sender: owner-matrix-sig@python.org Precedence: bulk Date: Tue, 16 Jan 1996 12:17:41 -0500 From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad) importing Matrix and Array. Is there any reason for the user to import Array directly? I am strongly opposed to using "Array" unless Certainly, e.g. if the user doesn't want anything else. Besides, as I pointed out in my last mail, a general "Numeric" module is not such a terrific idea. So someone who wants to create a matrix object will do something like: import Numeric m=Numeric.Matrix_d(...) Speaking about constructors... I still prefer (strongly) more meaningful names, like IntegerMatrix. Noone should be forced to learn these silly type codes for standard applications. Well, actually I guess I wasn't clear enough about my choice for constructors. The array constructor (for PyArrayObjects): array([1,2,3], 'd') OR array([1,2,3], types.FloatType) will both produce a 1d array of doubles. Similarly for Arrays (and Matrix's): Array([1,2,3], 'd') OR Array([1.,2.,3.]) OR Array([1,2,3], types.FloatType) One dilemma left is what should be used as the standard string representation of an Array (or an array). I vote for Array([1,2,3], 'd'), but I'm open to other proposals. Now, I happen to really like the old notation, and I'd love to come up with a way to keep it around as a shorthand input notation. I really did find that the removal of a set of parenthesis made some of my denser code a lot easier to read. So, I propose to add A_d(1,2,3) as a convenient shorthand for Array([1,2,3], "d"). Please, let me know how terrible an idea this is. I'd also be happy to have FloatArray(1,2,3) instead. If something like this is desired, then I might even choose to use it as the standard string representation. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 16 13:42:33 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA04973 for matrix-sig-people; Tue, 16 Jan 1996 13:42:33 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id NAA04969 for ; Tue, 16 Jan 1996 13:42:30 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id SAA04238; Tue, 16 Jan 1996 18:42:37 GMT From: "Jim Fulton, U.S. Geological Survey" Message-Id: <9601161342.ZM4236@dsdbqvarsa.er.usgs.gov> Date: Tue, 16 Jan 1996 13:42:37 -0500 In-Reply-To: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) "Re: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging" (Jan 16, 12:39pm) References: <9601161739.AA19446@baribal.LCS.MIT.EDU.LCS.MIT.EDU> X-Phone: (703) 648-5622 X-Face: (;vf"lccr&V=y Subject: Re: [PYTHON MATRIX-SIG] Final matrix object renaming and packagin > > From: "Jim Fulton, U.S. Geological Survey" > > > 2) One C module that must be statically linked called "multiarraymodule.c" > > By statically linked, I assume you mean statically linked into the interpreter. > Right? > > > Having this particular module statically linked will eliminate the > > need for getting the CObject proposal working before release. > > As Guido said, the CObject proposal is working. I'll send it to you in a > separate note. > > I finished the (very small) CObject implementation very soon after the > workshop because I feel it is important to use it for the Matrix (Numeric) > module. > > I feel strongly that the Matrix module should export it's C interface > using CObjects so that modules using matrices to not require that any > of the matrix software be statically linked. In my distribution of > Python, I statically link as little as possible to keep the > interpreter and the interpreter start-up time small. I > plan to modify FIDL to use the CObject-exported interface. > > I'd be happy to assist you with this if you wish. > > I guess we run on faster networks at MIT, I never bother with dynamic > linking, and find that 4MB binaries launch as fast as I wish. Just out of curiousity, how fast is that? What do you get from: time python -c '' On some of our slower systems, this takes about a second, even with almost all modules dynamically loaded. This is much slower than I'd like it to be, since Python is often used here for very small scripts in which this startup time is a significant part of the overall execution time. Starting up another version of the interpreter with Tk linked in takes nearly twice as long. > I'd be > more than happy to have somebody else implement the CObject interface, > but I just don't see the time it would take (including getting myself > up to speed on the vagaries of dynamic linking) is worthwhile. If you > (Jim Fulton) want to try to add this interface, I'd be happy to help. I'll take a crack at this when you release 0.3. > > > > Use "PyArray_" as the name of the Matrix Object. This is a simple > > renaming of the existing "PyMatrix_". > > > > Use "array(sequence, typecode='d')" as the default > > constructor for this new C type. > > Is this a replacement for the existing array type? > > I decided it was not worth the huge set of compromises that would have > been necessary to make the matrix/array object truly compatible with > the existing array object. Still, the right name for this object > really is array. > > There's no problem with using PyArray as the C name for the object > because the existing array object does not export an interface. Also, > there's no problem with using the name "array" as a constructor > because avoiding these sorts of naming conflicts is why python has a > module system in the first place. No existing code that imports > "arraymodule" will be broken, but hopefully people in the future will > start using the new multiarraymodule for the same tasks. But existing imports of array on some systems may be broken. Even though the array module is stored in arraymodule._, it is imported with "import array". > > 4) Two python objects, "Array.py" and "Matrix.py" > > Are these imported by Numeric, or would the user be importing > these? > > I plan to have everything in the basic distribution imported into a > flat name space under the Numeric module. However, there's nothing to > stop people from importing these independently if they wish. > > Is the current built-in array module going away? If not, then there > is a name conflict on case-insensitive file systems. > > The array module is not going away. I'm confused about the problem of > case-insensitive file systems, what do they do with tkintermodule and > TkInter? Nothing, since Tkinter doesn't work on these systems yet. :-) > If this is in fact an issue, then "Array.py" can be changed > to "UserArray.py" in the spirit of UserList, etc. I think it should. > I like the idea > of having the user import Numeric and then use the > Numeric.Matrix_d(...) or Numeric.Array_f(...) rather than than > importing Matrix and Array. Is there any reason for the user to > import Array directly? I am strongly opposed to using "Array" unless > the "array" module goes away. > > I agree with all this (except the last line, which I'm willing to > conceed after you explain my tkinter question). Great. ;-) > > In order to support these python objects (and others like them), two > > special data members will be added, "__array__", and "__object__". If > > an object has the member "__array__", then the C functions that handle > > matrices will attempt to retrieve the matrix from this member when > > passed in a python object. > > Are we taking about python members or C structure members? Is the > __array__ member supposed to be the C pointer to a block of memory? > > The __array__ member is a member of a python object which is expected > to contain a python object of type array (the type created in C that > this whole thing is based on). > > > In addition, they will attempt to convert > > their result to an object of class "__object__" upon return. > > Class __object__? So __object__ is a pointer to a Python class > object? > > This is still a python member. In python what it would do is call > m.__object__(new_array). I assume that a similar thing can be done in C > (I haven't implemented this in C yet). And new_array is one of the new built-in array objects? > > This > > means that umath.sin(Array([0, pi/2, pi])) == Array([0.,1.,0.]). > > OK. This makes sense > > Remember that Array is a python object here, that's the trick I'm > trying to make work out. So all of this is really about being able to derive Python classes from built-in types? That is, you want an Array (which is an instance of a Python class) to store it's data in an array (which is an object of type PyArrayType), and you want functions that you pass an Array to to get at it's array. Have I got this right? (BTW, you should export the actual type objects.) > > Hopefully, this convention will allow these python objects to coexist > > well with any numeric libraries. > > Could you provide some additional details? > > Here's a bit of code for a unary function expecting a single PyArray > argument of type "double" of two dimensions: > > PyObject *op; > PyArrayObject *ap, *rp; > > TRY(PyArg_ParseTuple(args, "O", &op)); > TRY(ap = PyArray_ContiguousFromObject(op, PyArray_DOUBLE, 2, 2)); > > // Do something with ap to get rp > > Py_DECREF(ap); > > return PyArray_Return(rp, op); > > With the exception of the second argument to PyArray_Return, this is > the current way of writing such a chunk of code. > > PyArray_ContiguousFromObject will convert any python sequence type to > an array of the appropriate type and dimensions if possible. If the > argument is already an array of the appropriate type and dimensions, > then that array will be increfed and returned (unless its data points > to a discontiguous chunk of memory in which case it will be copied > into a new array with contiguous memory). > > The new feature that I want to add to this function is that if its > argument is a python object with the attribute "__array__", then this > function wil return the PyArrayObject contained in that attribute (if > this is indeed the case). OK. If my statement above is right, then I understand this. > PyArray_Return is used because some operations wind up producing a > 0-dimensional array. These will be converted to the appropriate > python scalars on return. > > The new feature that I want to add here is that if the second argument > has a "__class__" attribute, then the constructor for that class will You mean __object__? (I like __class__, or maybe even __return_constructor__ better.) > be used to return a new python object with the returned PyArrayObject > in its "__array__" attribute. So PyArray_Return checks to see of op has a callable __object__ member and if it does, returns the result of calling this member with rp as an argument. Right? > This is the simplest method I could come up with to get my > "sin(Array())" example to work. Whew. I need to think about this. I'm not faulting your approach, but it feels a bit complicated. What if you had a function with multiple arguments and you wanted the returned object to have the same type as the arguments? For example, what if you wanted spam(some_Array, some_other_Array) to return an Array and spam(some_Matric, some_other_Matrix) to return a Matrix? Would you use the first argument's __object__ or the second's? I'll probably have more to say about this after I take some time to mull it over. > > > > > 6) Great documentation and tutorials (hopefully written by Paul > > DuBois). > > Wat cool. Will we also get doc strings? > > doc strings are already done (probably could use some polishing, but... Great! Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 16 13:45:09 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA04990 for matrix-sig-people; Tue, 16 Jan 1996 13:45:09 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id NAA04986 for ; Tue, 16 Jan 1996 13:45:02 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id NAA12259 (8.6.11/IDA-1.6); Tue, 16 Jan 1996 13:43:11 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA12815; Tue, 16 Jan 1996 13:43:08 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA13986; Tue, 16 Jan 1996 13:43:07 -0500 Date: Tue, 16 Jan 1996 13:43:07 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601161843.NAA13986@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: matrix-sig@python.org In-reply-to: <9601161753.AA19495@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging Sender: owner-matrix-sig@python.org Precedence: bulk I'm willing to be a little bit sloppy with some of the details of types, and force the type of a scalar argument to correspond to the type of the matrix. This is essentially what I'm doing anyway when I Does that mean that in "2.5*array([1,2,3], types.Integer)" the prefactor will be silently cast to int? That would be a most undesirable form of automatic type conversion. One of the principles of operation should be that scalars are considered as arrays of rank 0. From that it follows that they can be cast explicitly to a rank-0 array of any type (including C-float), and there are no more type problems. BTW, that reminds me of another old constructor debate: should array([1,2,3]) be a rank-1 or a rank-2 array? I still propose that all constructors should take exactly one argument, scalar or list, and that the rank of the array depends on the nesting of the list. And now I have another argument for this: your current constructors don't permit the construction of rank-0 arrays. No data yet on that, if you want to send me a nice simple "real" benchmark code for 3d vectors I'd be happy to add it to my bag of Unfortunately that doesn't exist yet... tricks. Right now everything is being optimized for LLNL style physics code because they sent me such a nice benchmark to run. Which is hardly a good way to ensure the general usefulness of a program ;-) I disagree here. I find that there is a certain core of functions that are very nice to be able to grab with a single import statement. I would hate to have to start my typical script with import umath, UMathMore, Matrix, Array, ArrayConstructors, ArrayUtilities, ... Me too. But I'd rather write this "collection" module myself, with everything I want in it, instead of having a standard module that doesn't meet my needs. You're obviously right here. Part of what I was hoping to do with this was to create a simple module so that people (even those without a FORTRAN compiler) could take the inverse of a matrix in a reasonable amount of time, and possibly to set up some sort of API, so that when somebody does write the ultimate linear algebra module they will be likely include a function called "inverse" so that I could then trivially plug in this new more powerful system. We should learn a lesson from the old array module: never make an interim solution a standard module. If we now quickly hack together some collection of "useful" routines, then people will use this interface and have problems later when they discover that they need something better. Of course the implementation can always be improved, but if we define a standard interface for anything, it should be a good one. For linear algebra, the best solution would probably be a module based on LAPACK, which is free and also available in C (although the Fortran version is more efficient on most machines). I don't know if similar good and free packages exist for other applications, but if we can't find out, we should leave this to other people. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 16 13:48:03 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA05006 for matrix-sig-people; Tue, 16 Jan 1996 13:48:03 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id NAA05002 for ; Tue, 16 Jan 1996 13:47:59 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id NAB12354 (8.6.11/IDA-1.6); Tue, 16 Jan 1996 13:46:34 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA13974; Tue, 16 Jan 1996 13:46:32 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA14142; Tue, 16 Jan 1996 13:46:30 -0500 Date: Tue, 16 Jan 1996 13:46:30 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601161846.NAA14142@cyclone.ERE.UMontreal.CA> To: jfulton@usgs.gov CC: jjh@Goldilocks.LCS.MIT.EDU, matrix-sig@python.org In-reply-to: <9601161300.ZM4174@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging Sender: owner-matrix-sig@python.org Precedence: bulk function of what order libraries appear in sys.path. I just think that it would be best to avoid this problem altogther by using a different name. I'd rather investigate a bit more and choose a reasonable name if at all possible. After all, we are creating an API to be used by (hopefully ;-) millions of users, who probably don't want to dig into Python history to find out why something called an "array" everywhere has some strange name in Python. The compromises made necessary by Python's strange comparison implementation are bad enough. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 16 13:48:06 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA05006 for matrix-sig-people; Tue, 16 Jan 1996 13:48:03 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id NAA05002 for ; Tue, 16 Jan 1996 13:47:59 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id NAB12354 (8.6.11/IDA-1.6); Tue, 16 Jan 1996 13:46:34 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA13974; Tue, 16 Jan 1996 13:46:32 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA14142; Tue, 16 Jan 1996 13:46:30 -0500 Date: Tue, 16 Jan 1996 13:46:30 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601161846.NAA14142@cyclone.ERE.UMontreal.CA> To: jfulton@usgs.gov CC: jjh@Goldilocks.LCS.MIT.EDU, matrix-sig@python.org In-reply-to: <9601161300.ZM4174@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging Sender: owner-matrix-sig@python.org Precedence: bulk function of what order libraries appear in sys.path. I just think that it would be best to avoid this problem altogther by using a different name. I'd rather investigate a bit more and choose a reasonable name if at all possible. After all, we are creating an API to be used by (hopefully ;-) millions of users, who probably don't want to dig into Python history to find out why something called an "array" everywhere has some strange name in Python. The compromises made necessary by Python's strange comparison implementation are bad enough. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 16 14:07:51 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA05171 for matrix-sig-people; Tue, 16 Jan 1996 14:07:51 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id OAA05167 for ; Tue, 16 Jan 1996 14:07:46 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA21369; Tue, 16 Jan 96 14:07:25 EST Date: Tue, 16 Jan 96 14:07:25 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601161907.AA21369@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: jfulton@usgs.gov Cc: matrix-sig@python.org In-Reply-To: <9601161342.ZM4236@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging Sender: owner-matrix-sig@python.org Precedence: bulk From: "Jim Fulton, U.S. Geological Survey" > I guess we run on faster networks at MIT, I never bother with dynamic > linking, and find that 4MB binaries launch as fast as I wish. Just out of curiousity, how fast is that? What do you get from: time python -c '' On some of our slower systems, this takes about a second, even with almost all modules dynamically loaded. This is much slower than I'd like it to be, since Python is often used here for very small scripts in which this startup time is a significant part of the overall execution time. Starting up another version of the interpreter with Tk linked in takes nearly twice as long. ~:jjh@baribal: ls -l ~/PythonSun4/python -rwxrwxr-x 1 jjh 3227648 Jan 16 11:09 /usr/users/jjh/PythonSun4/python* ~:jjh@baribal: time ~/PythonSun4/python -c '' 0.060u 0.060s 0:00.14 85.7% 0+234k 0+0io 0pf+0w If you're curious about the details: All our active files are stored on a Pentium box running a high-performance NFS server kernel (no real operating system) this links into the network through a 100MB/s link to a switched hub. > I'd be > more than happy to have somebody else implement the CObject interface, > but I just don't see the time it would take (including getting myself > up to speed on the vagaries of dynamic linking) is worthwhile. If you > (Jim Fulton) want to try to add this interface, I'd be happy to help. I'll take a crack at this when you release 0.3. Great! > > > > Use "PyArray_" as the name of the Matrix Object. This is a simple > > renaming of the existing "PyMatrix_". > > > > Use "array(sequence, typecode='d')" as the default > > constructor for this new C type. > > Is this a replacement for the existing array type? > > I decided it was not worth the huge set of compromises that would have > been necessary to make the matrix/array object truly compatible with > the existing array object. Still, the right name for this object > really is array. > > There's no problem with using PyArray as the C name for the object > because the existing array object does not export an interface. Also, > there's no problem with using the name "array" as a constructor > because avoiding these sorts of naming conflicts is why python has a > module system in the first place. No existing code that imports > "arraymodule" will be broken, but hopefully people in the future will > start using the new multiarraymodule for the same tasks. But existing imports of array on some systems may be broken. Even though the array module is stored in arraymodule._, it is imported with "import array". You're talking about something else here. In my setup you'd do something like: from multiarray import array this would not conflict in any way with the existing arraymodule. Now arguments about Array.py on systems with case-insensitive filesystems are a different matter. > > In order to support these python objects (and others like them), two > > special data members will be added, "__array__", and "__object__". If > > an object has the member "__array__", then the C functions that handle > > matrices will attempt to retrieve the matrix from this member when > > passed in a python object. > > Are we taking about python members or C structure members? Is the > __array__ member supposed to be the C pointer to a block of memory? > > The __array__ member is a member of a python object which is expected > to contain a python object of type array (the type created in C that > this whole thing is based on). > > > In addition, they will attempt to convert > > their result to an object of class "__object__" upon return. > > Class __object__? So __object__ is a pointer to a Python class > object? > > This is still a python member. In python what it would do is call > m.__object__(new_array). I assume that a similar thing can be done in C > (I haven't implemented this in C yet). And new_array is one of the new built-in array objects? Exactly. > > This > > means that umath.sin(Array([0, pi/2, pi])) == Array([0.,1.,0.]). > > OK. This makes sense > > Remember that Array is a python object here, that's the trick I'm > trying to make work out. So all of this is really about being able to derive Python classes from built-in types? That is, you want an Array (which is an instance of a Python class) to store it's data in an array (which is an object of type PyArrayType), and you want functions that you pass an Array to to get at it's array. Have I got this right? Yep. In addition, I want these functions to return an Array instead of an array. (Even I'm beggining to doubt the value of this case-based differentiation of names by now). (BTW, you should export the actual type objects.) Of course, but this still doesn't get me the behavior I want. > > Hopefully, this convention will allow these python objects to coexist > > well with any numeric libraries. > > Could you provide some additional details? > > Here's a bit of code for a unary function expecting a single PyArray > argument of type "double" of two dimensions: > > PyObject *op; > PyArrayObject *ap, *rp; > > TRY(PyArg_ParseTuple(args, "O", &op)); > TRY(ap = PyArray_ContiguousFromObject(op, PyArray_DOUBLE, 2, 2)); > > // Do something with ap to get rp > > Py_DECREF(ap); > > return PyArray_Return(rp, op); > > With the exception of the second argument to PyArray_Return, this is > the current way of writing such a chunk of code. > > PyArray_ContiguousFromObject will convert any python sequence type to > an array of the appropriate type and dimensions if possible. If the > argument is already an array of the appropriate type and dimensions, > then that array will be increfed and returned (unless its data points > to a discontiguous chunk of memory in which case it will be copied > into a new array with contiguous memory). > > The new feature that I want to add to this function is that if its > argument is a python object with the attribute "__array__", then this > function wil return the PyArrayObject contained in that attribute (if > this is indeed the case). OK. If my statement above is right, then I understand this. > PyArray_Return is used because some operations wind up producing a > 0-dimensional array. These will be converted to the appropriate > python scalars on return. > > The new feature that I want to add here is that if the second argument > has a "__class__" attribute, then the constructor for that class will You mean __object__? (I like __class__, or maybe even __return_constructor__ better.) I guess I still haven't finalized the name of this yet. I like __return_constructor__, so I'll stick with that for now. > be used to return a new python object with the returned PyArrayObject > in its "__array__" attribute. So PyArray_Return checks to see of op has a callable __object__ member and if it does, returns the result of calling this member with rp as an argument. Right? Right, except of course now it checks for a __return_constructor__ member. > This is the simplest method I could come up with to get my > "sin(Array())" example to work. Whew. I need to think about this. I'm not faulting your approach, but it feels a bit complicated. What if you had a function with multiple arguments and you wanted the returned object to have the same type as the arguments? For example, what if you wanted spam(some_Array, some_other_Array) to return an Array and spam(some_Matric, some_other_Matrix) to return a Matrix? Would you use the first argument's __object__ or the second's? If they had different __return_constructor__'s, then I'd raise an exception. If they were the same, or only one of them had one, then I'd use the one that was present. I'll probably have more to say about this after I take some time to mull it over. Please say more about this. I'm just trying to come up with a way to make it reasonable to subclass the array/Array object for purposes like creating a Matrix object with as little pain as possible. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 16 14:27:54 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA05290 for matrix-sig-people; Tue, 16 Jan 1996 14:27:54 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id OAA05286 for ; Tue, 16 Jan 1996 14:27:49 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA14824; Tue, 16 Jan 96 11:27:33 PST Received: from kristen (localhost) by kristen.llnl.gov (5.x/SMI-SVR4) id AA17806; Tue, 16 Jan 1996 11:27:31 -0800 Message-Id: <30FBFC23.6D98@llnl.gov> Date: Tue, 16 Jan 1996 11:27:31 -0800 From: "Paul. Dubois" Organization: Lawrence Livermore National Laboratory X-Mailer: Mozilla 2.0b5 (X11; I; SunOS 5.4 sun4m) Mime-Version: 1.0 To: James Hugunin Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Thoughts on latest messages References: <9601161807.AA19538@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk Congratulations on your marriage and best wishes to you and your wife. We all hope you can "juggle" the matrix work in with the tasks you will receive from The Boss. Some thoughts on the latest messages. I think it best if I simply phrase them as what I need to do and let you mull over how this fits with various schemes. In what follows I use the name Thing to stand for "one of them thar matrix objects however it turns out you name it") a. In a C extension module, I need to create a Thing. The creation routines you have supplied in the present release are sufficient to my needs. b. Given a block of memory representing an array stored in column-major order, I need to turn it into a Thing. Things are natively row-major but given the current implementation I can diddle with the strides in the Thing and it works. This means Fortran interfacing is easy. Cool. Try not to lose this, and maybe we could make a constructor that did the kludge officially. c. If you want to make a Whatzit in Python that encapsulates a Thing but adds some special semantics (say, for example, arrays whose lower index is not 0) you can almost do it perfectly with a wrapper class but the one thing you can't spoof is when the object is an argument to something that expects a Thing. viz, sqrt(m). If I understood you, the idea would be that all routines expecting a Thing would, finding that they hadn't gotten one, try to do getattr(m, '__array__') so that objects that wished to pretend they are Things could do so. I think this will prove very valuable and can testify that I already ran into a case where it would have been a nice solution for me. Opinions: I like the coercion properties of the current package VERY much. If you need to do something special like keep expressions from drifting up from float to double, then some explicit handling of scalar constants is clearly required in either case, so I'm not persuaded anything would be gained by undoing your good work. About the packaging of mathematical software. A glance at the humongous documentation for Nag or IMSL will suggest that organizing the software into "clusters" by subject would be a good idea. In the EiffelMath library we have clusters for ODES, probability and statistics, linear solvers, optimization, root finding, time series analysis, etc. This helps people find what they need and also not have to bring in too much code just to get at a given piece. The Thing class is very likely to get imported via 'from Thing import *' because of the need to make the sqrt etc. work right; therefore, this namespace should be kept minimal, with more specialized software kept in other modules. Since it is easy to make supermodules that import sets of other modules, one can package the facilities to suit the style of work one does. If it is really true that you can make a PythonThing level object that has performance only 5% off of Thing, then that would be useful since we could inherit from it. Especially if (c) is done. -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 16 14:37:29 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA05365 for matrix-sig-people; Tue, 16 Jan 1996 14:37:29 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id OAA05361 for ; Tue, 16 Jan 1996 14:37:22 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA15180; Tue, 16 Jan 96 11:36:58 PST Received: from kristen (localhost) by kristen.llnl.gov (5.x/SMI-SVR4) id AA17859; Tue, 16 Jan 1996 11:36:55 -0800 Message-Id: <30FBFE57.4A68@llnl.gov> Date: Tue, 16 Jan 1996 11:36:55 -0800 From: "Paul. Dubois" Organization: Lawrence Livermore National Laboratory X-Mailer: Mozilla 2.0b5 (X11; I; SunOS 5.4 sun4m) Mime-Version: 1.0 To: Hinsen Konrad Cc: jfulton@usgs.gov, jjh@goldilocks.lcs.mit.edu, matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging References: <199601161846.NAA14142@cyclone.ERE.UMontreal.CA> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk Before we get too concerned about the old "array", has anyone checked the library and found out how many places it is used? Could the new one be plugged in pretty easily in most places that use it now? (I suspect so.) Could someone with access to the contributer library grep it? While requiring users to change existing code is very undesireable, this needs to be balanced by asking how many users, how much code, how much trouble. In the case of a central facility like this one, it is worth a little pain to use the right name, which surely is array. -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 16 15:42:57 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA05858 for matrix-sig-people; Tue, 16 Jan 1996 15:42:57 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id PAA05854 for ; Tue, 16 Jan 1996 15:42:52 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA10049; Tue, 16 Jan 96 15:37:41 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) for matrix-sig@python.org id PAA12207; Tue, 16 Jan 1996 15:47:51 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199601162047.PAA12207@maigret> Subject: Re: [PYTHON MATRIX-SIG] Thoughts on latest messages To: matrix-sig@python.org Date: Tue, 16 Jan 1996 15:47:50 -0500 (EST) In-Reply-To: <30FBFC23.6D98@llnl.gov> from "Paul. Dubois" at Jan 16, 96 11:27:31 am X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 1165 Sender: owner-matrix-sig@python.org Precedence: bulk Paul Dubois: >c. If you want to make a Whatzit in Python that encapsulates a Thing but >adds some special semantics (say, for example, arrays whose lower index >is not 0) you can almost do it perfectly with a wrapper class but the >one thing you can't spoof is when the object is an argument to something >that expects a Thing. viz, sqrt(m). If I understood you, the idea would >be that all routines expecting a Thing would, finding that they hadn't >gotten one, try to do getattr(m, '__array__') so that objects that >wished to pretend they are Things could do so. I think this will prove >very valuable and can testify that I already ran into a case where it >would have been a nice solution for me. Same here. It's a scheme similar to that used by Brian Werkentine in Rivet (you can call rivet functions w/ rivetobjs or with classes with a __widget attribute which is a rivetobj). It makes lots of things much much simpler. Examples that I've been thinking about are "documented matrices", (class instances with names and properties for the various dimensions) and "visible matrices" (class instances with state information for displayed matrices. --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 16 17:16:36 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA06463 for matrix-sig-people; Tue, 16 Jan 1996 17:16:36 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id RAA06459 for ; Tue, 16 Jan 1996 17:16:31 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id NAA12595 (8.6.11/IDA-1.6); Tue, 16 Jan 1996 13:52:38 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA16976; Tue, 16 Jan 1996 13:52:35 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA14638; Tue, 16 Jan 1996 13:52:27 -0500 Date: Tue, 16 Jan 1996 13:52:27 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601161852.NAA14638@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: jfulton@usgs.gov, matrix-sig@python.org In-reply-to: <9601161807.AA19538@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging Sender: owner-matrix-sig@python.org Precedence: bulk The array constructor (for PyArrayObjects): array([1,2,3], 'd') OR array([1,2,3], types.FloatType) will both produce a 1d array of doubles. OK. Similarly for Arrays (and Matrix's): Array([1,2,3], 'd') OR Array([1.,2.,3.]) OR Array([1,2,3], types.FloatType) Also OK. One dilemma left is what should be used as the standard string representation of an Array (or an array). I vote for Array([1,2,3], 'd'), but I'm open to other proposals. For the high-level arrays, I'd prefer simply Array([1,2,3]) (which is the simplest equivalent input format). For the low-level arrays, printing the type code is probably the best solution. Now, I happen to really like the old notation, and I'd love to come up with a way to keep it around as a shorthand input notation. I really did find that the removal of a set of parenthesis made some of my denser code a lot easier to read. So, I propose to add A_d(1,2,3) as a convenient shorthand for Array([1,2,3], "d"). Please, let me know how terrible an idea this is. Terrible enough for me... But do we need such abbreviations as part of the standard modules? It would probably add to the confusion of new users. Since it's a trivial Python definition, without speed penalties later on, why not let everyone define his own preferred abbreviations? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 19 10:45:38 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA15390 for matrix-sig-people; Fri, 19 Jan 1996 10:45:38 -0500 Received: from goldilocks ([18.27.0.167]) by python.org (8.6.12/8.6.12) with SMTP id KAA15386 for ; Fri, 19 Jan 1996 10:45:33 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU by goldilocks (4.1/SLS-4.1.1) id AA29079; Fri, 19 Jan 96 10:44:50 EST Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA13543; Fri, 19 Jan 96 10:43:53 EST Date: Fri, 19 Jan 96 10:43:53 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601191543.AA13543@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully) Sender: owner-matrix-sig@python.org Precedence: bulk Well, the comments on my previous post seem to have fallen off, so I'll try to summarize again and see if everybody's more or less happy. 1) No argument about including Konrad's latest patches (which I now have) 2) Naming conventions for the new module (this is new) I think that it's important to understand the value of python's module system in avoiding conflicts with the current array module, while still using the name array. The new array module will be called "multiarraymodule.c". To use this module I need to do the following: import multiarray a = multiarray.array([1,2,3]) Thus, the creation operator uses the name array (which has a different meaning in the old arraymodule). I will reccommend that people actually use the following style instead to import this module: import Numeric a = Numeric.array([1,2,3]) #an array of longs b = Numeric.array([1,2,3], types.FloatType) #an array of doubles c = Numeric.array([1,2,3], 'f') #an array of floats This C type WILL continue to implement automatic type coercion, but I reserve the right to remove it if I continue to have problems making my floating point code work in floats instead of doubles. I'll see how Jim Fulton's CObjects work out before making a final decision on when to add them to the release. An array of doubles will have the standard string representation of: array([1,2,3], 'd') As before, the print function is coded in python, and I encourage people to design friendly array print functions. 3) The two math modules will be umath and fast_umath. 4) Two python object UserArray.py and Matrix.py where Matrix inherits from UserArray. The two special data members "__array__", and "__return_constructor__" will be supported for python objects that want to pretend to be an array. 5) I will have an overall library Numeric.py including what I consider to be the basic functions desired by a numeric program. Note: after considering Konrad's and Paul's comments, this set of basic functions will be kept to a minimum. 6) doc strings are still the only docs for 0.30, this should change in BETA1 7) Test suite remains 8) pickle remains 9) "numericmodule.c" is scrapped for now One last question on installation: I plan to package this up as an addition to the standard python distribution, therefore, installation will consist of adding files to Modules, Lib, and Include. Does this strike anybody as an unreasonable thing to do? -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 19 11:17:50 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA15536 for matrix-sig-people; Fri, 19 Jan 1996 11:17:50 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id LAA15532 for ; Fri, 19 Jan 1996 11:17:46 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id QAA12375; Fri, 19 Jan 1996 16:17:34 GMT From: "Jim Fulton, U.S. Geological Survey" Message-Id: <9601191117.ZM12373@dsdbqvarsa.er.usgs.gov> Date: Fri, 19 Jan 1996 11:17:33 -0500 In-Reply-To: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) "[PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully)" (Jan 19, 10:43am) References: <9601191543.AA13543@baribal.LCS.MIT.EDU.LCS.MIT.EDU> X-Phone: (703) 648-5622 X-Face: (;vf"lccr&V=y Subject: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully) > > Well, the comments on my previous post seem to have fallen off, so > I'll try to summarize again and see if everybody's more or less happy. (snip) > The two special data members "__array__", and "__return_constructor__" > will be supported for python objects that want to pretend to be an > array. After thinking about this a bit, I'm quite comformable with the __array__ member idea, because there is a precedent with __float__, __int__, and __long__. However, I have a *strong* opinion that __array__ should be a member *function* that returns an instance as an array. This is in keeping with __float__, __long__, etc. If a user-defined type wants to store it's data in an array data member and return the member, it can, but other user-defined classes may want to compute and return array data on demand. I would also recomment that the __array__ member function should have an optional type code argument so that an extension that wants, say, a double precision array can request one. With an __array__ member function, I could, for example, immagine database table objects that extracted their numeric data as an array on demand. This could be quite useful. I'm not so comfortable with the __return_constructor__ idea. It seems to only make sense for single-argument functions. I have an alternate proposal. Suppose that certain functions allowed an optional (keyword?) argument that provided this constructor. So, for example, instead of: bar=sin(foo) where bar and foo are both Spams (e.g. Arrays) you would have: bar=sin(foo,Spam) Now, you don't want to have to type ",Spam" everytime, not because you don't like typing, but because you don't want the clutter, so you define a "sin" method in Spam: def sin(self): return sin(self,self.class) so then you could do: bar=foo.sin() I realize that this is not quite as pretty as the first version (or is it?), but it feels cleaner overall to me. In the case of multi-parameter functions, it will be clear which object (the reciever of the message) determines the return type. (snip) > One last question on installation: > > I plan to package this up as an addition to the standard python > distribution, therefore, installation will consist of adding files to > Modules, Lib, and Include. Does this strike anybody as an > unreasonable thing to do? For now, it's OK, but the released version should be a stand-alone module. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 19 11:26:12 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA15536 for matrix-sig-people; Fri, 19 Jan 1996 11:17:50 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id LAA15532 for ; Fri, 19 Jan 1996 11:17:46 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id QAA12375; Fri, 19 Jan 1996 16:17:34 GMT From: "Jim Fulton, U.S. Geological Survey" Message-Id: <9601191117.ZM12373@dsdbqvarsa.er.usgs.gov> Date: Fri, 19 Jan 1996 11:17:33 -0500 In-Reply-To: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) "[PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully)" (Jan 19, 10:43am) References: <9601191543.AA13543@baribal.LCS.MIT.EDU.LCS.MIT.EDU> X-Phone: (703) 648-5622 X-Face: (;vf"lccr&V=y Subject: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully) > > Well, the comments on my previous post seem to have fallen off, so > I'll try to summarize again and see if everybody's more or less happy. (snip) > The two special data members "__array__", and "__return_constructor__" > will be supported for python objects that want to pretend to be an > array. After thinking about this a bit, I'm quite comformable with the __array__ member idea, because there is a precedent with __float__, __int__, and __long__. However, I have a *strong* opinion that __array__ should be a member *function* that returns an instance as an array. This is in keeping with __float__, __long__, etc. If a user-defined type wants to store it's data in an array data member and return the member, it can, but other user-defined classes may want to compute and return array data on demand. I would also recomment that the __array__ member function should have an optional type code argument so that an extension that wants, say, a double precision array can request one. With an __array__ member function, I could, for example, immagine database table objects that extracted their numeric data as an array on demand. This could be quite useful. I'm not so comfortable with the __return_constructor__ idea. It seems to only make sense for single-argument functions. I have an alternate proposal. Suppose that certain functions allowed an optional (keyword?) argument that provided this constructor. So, for example, instead of: bar=sin(foo) where bar and foo are both Spams (e.g. Arrays) you would have: bar=sin(foo,Spam) Now, you don't want to have to type ",Spam" everytime, not because you don't like typing, but because you don't want the clutter, so you define a "sin" method in Spam: def sin(self): return sin(self,self.class) so then you could do: bar=foo.sin() I realize that this is not quite as pretty as the first version (or is it?), but it feels cleaner overall to me. In the case of multi-parameter functions, it will be clear which object (the reciever of the message) determines the return type. (snip) > One last question on installation: > > I plan to package this up as an addition to the standard python > distribution, therefore, installation will consist of adding files to > Modules, Lib, and Include. Does this strike anybody as an > unreasonable thing to do? For now, it's OK, but the released version should be a stand-alone module. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 19 11:40:52 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA15661 for matrix-sig-people; Fri, 19 Jan 1996 11:40:52 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id LAA15657 for ; Fri, 19 Jan 1996 11:40:48 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA13724; Fri, 19 Jan 96 11:39:18 EST Date: Fri, 19 Jan 96 11:39:18 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601191639.AA13724@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: jfulton@usgs.gov Cc: matrix-sig@python.org In-Reply-To: <9601191117.ZM12373@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully) Sender: owner-matrix-sig@python.org Precedence: bulk From: "Jim Fulton, U.S. Geological Survey" > The two special data members "__array__", and "__return_constructor__" > will be supported for python objects that want to pretend to be an > array. After thinking about this a bit, I'm quite comformable with the __array__ member idea, because there is a precedent with __float__, __int__, and __long__. However, I have a *strong* opinion that __array__ should be a member *function* that returns an instance as an array. This is in keeping with __float__, __long__, etc. If a user-defined type wants to store it's data in an array data member and return the member, it can, but other user-defined classes may want to compute and return array data on demand. This makes sense. I would also recomment that the __array__ member function should have an optional type code argument so that an extension that wants, say, a double precision array can request one. I don't agree with this. I really think that the __array__ member function should return whatever the internal array is used by the object, and let the extension convert it as it sees fit. With an __array__ member function, I could, for example, immagine database table objects that extracted their numeric data as an array on demand. This could be quite useful. This sounds useful. I'm not so comfortable with the __return_constructor__ idea. It seems to only make sense for single-argument functions. I have an alternate proposal. Suppose that certain functions allowed an optional (keyword?) argument that provided this constructor. So, for example, instead of: bar=sin(foo) where bar and foo are both Spams (e.g. Arrays) you would have: bar=sin(foo,Spam) Now, you don't want to have to type ",Spam" everytime, not because you don't like typing, but because you don't want the clutter, so you define a "sin" method in Spam: def sin(self): return sin(self,self.class) so then you could do: bar=foo.sin() I realize that this is not quite as pretty as the first version (or is it?), but it feels cleaner overall to me. In the case of multi-parameter functions, it will be clear which object (the reciever of the message) determines the return type. This is actually the version that I currently have running (more or less) it can infact be implemented entirely in python as a wrapper around the underlying math functions. My principle problem with this approach is that it requires me to know about all of the available numeric functions and define equivalents for them whenever I define a new Spam. This just seems unreasonable to me. For example, let's say I make a OneIndexArray (FORTRAN style indexes start at 1). Obviously, I want this array to work with all of the existing operations on arrays, and to return a OneIndexArray back. I think that Jim F sees a problem with power(OneIndexArray, Matrix). I would simply have this function call raise an exception saying that it was called with arrays having different __return_constructor__'s. > One last question on installation: > > I plan to package this up as an addition to the standard python > distribution, therefore, installation will consist of adding files to > Modules, Lib, and Include. Does this strike anybody as an > unreasonable thing to do? For now, it's OK, but the released version should be a stand-alone module. Maybe I'll let somebody else take care of these final packaging details. Jim -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 19 11:58:50 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA15755 for matrix-sig-people; Fri, 19 Jan 1996 11:58:50 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id LAA15751 for ; Fri, 19 Jan 1996 11:58:37 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id QAA12404; Fri, 19 Jan 1996 16:58:22 GMT From: "Jim Fulton, U.S. Geological Survey" Message-Id: <9601191158.ZM12402@dsdbqvarsa.er.usgs.gov> Date: Fri, 19 Jan 1996 11:58:21 -0500 In-Reply-To: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) "Re: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully)" (Jan 19, 11:39am) References: <9601191639.AA13724@baribal.LCS.MIT.EDU.LCS.MIT.EDU> X-Phone: (703) 648-5622 X-Face: (;vf"lccr&V=y Subject: Re: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully) > > From: "Jim Fulton, U.S. Geological Survey" > > > The two special data members "__array__", and "__return_constructor__" > > will be supported for python objects that want to pretend to be an > > array. > > After thinking about this a bit, I'm quite comformable with the > __array__ member idea, because there is a precedent with __float__, > __int__, and __long__. However, I have a *strong* opinion that > __array__ should be a member *function* that returns an instance as an > array. This is in keeping with __float__, __long__, etc. If a > user-defined type wants to store it's data in an array data member and > return the member, it can, but other user-defined classes may want to > compute and return array data on demand. > > This makes sense. > > I would also recomment that the __array__ member function should have > an optional type code argument so that an extension that wants, say, a > double precision array can request one. > > I don't agree with this. I really think that the __array__ member > function should return whatever the internal array is used by the > object, and let the extension convert it as it sees fit. I don't feel too strongly about this second point, however, I suspect that there could be many interesting classes that don't store their data in arrays at all. These will have to make some decision about what to return and this could cause an extra array to be allocated. Also, having these conversion functions take a type argument is sort of symetric with the type argument in the array constructor. > > With an __array__ member function, I could, for example, immagine database > table objects that extracted their numeric data as an array on demand. > This could be quite useful. > > This sounds useful. > > I'm not so comfortable with the __return_constructor__ idea. It seems > to only make sense for single-argument functions. I have an alternate > proposal. Suppose that certain functions allowed an optional > (keyword?) argument that provided this constructor. So, for example, > instead of: > > bar=sin(foo) > > where bar and foo are both Spams (e.g. Arrays) you would have: > > bar=sin(foo,Spam) > > Now, you don't want to have to type ",Spam" everytime, not because you > don't like typing, but because you don't want the clutter, so you > define a "sin" method in Spam: > > def sin(self): return sin(self,self.class) > > so then you could do: > > bar=foo.sin() > > I realize that this is not quite as pretty as the first version (or is > it?), but it feels cleaner overall to me. In the case of > multi-parameter functions, it will be clear which object (the > reciever of the message) determines the return type. > > This is actually the version that I currently have running (more or > less) it can infact be implemented entirely in python as a wrapper > around the underlying math functions. My principle problem with this > approach is that it requires me to know about all of the available > numeric functions and define equivalents for them whenever I define a > new Spam. This just seems unreasonable to me. > > For example, let's say I make a OneIndexArray (FORTRAN style indexes > start at 1). Obviously, I want this array to work with all of the > existing operations on arrays, and to return a OneIndexArray back. > > I think that Jim F sees a problem with Actually, I'm having more of a "gut" reaction. > > power(OneIndexArray, Matrix). > > I would simply have this function call raise an exception saying that > it was called with arrays having different __return_constructor__'s. I think I'd rather have this thing return an array in this case. I guess I think it would be better to make this explicit, like: OneIndexArray.power(Matrix) or like: OneIndexArray(power(OneIndexArray, Matrix)) > > > One last question on installation: > > > > I plan to package this up as an addition to the standard python > > distribution, therefore, installation will consist of adding files to > > Modules, Lib, and Include. Does this strike anybody as an > > unreasonable thing to do? > > For now, it's OK, but the released version should be a stand-alone module. > > Maybe I'll let somebody else take care of these final packaging details. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 19 12:00:52 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA15772 for matrix-sig-people; Fri, 19 Jan 1996 12:00:52 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id MAA15768 for ; Fri, 19 Jan 1996 12:00:48 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id LAA25992 (8.6.11/IDA-1.6); Fri, 19 Jan 1996 11:59:00 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id LAA04645; Fri, 19 Jan 1996 11:58:59 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id LAA12068; Fri, 19 Jan 1996 11:58:58 -0500 Date: Fri, 19 Jan 1996 11:58:58 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601191658.LAA12068@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: matrix-sig@python.org In-reply-to: <9601191543.AA13543@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully) Sender: owner-matrix-sig@python.org Precedence: bulk This C type WILL continue to implement automatic type coercion, but I reserve the right to remove it if I continue to have problems making my floating point code work in floats instead of doubles. But at some point it will have to be stable! 4) Two python object UserArray.py and Matrix.py where Matrix inherits from UserArray. I wonder about the meaning of "User" in the name "UserArray". Does it mean "the arrays that users really want"? My first idea when seeing such a name would be "user-defined arrays", which is not at all what we are talking about. I still propose to explore whether it is really definitely impossible to use the name "Array", but if that is the case, then something like "NumericArray" would make more sense than "UserArray". I plan to package this up as an addition to the standard python distribution, therefore, installation will consist of adding files to Modules, Lib, and Include. Does this strike anybody as an unreasonable thing to do? It's probably the only reasonable thing to do for now. When my patches are part of the standard distribution (i.e. in Python 1.4), it should be possible to package everything as a standard extension. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 19 12:04:31 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA15808 for matrix-sig-people; Fri, 19 Jan 1996 12:04:31 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id MAA15803 for ; Fri, 19 Jan 1996 12:04:26 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id MAA26130 (8.6.11/IDA-1.6); Fri, 19 Jan 1996 12:03:02 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA06636; Fri, 19 Jan 1996 12:03:02 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA13454; Fri, 19 Jan 1996 12:03:00 -0500 Date: Fri, 19 Jan 1996 12:03:00 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601191703.MAA13454@cyclone.ERE.UMontreal.CA> To: jfulton@usgs.gov CC: jjh@Goldilocks.LCS.MIT.EDU, matrix-sig@python.org In-reply-to: <9601191117.ZM12373@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully) Sender: owner-matrix-sig@python.org Precedence: bulk After thinking about this a bit, I'm quite comformable with the __array__ member idea, because there is a precedent with __float__, __int__, and __long__. However, I have a *strong* opinion that __array__ should be a member *function* that returns an instance as an array. This is in keeping with __float__, __long__, etc. If a user-defined type wants to store it's data in an array data member and return the member, it can, but other user-defined classes may want to compute and return array data on demand. I agree that this would be better, but I worry about efficiency. Function calls in Python are notoriously slow and should be avoided. Now, you don't want to have to type ",Spam" everytime, not because you don't like typing, but because you don't want the clutter, so you define a "sin" method in Spam: def sin(self): return sin(self,self.class) so then you could do: bar=foo.sin() I realize that this is not quite as pretty as the first version (or is it?), but it feels cleaner overall to me. In the case of Actually, when you import "sin" from umath, sin(foo) will automatically be translated into foo.sin(). Again, I worry about the additional overhead of a Python function call. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 19 12:37:08 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA16076 for matrix-sig-people; Fri, 19 Jan 1996 12:37:08 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id MAA16071 for ; Fri, 19 Jan 1996 12:37:04 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA14476; Fri, 19 Jan 96 12:36:30 EST Date: Fri, 19 Jan 96 12:36:30 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601191736.AA14476@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: hinsenk@ere.umontreal.ca Cc: matrix-sig@python.org In-Reply-To: <199601191658.LAA12068@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA) Subject: Re: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully) Sender: owner-matrix-sig@python.org Precedence: bulk Date: Fri, 19 Jan 1996 11:58:58 -0500 From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad) This C type WILL continue to implement automatic type coercion, but I reserve the right to remove it if I continue to have problems making my floating point code work in floats instead of doubles. But at some point it will have to be stable! I agree. It will be stable in the BETA1 release. I'll use release 0.30 to see how well things work for me. 4) Two python object UserArray.py and Matrix.py where Matrix inherits from UserArray. I wonder about the meaning of "User" in the name "UserArray". Does it mean "the arrays that users really want"? My first idea when seeing such a name would be "user-defined arrays", which is not at all what we are talking about. I still propose to explore whether it is really definitely impossible to use the name "Array", but if that is the case, then something like "NumericArray" would make more sense than "UserArray". This is just an extension of the current UserList.py and UserDictionary.py names in the standard python distribution to refer to python classes that are derived from C-types. The principle purpose for these classes (and for UserArray as well) is to be used as a superclass for python defined subclasses. I don't think this convention is too bad, so I think it's best to stay consistent. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 19 14:29:47 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA16905 for matrix-sig-people; Fri, 19 Jan 1996 14:29:47 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id OAA16901 for ; Fri, 19 Jan 1996 14:29:35 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA21539; Fri, 19 Jan 96 11:29:07 PST Received: from kristen (localhost) by kristen.llnl.gov (5.x/SMI-SVR4) id AA04502; Fri, 19 Jan 1996 11:29:04 -0800 Message-Id: <30FFF100.7203@llnl.gov> Date: Fri, 19 Jan 1996 11:29:04 -0800 From: "Paul. Dubois" Organization: Lawrence Livermore National Laboratory X-Mailer: Mozilla 2.0b5 (X11; I; SunOS 5.4 sun4m) Mime-Version: 1.0 To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Attributes vs. methods References: <9601191639.AA13724@baribal.LCS.MIT.EDU.LCS.MIT.EDU> <9601191158.ZM12402@dsdbqvarsa.er.usgs.gov> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk Pardon in advance if this gets too pedantic; I figured the range of OOP experience varies so I go somewhat slow. The question before the SIG is whether, for example, it should be x.shape or x.shape(). First, speaking in abstract OOP terms, a class can have two kinds of members, attributes and methods. The former is data, the latter is executable. In some languages (but not all) you can tell which kind of access is being made just by looking at it: x.f -- f is data x.f() -- f is a method (Aside: this is not a GoodThing. It forces the implementor of a class to commit to a representation for f, ie. to decide once and for all how clients will refer to f. For example, if f represents a certain quantity, say pressure, a decision is made immediately as to whether f is to be a primary state variable or whether to calculate it from something else, like the temperature. It is better if a data attribute is indistinguishable from the client's point of view from a function that takes no arguments. Eiffel has this right.) That said, one then has to ask whether a certain language allows statements of the form x.f = something If it does, then maintaining the object as an abstract data type become difficult. For example, we may have two attributes x and y and we want to ensure a certain relationship always holds between them. Obviously, if any old user can come along and assign something to x without changing y, we have problems. Thus, in many languages that allow assignment to x.f, books are written advising that all such f's be hidden (private) and that a function x.f() be supplied to the users instead. Then one worries about inlining, etc. In Python, the situation is more complicated. There is no "private". So, a convention has developed that if an attribute has a name beginning with an underscore, then the warranty on the object is void if the client assigns to it. This is a reasonable compromise in that the usual strategy of having an f() and a set_f(value) may suffer some performance penalty since inlining isn't going to be done, and anyway there is no private data anyway. Next, Python allows one to make it appear that f is an attribute even if it is not, so that while x.f "works" there really isn't such a data member and x.f = something would not be setting it (and may even fail, depending on the way the class was written). However, learning about this trick implies a level of sophistication beyond the level to be expected in the user of an application in which Python is used as the scripting language. Such a user is too likely to try to change the shape of the matrix by assignment. So, my conclusion is that if x.shape in the matrix class is an attribute, and we don't want any user to do x.shape=something, we should call it _shape, and supply an x.shape() for normal access. I think this argument applies to the level intended for use by the general public. They need a simple, consistent set of rules to live by. -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 19 14:57:27 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA17116 for matrix-sig-people; Fri, 19 Jan 1996 14:57:27 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU ([18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id OAA17095 for ; Fri, 19 Jan 1996 14:54:46 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA15133; Fri, 19 Jan 96 14:53:55 EST Date: Fri, 19 Jan 96 14:53:55 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601191953.AA15133@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: dubois1@llnl.gov Cc: matrix-sig@python.org In-Reply-To: <30FFF100.7203@llnl.gov> (dubois1@llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] Attributes vs. methods Sender: owner-matrix-sig@python.org Precedence: bulk Your last line made me realize that there is essentially a bug in the current implementation of the matrix class (I just wasn't thinking clearly) and this might be coloring the analysis. > So, my conclusion is that if x.shape in the matrix class is an > attribute, and we don't want any user to do x.shape=something, we should > call it _shape, and supply an x.shape() for normal access. For types written in C (as the array type is) access times between attributes and function calls are really indistinguishable, so the issue is whether to use x.shape() to return the current shape and x.reshape(new_shape) to set a new one, or to use x.shape and x.shape=new_shape instead (or maybe both). The fact that you can't do x.shape=new_shape in the current implementation is more an accident than a matter of design. It would be very easy to make this work properly (using the set_attribute facilities of python). Here's what I propose (after thinking about things one more time): a.typecode, a.itemsize, a.contiguous should all become methods, because setting these values makes no sense. a.shape, a.real, and a.imaginary should be implemented as pseudo-data in a self-consistent manner. This means that I'll fix the code so that a.shape = new_shape will work properly. Does this sound reasonable? -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 19 15:05:06 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA17171 for matrix-sig-people; Fri, 19 Jan 1996 15:05:06 -0500 Received: from dsdbqvarsa.er.usgs.GOV ([130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id PAA17140 for ; Fri, 19 Jan 1996 15:02:25 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id UAA13149; Fri, 19 Jan 1996 20:01:57 GMT From: "Jim Fulton, U.S. Geological Survey" Message-Id: <9601191501.ZM13147@dsdbqvarsa.er.usgs.gov> Date: Fri, 19 Jan 1996 15:01:56 -0500 In-Reply-To: "Paul. Dubois" "[PYTHON MATRIX-SIG] Attributes vs. methods" (Jan 19, 11:29am) References: <9601191639.AA13724@baribal.LCS.MIT.EDU.LCS.MIT.EDU> <9601191158.ZM12402@dsdbqvarsa.er.usgs.gov> <30FFF100.7203@llnl.gov> X-Phone: (703) 648-5622 X-Face: (;vf"lccr&V=y, matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Attributes vs. methods Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-matrix-sig@python.org Precedence: bulk On Jan 19, 11:29am, Paul. Dubois wrote: > Subject: [PYTHON MATRIX-SIG] Attributes vs. methods > Pardon in advance if this gets too pedantic; I figured the range of OOP > experience varies so I go somewhat slow. The question before the SIG is > whether, for example, it should be x.shape or x.shape(). > > First, speaking in abstract OOP terms, a class can have two kinds of > members, attributes and methods. The former is data, the latter is > executable. In some languages (but not all) you can tell which kind of > access is being made just by looking at it: > x.f -- f is data > x.f() -- f is a method > > (Aside: this is not a GoodThing. It forces the implementor of a class to > commit to a representation for f, ie. to decide once and for all how > clients will refer to f. For example, if f represents a certain > quantity, say pressure, a decision is made immediately as to whether f > is to be a primary state variable or whether to calculate it from > something else, like the temperature. It is better if a data attribute > is indistinguishable from the client's point of view from a function > that takes no arguments. Eiffel has this right.) > > That said, one then has to ask whether a certain language allows > statements of the form > > x.f = something > > If it does, then maintaining the object as an abstract data type become > difficult. For example, we may have two attributes x and y and we want > to ensure a certain relationship always holds between them. Obviously, > if any old user can come along and assign something to x without > changing y, we have problems. Thus, in many languages that allow > assignment to x.f, books are written advising that all such f's be > hidden (private) and that a function x.f() be supplied to the users > instead. Then one worries about inlining, etc. > > In Python, the situation is more complicated. There is no "private". All data in built-in types is private. Data in class instances can be made semi-private, with some effort. Actually, I hope that in the future there will be alternate class types that support features such as private attributes. > So, > a convention has developed that if an attribute has a name beginning > with an underscore, then the warranty on the object is void if the > client assigns to it. This is a reasonable compromise in that the usual > strategy of having an f() and a set_f(value) may suffer some performance > penalty since inlining isn't going to be done, and anyway there is no > private data anyway. > > Next, Python allows one to make it appear that f is an attribute even if > it is not, so that while x.f "works" there really isn't such a data > member and x.f = something would not be setting it (and may even fail, > depending on the way the class was written). However, learning about > this trick implies a level of sophistication beyond the level to be > expected in the user of an application in which Python is used as the > scripting language. Such a user is too likely to try to change the shape > of the matrix by assignment. This argument assumes that the unsophisticated user is sophisticated enough to worry about the implementation in the first place. > So, my conclusion is that if x.shape in the matrix class is an > attribute, and we don't want any user to do x.shape=something, we should I'd be inclined to think about this a little differently. I think it is reasonable for an interface to expose properties, where properties represent information that you can query and set for the object, without regard to implementation. I'd be inclined to agree that if you don't want to allow: x.shape = something then shape should probably not be modeled as an attribute, however, these same argument might be used to say that if you can say: "spam"[0] you should also able to say: "spam"[0]="S" > call it _shape, and supply an x.shape() for normal access. I think this > argument applies to the level intended for use by the general public. > They need a simple, consistent set of rules to live by. I'm not really disagreeing with you, but I find the idea of abstract attributes to be interesting. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 19 15:26:16 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA18723 for matrix-sig-people; Fri, 19 Jan 1996 15:26:16 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU ([18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id OAA17095 for ; Fri, 19 Jan 1996 14:54:46 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA15133; Fri, 19 Jan 96 14:53:55 EST Date: Fri, 19 Jan 96 14:53:55 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601191953.AA15133@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: dubois1@llnl.gov Cc: matrix-sig@python.org In-Reply-To: <30FFF100.7203@llnl.gov> (dubois1@llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] Attributes vs. methods Sender: owner-matrix-sig@python.org Precedence: bulk Your last line made me realize that there is essentially a bug in the current implementation of the matrix class (I just wasn't thinking clearly) and this might be coloring the analysis. > So, my conclusion is that if x.shape in the matrix class is an > attribute, and we don't want any user to do x.shape=something, we should > call it _shape, and supply an x.shape() for normal access. For types written in C (as the array type is) access times between attributes and function calls are really indistinguishable, so the issue is whether to use x.shape() to return the current shape and x.reshape(new_shape) to set a new one, or to use x.shape and x.shape=new_shape instead (or maybe both). The fact that you can't do x.shape=new_shape in the current implementation is more an accident than a matter of design. It would be very easy to make this work properly (using the set_attribute facilities of python). Here's what I propose (after thinking about things one more time): a.typecode, a.itemsize, a.contiguous should all become methods, because setting these values makes no sense. a.shape, a.real, and a.imaginary should be implemented as pseudo-data in a self-consistent manner. This means that I'll fix the code so that a.shape = new_shape will work properly. Does this sound reasonable? -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 19 15:29:51 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA18757 for matrix-sig-people; Fri, 19 Jan 1996 15:29:51 -0500 Received: from dsdbqvarsa.er.usgs.GOV ([130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id PAA17140 for ; Fri, 19 Jan 1996 15:02:25 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id UAA13149; Fri, 19 Jan 1996 20:01:57 GMT From: "Jim Fulton, U.S. Geological Survey" Message-Id: <9601191501.ZM13147@dsdbqvarsa.er.usgs.gov> Date: Fri, 19 Jan 1996 15:01:56 -0500 In-Reply-To: "Paul. Dubois" "[PYTHON MATRIX-SIG] Attributes vs. methods" (Jan 19, 11:29am) References: <9601191639.AA13724@baribal.LCS.MIT.EDU.LCS.MIT.EDU> <9601191158.ZM12402@dsdbqvarsa.er.usgs.gov> <30FFF100.7203@llnl.gov> X-Phone: (703) 648-5622 X-Face: (;vf"lccr&V=y, matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Attributes vs. methods Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-matrix-sig@python.org Precedence: bulk On Jan 19, 11:29am, Paul. Dubois wrote: > Subject: [PYTHON MATRIX-SIG] Attributes vs. methods > Pardon in advance if this gets too pedantic; I figured the range of OOP > experience varies so I go somewhat slow. The question before the SIG is > whether, for example, it should be x.shape or x.shape(). > > First, speaking in abstract OOP terms, a class can have two kinds of > members, attributes and methods. The former is data, the latter is > executable. In some languages (but not all) you can tell which kind of > access is being made just by looking at it: > x.f -- f is data > x.f() -- f is a method > > (Aside: this is not a GoodThing. It forces the implementor of a class to > commit to a representation for f, ie. to decide once and for all how > clients will refer to f. For example, if f represents a certain > quantity, say pressure, a decision is made immediately as to whether f > is to be a primary state variable or whether to calculate it from > something else, like the temperature. It is better if a data attribute > is indistinguishable from the client's point of view from a function > that takes no arguments. Eiffel has this right.) > > That said, one then has to ask whether a certain language allows > statements of the form > > x.f = something > > If it does, then maintaining the object as an abstract data type become > difficult. For example, we may have two attributes x and y and we want > to ensure a certain relationship always holds between them. Obviously, > if any old user can come along and assign something to x without > changing y, we have problems. Thus, in many languages that allow > assignment to x.f, books are written advising that all such f's be > hidden (private) and that a function x.f() be supplied to the users > instead. Then one worries about inlining, etc. > > In Python, the situation is more complicated. There is no "private". All data in built-in types is private. Data in class instances can be made semi-private, with some effort. Actually, I hope that in the future there will be alternate class types that support features such as private attributes. > So, > a convention has developed that if an attribute has a name beginning > with an underscore, then the warranty on the object is void if the > client assigns to it. This is a reasonable compromise in that the usual > strategy of having an f() and a set_f(value) may suffer some performance > penalty since inlining isn't going to be done, and anyway there is no > private data anyway. > > Next, Python allows one to make it appear that f is an attribute even if > it is not, so that while x.f "works" there really isn't such a data > member and x.f = something would not be setting it (and may even fail, > depending on the way the class was written). However, learning about > this trick implies a level of sophistication beyond the level to be > expected in the user of an application in which Python is used as the > scripting language. Such a user is too likely to try to change the shape > of the matrix by assignment. This argument assumes that the unsophisticated user is sophisticated enough to worry about the implementation in the first place. > So, my conclusion is that if x.shape in the matrix class is an > attribute, and we don't want any user to do x.shape=something, we should I'd be inclined to think about this a little differently. I think it is reasonable for an interface to expose properties, where properties represent information that you can query and set for the object, without regard to implementation. I'd be inclined to agree that if you don't want to allow: x.shape = something then shape should probably not be modeled as an attribute, however, these same argument might be used to say that if you can say: "spam"[0] you should also able to say: "spam"[0]="S" > call it _shape, and supply an x.shape() for normal access. I think this > argument applies to the level intended for use by the general public. > They need a simple, consistent set of rules to live by. I'm not really disagreeing with you, but I find the idea of abstract attributes to be interesting. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 19 15:39:45 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA18828 for matrix-sig-people; Fri, 19 Jan 1996 15:39:45 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id PAA18824 for ; Fri, 19 Jan 1996 15:39:39 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id PAA03994 (8.6.11/IDA-1.6); Fri, 19 Jan 1996 15:36:16 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA25601; Fri, 19 Jan 1996 15:36:13 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA25872; Fri, 19 Jan 1996 15:36:12 -0500 Date: Fri, 19 Jan 1996 15:36:12 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601192036.PAA25872@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: dubois1@llnl.gov, matrix-sig@python.org In-reply-to: <9601191953.AA15133@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] Attributes vs. methods Sender: owner-matrix-sig@python.org Precedence: bulk a.typecode, a.itemsize, a.contiguous should all become methods, because setting these values makes no sense. They could also be read-only attributes. BTW, setting a.typecode does make sense, as a way of casting an array to another type. Of course this is not the recommended style. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 19 16:21:15 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA19299 for matrix-sig-people; Fri, 19 Jan 1996 16:21:15 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id QAA19295 for ; Fri, 19 Jan 1996 16:21:12 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA15839; Fri, 19 Jan 96 16:20:44 EST Date: Fri, 19 Jan 96 16:20:44 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601192120.AA15839@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: hinsenk@ere.umontreal.ca Cc: dubois1@llnl.gov, matrix-sig@python.org In-Reply-To: <199601192036.PAA25872@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA) Subject: Re: [PYTHON MATRIX-SIG] Attributes vs. methods Sender: owner-matrix-sig@python.org Precedence: bulk Date: Fri, 19 Jan 1996 15:36:12 -0500 From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad) Cc: dubois1@llnl.gov, matrix-sig@python.org a.typecode, a.itemsize, a.contiguous should all become methods, because setting these values makes no sense. They could also be read-only attributes. BTW, setting a.typecode does make sense, as a way of casting an array to another type. Of course this is not the recommended style. Unfortunately, this can not be done with the current implementation of array objects. Because it is possible to have arrays that are references to the memory of another array, it is essential that casting produce a new array rather than modify the existing one. Right now they are read-only attributes, but I think I've come to agree with Paul that read-only attributes are a little bit confusing. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 19 18:42:21 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA20442 for matrix-sig-people; Fri, 19 Jan 1996 18:42:21 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id SAA20438 for ; Fri, 19 Jan 1996 18:42:18 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id QAA05861 (8.6.11/IDA-1.6 for ); Fri, 19 Jan 1996 16:26:04 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA06857; Fri, 19 Jan 1996 16:25:25 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA28242; Fri, 19 Jan 1996 16:25:24 -0500 Date: Fri, 19 Jan 1996 16:25:24 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601192125.QAA28242@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Evaluation release of expression compiler Sender: owner-matrix-sig@python.org Precedence: bulk For those of you who still don't know what to do during the weekend, I offer a nice toy. In the next message, I will send the source code for the very first incomplete, non-optimized version of my numerical expression compiler. It is a single C extension module called "numexprmodule.c". For now, both this module and the module "cmath" must be statically linked with the interpreter, since numexpr calls functions from cmath. Here's an example for an application: -------------------------------------------------- from numexpr import * from omath import * def f(x): return sin(x*x)+0.5*sqrt(x/7)*cos(2/x)*log(x+exp(-x))-sqrt(2)*tanh(x) f_compiled = f(FloatVariable()) print f(5) print f_compiled(5) -------------------------------------------------- The module "numexpr" exports three functions: IntegerVariable, FloatVariable, and ComplexVariable. Each of these takes an optional argument (default is zero) that indicates the position of this variable in the argument list of the compiled expression. So if you want a compiled expression for "x+y", write f = FloatVariable(0)+FloatVariable(1) The range of mathematical functions is the same as for cmath. All arithmetic operators are supported, with exception of exponentiation, which with the current coercion scheme cannot be implemented efficiently, so I decided to wait a bit. With that restriction, all functions that contain no conditional or loop statements can be compiled. Speed: First tests have shown that the asymptotic speed (i.e. for infinitely long functions) relative to uncompiled functions is about 40. For one-line expressions such as the one in the example above, expect a factor of 7 (which shows that the overhead if the interpreter is still substantial). Note that the expressions are not optimized in any way. If you compile something like temp = sin(x+y) f = temp*temp, then sin(x+y) will actually be calculated twice at execution time. So elimination of common subexpressions is the very least amount of optimization that will have to be done. To do: - Control structures - Optimization - Array expressions I'd appreciate any comments whatsoever. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 19 18:47:13 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA20478 for matrix-sig-people; Fri, 19 Jan 1996 18:47:13 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id SAA20474 for ; Fri, 19 Jan 1996 18:47:02 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id QAA05866 (8.6.11/IDA-1.6 for ); Fri, 19 Jan 1996 16:26:29 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA06999; Fri, 19 Jan 1996 16:25:49 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA28264; Fri, 19 Jan 1996 16:25:48 -0500 Date: Fri, 19 Jan 1996 16:25:48 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601192125.QAA28264@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] numexprmodule.c Sender: owner-matrix-sig@python.org Precedence: bulk /* Numexpr objects */ #include "allobjects.h" #include /* Type of stack entries */ typedef union { long i; double f; complex c; } stack_entry; /* Error object */ static PyObject *ErrorObject; /* Object definition */ typedef struct { PyObject_HEAD int nargs; /* number of parameters */ char *argtypes; /* types of parameters */ stack_entry *args; /* array to store actual arguments */ int nconstants; /* number of constants */ stack_entry *constants; /* constant array */ int codesize; /* number of code words */ int *code; /* compiled expression */ int stacksize; /* stack size needed for evaluation */ stack_entry *stack; /* execution stack */ char type; /* return type of the expression */ } PyNumExprObject; staticforward PyTypeObject PyNumExpr_Type; #define PyNumExpr_Check(v) ((v)->ob_type == &PyNumExpr_Type) /* Type codes */ #define INTEGER 1 #define FLOAT 2 #define COMPLEX 3 #define NTYPES 4 /* Opcodes used in compiled code */ #define GET_ARG 1 #define GET_CONSTANT 2 #define ADD 10 #define ADD_INT 11 #define ADD_FLOAT 12 #define ADD_COMPLEX 13 #define SUB 15 #define SUB_INT 16 #define SUB_FLOAT 17 #define SUB_COMPLEX 18 #define MUL 20 #define MUL_INT 21 #define MUL_FLOAT 22 #define MUL_COMPLEX 23 #define DIV 25 #define DIV_INT 26 #define DIV_FLOAT 27 #define DIV_COMPLEX 28 #define POW 30 #define NEG 40 #define NEG_INT 41 #define NEG_FLOAT 42 #define NEG_COMPLEX 43 #define ABS 45 #define ABS_INT 46 #define ABS_FLOAT 47 #define ABS_COMPLEX 48 #define TYPE_CAST 50 #define INTEGER_FLOAT 59 #define INTEGER_COMPLEX 63 #define FLOAT_INTEGER 56 #define FLOAT_COMPLEX 64 #define COMPLEX_INTEGER 57 #define COMPLEX_FLOAT 61 #define ACOS_FLOAT 100 #define ACOS_COMPLEX 101 #define ACOSH_FLOAT 102 #define ACOSH_COMPLEX 103 #define ASIN_FLOAT 104 #define ASIN_COMPLEX 105 #define ASINH_FLOAT 106 #define ASINH_COMPLEX 107 #define ATAN_FLOAT 108 #define ATAN_COMPLEX 109 #define ATANH_FLOAT 110 #define ATANH_COMPLEX 111 #define COS_FLOAT 112 #define COS_COMPLEX 113 #define COSH_FLOAT 114 #define COSH_COMPLEX 115 #define EXP_FLOAT 116 #define EXP_COMPLEX 117 #define LOG_FLOAT 118 #define LOG_COMPLEX 119 #define LOG10_FLOAT 120 #define LOG10_COMPLEX 121 #define SIN_FLOAT 122 #define SIN_COMPLEX 123 #define SINH_FLOAT 124 #define SINH_COMPLEX 125 #define SQRT_FLOAT 126 #define SQRT_COMPLEX 127 #define TAN_FLOAT 128 #define TAN_COMPLEX 129 #define TANH_FLOAT 130 #define TANH_COMPLEX 131 /* Utility functions */ #define max(a,b) (((a) > (b)) ? (a) : (b)) #define min(a,b) (((a) < (b)) ? (a) : (b)) /* Allocation and deallocation of numexpr objects */ static PyNumExprObject * new_numexprobject() { PyNumExprObject *self; self = NEWOBJ(PyNumExprObject, &PyNumExpr_Type); if (self == NULL) return NULL; self->nargs = 0; self->argtypes = NULL; self->args = NULL; self->nconstants = 0; self->constants = NULL; self->codesize = 0; self->code = NULL; self->stacksize = 0; self->stack = NULL; self->type = 0; return self; } static void numexpr_dealloc(self) PyNumExprObject *self; { PyMem_XDEL(self->argtypes); PyMem_XDEL(self->args); PyMem_XDEL(self->constants); PyMem_XDEL(self->code); PyMem_XDEL(self->stack); PyMem_DEL(self); } /* Attribute access. The only attributes are methods. */ struct PyMethodDef numexpr_methods[]; static PyObject * numexpr_getattr(self, name) PyNumExprObject *self; char *name; { return Py_FindMethod(numexpr_methods, (PyObject *)self, name); } /* Printed representation */ static PyObject * numexpr_repr(self) PyNumExprObject *self; { char buf[100]; sprintf(buf, "", self->argtypes); return PyString_FromString(buf); } /* Evaluation of a compiled expression */ extern complex c_sqrt(); extern complex c_acos(); extern complex c_acosh(); extern complex c_asin(); extern complex c_asinh(); extern complex c_atan(); extern complex c_atanh(); extern complex c_cos(); extern complex c_cosh(); extern complex c_exp(); extern complex c_log(); extern complex c_log10(); extern complex c_sin(); extern complex c_sinh(); extern complex c_sqrt(); extern complex c_tan(); extern complex c_tanh(); static PyObject * numexpr_call(self, args) PyNumExprObject *self; PyObject *args; { stack_entry *stack = self->stack; int *cp, *end; char text[50]; stack_entry *sp; PyObject *arg; char argtype[2]; PyObject *result; int i; if (PyObject_Length(args) != self->nargs) { PyErr_SetString(TypeError, "wrong number of arguments in compiled expression"); return NULL; } argtype[1] = '\0'; for (i = 0; i < self->nargs; i++) { arg = PySequence_GetItem(args, i); argtype[0] = self->argtypes[i]; if (argtype[0] == ' ') PyArg_Parse(arg, "O", &arg); else PyArg_Parse(arg, argtype, self->args+i); } end = self->code + self->codesize; cp = self->code; sp = stack-1; while (cp < end) { switch (*cp) { case GET_ARG: cp++; *++sp = self->args[*cp]; break; case GET_CONSTANT: cp++; *++sp = self->constants[*cp]; break; case ADD_INT: sp[-1].i += sp[0].i; sp--; break; case ADD_FLOAT: sp[-1].f += sp[0].f; sp--; break; case ADD_COMPLEX: sp[-1].c.real += sp[0].c.real; sp[-1].c.imag += sp[0].c.imag; sp--; break; case SUB_INT: sp[-1].i -= sp[0].i; sp--; break; case SUB_FLOAT: sp[-1].f -= sp[0].f; sp--; break; case SUB_COMPLEX: sp[-1].c.real -= sp[0].c.real; sp[-1].c.imag -= sp[0].c.imag; sp--; break; case MUL_INT: sp[-1].i *= sp[0].i; sp--; break; case MUL_FLOAT: sp[-1].f *= sp[0].f; sp--; break; case MUL_COMPLEX: sp[-1].c = c_prod(sp[-1].c, sp[0].c); sp--; break; case DIV_INT: sp[-1].i /= sp[0].i; sp--; break; case DIV_FLOAT: sp[-1].f /= sp[0].f; sp--; break; case DIV_COMPLEX: sp[-1].c = c_quot(sp[-1].c, sp[0].c); sp--; break; case INTEGER_FLOAT: sp->f = sp->i; break; case INTEGER_COMPLEX: sp->c.real = sp->i; sp->c.imag = 0.; break; case FLOAT_INTEGER: sp->i = sp->f; break; case FLOAT_COMPLEX: sp->c.real = sp->f; sp->c.imag = 0.; break; case COMPLEX_INTEGER: sp->i = sp->c.real; break; case COMPLEX_FLOAT: sp->f = sp->c.real; break; case NEG_INT: sp->i = -sp->i; break; case NEG_FLOAT: sp->f = -sp->f; break; case NEG_COMPLEX: sp->c.real = -sp->c.real; sp->c.imag = -sp->c.imag; break; case ABS_INT: if (sp->i < 0) sp->i = -sp->i; break; case ABS_FLOAT: if (sp->f < 0) sp->f = -sp->f; break; case ABS_COMPLEX: sp->f = hypot(sp->c.real, sp->c.imag); break; case ACOS_FLOAT: sp->f = acos(sp->f); break; case ACOS_COMPLEX: sp->c = c_acos(sp->c); break; case ACOSH_FLOAT: sp->f = acosh(sp->f); break; case ACOSH_COMPLEX: sp->c = c_acosh(sp->c); break; case ASIN_FLOAT: sp->f = asin(sp->f); break; case ASIN_COMPLEX: sp->c = c_asin(sp->c); break; case ASINH_FLOAT: sp->f = asinh(sp->f); break; case ASINH_COMPLEX: sp->c = c_asinh(sp->c); break; case ATAN_FLOAT: sp->f = atan(sp->f); break; case ATAN_COMPLEX: sp->c = c_atan(sp->c); break; case ATANH_FLOAT: sp->f = atanh(sp->f); break; case ATANH_COMPLEX: sp->c = c_atanh(sp->c); break; case COS_FLOAT: sp->f = cos(sp->f); break; case COS_COMPLEX: sp->c = c_cos(sp->c); break; case COSH_FLOAT: sp->f = cosh(sp->f); break; case COSH_COMPLEX: sp->c = c_cosh(sp->c); break; case EXP_FLOAT: sp->f = exp(sp->f); break; case EXP_COMPLEX: sp->c = c_exp(sp->c); break; case LOG_FLOAT: sp->f = log(sp->f); break; case LOG_COMPLEX: sp->c = c_log(sp->c); break; case LOG10_FLOAT: sp->f = log10(sp->f); break; case LOG10_COMPLEX: sp->c = c_log10(sp->c); break; case SIN_FLOAT: sp->f = sin(sp->f); break; case SIN_COMPLEX: sp->c = c_sin(sp->c); break; case SINH_FLOAT: sp->f = sinh(sp->f); break; case SINH_COMPLEX: sp->c = c_sinh(sp->c); break; case SQRT_FLOAT: sp->f = sqrt(sp->f); break; case SQRT_COMPLEX: sp->c = c_sqrt(sp->c); break; case TAN_FLOAT: sp->f = tan(sp->f); break; case TAN_COMPLEX: sp->c = c_tan(sp->c); break; case TANH_FLOAT: sp->f = tanh(sp->f); break; case TANH_COMPLEX: sp->c = c_tanh(sp->c); break; default: sprintf(text, "unknown opcode %d", *cp); PyErr_SetString(ErrorObject, text); return NULL; break; } cp++; } if (sp != stack) { PyErr_SetString(ErrorObject, "stack error"); return NULL; } switch(self->type) { case INTEGER: result = (PyObject *)PyInt_FromLong(sp->i); break; case FLOAT: result = (PyObject *)PyFloat_FromDouble(sp->f); break; case COMPLEX: result = (PyObject *)PyComplex_FromCComplex(sp->c); break; default: Py_INCREF(None); result = None; } return result; } /* Arithmetic */ static char numexpr_commontype(type1, type2) char type1; char type2; { return max(type1, type2); } static PyNumExprObject * numexpr_binary_op(a, b, type, op) PyNumExprObject *a; PyNumExprObject *b; char type; int op; { PyNumExprObject *r = new_numexprobject(); int error = 0; int i, j; int *cp; if (r != NULL) { r->type = type; r->nargs = max(a->nargs, b->nargs); r->nconstants = a->nconstants + b->nconstants; r->stacksize = max(a->stacksize, b->stacksize) + 1; r->codesize = a->codesize + b->codesize + (a->type != type) + (b->type != type) + 1; if ((r->argtypes = PyMem_NEW(char, r->nargs+1)) != NULL) { for (i = 0; i < min(a->nargs, b->nargs); i++) { if (a->argtypes[i] == ' ') r->argtypes[i] = b->argtypes[i]; else if (b->argtypes[i] == ' ') r->argtypes[i] = a->argtypes[i]; else if (a->argtypes[i] == b->argtypes[i]) r->argtypes[i] = a->argtypes[i]; else { PyErr_SetString(ErrorObject, "inconsistent argument lists"); error = 1; } } for (; i < r->nargs; i++) if (i < a->nargs) r->argtypes[i] = a->argtypes[i]; else r->argtypes[i] = b->argtypes[i]; r->argtypes[i] = '\0'; } else error = 1; if ((r->args = PyMem_NEW(stack_entry, r->nargs)) == NULL) error = 1; if ((r->constants = PyMem_NEW(stack_entry, r->nconstants)) != NULL) { for (i = 0; i < a->nconstants; i++) r->constants[i] = a->constants[i]; for (j = 0; j < b->nconstants; i++, j++) r->constants[i] = b->constants[j]; } else error = 1; if ((r->code = PyMem_NEW(int, r->codesize)) != NULL) { cp = r->code; for (i = 0; i < a->codesize; i++) *cp++ = a->code[i]; if (a->type != type) *cp++ = TYPE_CAST + a->type + NTYPES*type; for (i = 0; i < b->codesize;) { *cp++ = j = b->code[i++]; if (j == GET_CONSTANT) { *cp++ = b->code[i++] + a->nconstants; } else if (j == GET_ARG) { *cp++ = b->code[i++]; } } if (b->type != type) *cp++ = TYPE_CAST + b->type + NTYPES*type; *cp = op; } else error = 1; if ((r->stack = PyMem_NEW(stack_entry, r->stacksize)) == NULL) error = 1; if (error) { numexpr_dealloc(r); r = NULL; } } return r; } static PyNumExprObject * numexpr_unary_op(a, op) PyNumExprObject *a; int op; { PyNumExprObject *r = new_numexprobject(); int error = 0; int i; int *cp; if (r != NULL) { r->type = a->type; r->nargs = a->nargs; r->nconstants = a->nconstants; r->stacksize = a->stacksize; r->codesize = a->codesize + 1; if ((r->argtypes = PyMem_NEW(char, r->nargs+1)) != NULL) strcpy(r->argtypes, a->argtypes); else error = 1; if ((r->args = PyMem_NEW(stack_entry, r->nargs)) == NULL) error = 1; if ((r->constants = PyMem_NEW(stack_entry, r->nconstants)) != NULL) for (i = 0; i < a->nconstants; i++) r->constants[i] = a->constants[i]; else error = 1; if ((r->code = PyMem_NEW(int, r->codesize)) != NULL) { cp = r->code; for (i = 0; i < a->codesize; i++) *cp++ = a->code[i]; *cp = op; } if ((r->stack = PyMem_NEW(stack_entry, r->stacksize)) == NULL) error = 1; if (error) { numexpr_dealloc(r); r = NULL; } } return r; } static PyObject * numexpr_add(a, b) PyNumExprObject *a; PyNumExprObject *b; { char type = numexpr_commontype(a->type, b->type); return (PyObject *)numexpr_binary_op(a, b, type, ADD+type); } static PyObject * numexpr_sub(a, b) PyNumExprObject *a; PyNumExprObject *b; { char type = numexpr_commontype(a->type, b->type); return (PyObject *)numexpr_binary_op(a, b, type, SUB+type); } static PyObject * numexpr_mul(a, b) PyNumExprObject *a; PyNumExprObject *b; { char type = numexpr_commontype(a->type, b->type); return (PyObject *)numexpr_binary_op(a, b, type, MUL+type); } static PyObject * numexpr_div(a, b) PyNumExprObject *a; PyNumExprObject *b; { char type = numexpr_commontype(a->type, b->type); return (PyObject *)numexpr_binary_op(a, b, type, DIV+type); } static PyObject * numexpr_pow(a, b) PyNumExprObject *a; PyNumExprObject *b; { char type = numexpr_commontype(a->type, b->type); return (PyObject *)numexpr_binary_op(a, b, type, POW+type); } static PyObject * numexpr_neg(a) PyNumExprObject *a; { return (PyObject *)numexpr_unary_op(a, NEG+a->type); } static PyObject * numexpr_pos(a) PyNumExprObject *a; { Py_INCREF(a); return (PyObject *)a; } static PyObject * numexpr_abs(a) PyNumExprObject *a; { PyNumExprObject *r = numexpr_unary_op(a, ABS+a->type); if (r != NULL && r->type == COMPLEX) r->type = FLOAT; return (PyObject *)r; } static int numexpr_nonzero(a) PyNumExprObject *a; { return 1; } /* Type conversion */ static void numexpr_convert(value, type_in, type_out) stack_entry *value; int type_in; int type_out; { switch (type_in) { case INTEGER: if (type_out == FLOAT) value->f = value->i; else value->c.real = value->i; value->c.imag = 0; break; case FLOAT: if (type_out == INTEGER) value->i = value->f; else value->c.real = value->f; value->c.imag = 0; break; case COMPLEX: if (type_out == INTEGER) value->i = value->c.real; else value->f = value->c.real; break; } } /* Type coercion: Numbers are converted to constant expressions */ static PyObject * numexpr_const(value, type_in, type_out) stack_entry value; int type_in, type_out; { PyNumExprObject *rv; int error = 0; rv = new_numexprobject(); if (rv != NULL) { rv->nargs = 0; rv->nconstants = 1; rv->codesize = 2; rv->stacksize = 1; rv->type = type_out; if (type_in != type_out) numexpr_convert(&value, type_in, type_out); if ((rv->constants = PyMem_NEW(stack_entry, rv->nconstants)) != NULL) *rv->constants = value; else error = 1; if ((rv->code = PyMem_NEW(int, rv->codesize)) != NULL) { rv->code[0] = GET_CONSTANT; rv->code[1] = 0; } else error = 1; if ((rv->stack = PyMem_NEW(stack_entry, rv->stacksize)) == NULL) error = 1; if (error) { numexpr_dealloc(rv); rv = NULL; } } return (PyObject *)rv; } static void numexpr_coercenumber(value, type, number, numexpr) stack_entry value; int type; PyNumExprObject **number; PyNumExprObject **numexpr; { int result_type = numexpr_commontype(type, (*numexpr)->type); *number = (PyNumExprObject *)numexpr_const(value, type, result_type); if ((*numexpr)->type != result_type) *numexpr = numexpr_unary_op(*numexpr, TYPE_CAST + (*numexpr)->type + NTYPES*result_type); else Py_INCREF(*numexpr); } static int numexpr_coerce(a, b) PyObject **a; PyObject **b; { PyObject **number; PyNumExprObject **numexpr; stack_entry value; if (PyNumExpr_Check(*a)) { numexpr = (PyNumExprObject **)a; number = b; } else { numexpr = (PyNumExprObject **)b; number = a; } if (PyInt_Check(*number)) { value.i = PyInt_AsLong(*number); numexpr_coercenumber(value, INTEGER, number, numexpr); return 0; } else if (PyFloat_Check(*number)) { value.f = PyFloat_AsDouble(*number); numexpr_coercenumber(value, FLOAT, number, numexpr); return 0; } else if (PyComplex_Check(*number)) { value.c = PyComplex_AsCComplex(*number); numexpr_coercenumber(value, COMPLEX, number, numexpr); return 0; } else return 1; } /* Module documentation string */ static char PyNumExpr_Type__doc__[] = "Compiled numerical expressions"; /* Type definition */ static PyNumberMethods numexpr_as_number = { (binaryfunc)numexpr_add, /*nb_add*/ (binaryfunc)numexpr_sub, /*nb_subtract*/ (binaryfunc)numexpr_mul, /*nb_multiply*/ (binaryfunc)numexpr_div, /*nb_divide*/ 0, /*nb_remainder*/ 0, /*nb_divmod*/ (ternaryfunc)numexpr_pow, /*nb_power*/ (unaryfunc)numexpr_neg, /*nb_negative*/ (unaryfunc)numexpr_pos, /*nb_positive*/ (unaryfunc)numexpr_abs, /*nb_absolute*/ (inquiry)numexpr_nonzero, /*nb_nonzero*/ 0, /*nb_invert*/ 0, /*nb_lshift*/ 0, /*nb_rshift*/ 0, /*nb_and*/ 0, /*nb_xor*/ 0, /*nb_or*/ (coercion)numexpr_coerce, /*nb_coerce*/ 0, /*nb_int*/ 0, /*nb_long*/ 0, /*nb_float*/ 0, /*nb_oct*/ 0, /*nb_hex*/ }; static PyTypeObject PyNumExpr_Type = { PyObject_HEAD_INIT(&PyType_Type) 0, /*ob_size*/ "numexpr", /*tp_name*/ sizeof(PyNumExprObject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ (destructor)numexpr_dealloc, /*tp_dealloc*/ 0, /*tp_print*/ (getattrfunc)numexpr_getattr, /*tp_getattr*/ 0, /*tp_setattr*/ 0, /*tp_compare*/ (reprfunc)numexpr_repr, /*tp_repr*/ &numexpr_as_number, /*tp_as_number*/ 0, /*tp_as_sequence*/ 0, /*tp_as_mapping*/ 0, /*tp_hash*/ (ternaryfunc)numexpr_call, /*tp_call*/ (reprfunc)numexpr_repr, /*tp_str*/ /* Space for future expansion */ 0L,0L,0L,0L, PyNumExpr_Type__doc__ /* Documentation string */ }; /* --------------------------------------------------------------------- */ /* Return a new variable object. The (integer) argument indicates the position of the variable in the argument list of the compiled expression; it defaults to zero. */ static PyNumExprObject * new_variable(args) PyObject *args; { PyNumExprObject *rv; int argnum = 0; int error = 0; int i; PyArg_ParseTuple(args, "|i:numexpr variable", &argnum); rv = new_numexprobject(); if (rv != NULL) { rv->nargs = argnum + 1; rv->stacksize = 1; rv->codesize = 2; if ((rv->argtypes = PyMem_NEW(char, rv->nargs+1)) != NULL) { for (i = 0; i < rv->nargs; i++) rv->argtypes[i] = ' '; rv->argtypes[i] = '\0'; } else error = 1; if ((rv->args = PyMem_NEW(stack_entry, rv->nargs)) == NULL) error = 1; if ((rv->code = PyMem_NEW(int, rv->codesize)) != NULL) { rv->code[0] = GET_ARG; rv->code[1] = argnum; } else error = 1; if ((rv->stack = PyMem_NEW(stack_entry, rv->stacksize)) == NULL) error = 1; if (error) { numexpr_dealloc(rv); rv = NULL; } } return rv; } static PyObject * numexpr_intvar(self, args) PyObject *self; /* Not used */ PyObject *args; { PyNumExprObject *rv = new_variable(args); if (rv != NULL) { rv->argtypes[rv->nargs-1] = 'l'; rv->type = INTEGER; } return (PyObject *)rv; } static PyObject * numexpr_floatvar(self, args) PyObject *self; /* Not used */ PyObject *args; { PyNumExprObject *rv = new_variable(args); if (rv != NULL) { rv->argtypes[rv->nargs-1] = 'd'; rv->type = FLOAT; } return (PyObject *)rv; } static PyObject * numexpr_complexvar(self, args) PyObject *self; /* Not used */ PyObject *args; { PyNumExprObject *rv = new_variable(args); if (rv != NULL) { rv->argtypes[rv->nargs-1] = 'D'; rv->type = COMPLEX; } return (PyObject *)rv; } /* List of functions defined in the module */ static struct PyMethodDef numexpr_functions[] = { {"FloatVariable", numexpr_floatvar, 1}, {"IntegerVariable", numexpr_intvar, 1}, {"ComplexVariable", numexpr_complexvar, 1}, {NULL, NULL} /* sentinel */ }; /* Various math functions */ #if 0 static PyObject * numexpr_sqrt(self) PyNumExprObject *self; { PyNumExprObject *temp = self; PyNumExprObject *r; if (self->type == INTEGER) { temp = numexpr_unary_op(self, INTEGER_FLOAT); temp->type = FLOAT; } if (temp->type == COMPLEX) r = numexpr_unary_op(temp, SQRT_COMPLEX); else r = numexpr_unary_op(temp, SQRT_FLOAT); if (temp != self) numexpr_dealloc(temp); return (PyObject *)r; } #endif static PyObject * numexpr_mathfunc(arg, float_op, complex_op) PyNumExprObject *arg; int float_op; int complex_op; { PyNumExprObject *temp = arg; PyNumExprObject *r; if (arg->type == INTEGER) { temp = numexpr_unary_op(arg, INTEGER_FLOAT); temp->type = FLOAT; } if (temp->type == COMPLEX) r = numexpr_unary_op(temp, complex_op); else r = numexpr_unary_op(temp, float_op); if (temp != arg) numexpr_dealloc(temp); return (PyObject *)r; } #define add_mathfunc(func_name, float_op, complex_op) \ static PyObject * \ func_name(self) \ PyNumExprObject *self; \ { \ return numexpr_mathfunc(self, float_op, complex_op); \ } add_mathfunc(numexpr_acos, ACOS_FLOAT, ACOS_COMPLEX) add_mathfunc(numexpr_acosh, ACOSH_FLOAT, ACOSH_COMPLEX) add_mathfunc(numexpr_asin, ASIN_FLOAT, ASIN_COMPLEX) add_mathfunc(numexpr_asinh, ASINH_FLOAT, ASINH_COMPLEX) add_mathfunc(numexpr_atan, ATAN_FLOAT, ATAN_COMPLEX) add_mathfunc(numexpr_atanh, ATANH_FLOAT, ATANH_COMPLEX) add_mathfunc(numexpr_cos, COS_FLOAT, COS_COMPLEX) add_mathfunc(numexpr_cosh, COSH_FLOAT, COSH_COMPLEX) add_mathfunc(numexpr_exp, EXP_FLOAT, EXP_COMPLEX) add_mathfunc(numexpr_log, LOG_FLOAT, LOG_COMPLEX) add_mathfunc(numexpr_log10, LOG10_FLOAT, LOG10_COMPLEX) add_mathfunc(numexpr_sin, SIN_FLOAT, SIN_COMPLEX) add_mathfunc(numexpr_sinh, SINH_FLOAT, SINH_COMPLEX) add_mathfunc(numexpr_sqrt, SQRT_FLOAT, SQRT_COMPLEX) add_mathfunc(numexpr_tan, TAN_FLOAT, TAN_COMPLEX) add_mathfunc(numexpr_tanh, TANH_FLOAT, TANH_COMPLEX) /* List of methods of type "numexpr" */ static struct PyMethodDef numexpr_methods[] = { {"acos", (PyCFunction)numexpr_acos, 1}, {"acosh", (PyCFunction)numexpr_acosh, 1}, {"asin", (PyCFunction)numexpr_asin, 1}, {"asinh", (PyCFunction)numexpr_asinh, 1}, {"atan", (PyCFunction)numexpr_atan, 1}, {"atanh", (PyCFunction)numexpr_atanh, 1}, {"cos", (PyCFunction)numexpr_cos, 1}, {"cosh", (PyCFunction)numexpr_cosh, 1}, {"exp", (PyCFunction)numexpr_exp, 1}, {"log", (PyCFunction)numexpr_log, 1}, {"log10", (PyCFunction)numexpr_log10, 1}, {"sin", (PyCFunction)numexpr_sin, 1}, {"sinh", (PyCFunction)numexpr_sinh, 1}, {"sqrt", (PyCFunction)numexpr_sqrt, 1}, {"tan", (PyCFunction)numexpr_tan, 1}, {"tanh", (PyCFunction)numexpr_tanh, 1}, {NULL, NULL} /* sentinel */ }; /* Initialization function for the module (*must* be called initnumexpr) */ void initnumexpr() { PyObject *m, *d; /* Create the module and add the functions */ m = Py_InitModule("numexpr", numexpr_functions); /* Add some symbolic constants to the module */ d = PyModule_GetDict(m); ErrorObject = PyString_FromString("NumExprError"); PyDict_SetItemString(d, "error", ErrorObject); /* Check for errors */ if (PyErr_Occurred()) Py_FatalError("can't initialize module numexpr"); } ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 23 13:01:36 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA10645 for matrix-sig-people; Tue, 23 Jan 1996 13:01:36 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id NAA10641 for ; Tue, 23 Jan 1996 13:01:33 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA29105; Tue, 23 Jan 96 13:00:16 EST Date: Tue, 23 Jan 96 13:00:16 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601231800.AA29105@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] floating point equalities Sender: owner-matrix-sig@python.org Precedence: bulk In preparation for the 0.30 release (still on for Friday) I've been cleaning up the array math functions. I'm now working on improving the floating point equality operator. My understanding is that doing a == b is unreasonable for floating point numbers, and instead the test should be something like abs(a-b) < TOLERANCE*abs(a) Rather than have me try and rederive what this should look like from my vague memories of the IEEE floating point standard, I was wondering if anybody out there has a good chunk of code for doing this comparision? Thanks, Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 23 13:38:44 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA10917 for matrix-sig-people; Tue, 23 Jan 1996 13:38:44 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id NAA10913 for ; Tue, 23 Jan 1996 13:38:40 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa14437; 23 Jan 96 13:27 EST Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id NAA15424; Tue, 23 Jan 1996 13:26:25 -0500 Message-Id: <199601231826.NAA15424@monty> To: James Hugunin cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] floating point equalities In-reply-to: Your message of "Tue, 23 Jan 1996 13:00:16 EST." <9601231800.AA29105@baribal.LCS.MIT.EDU.LCS.MIT.EDU> References: <9601231800.AA29105@baribal.LCS.MIT.EDU.LCS.MIT.EDU> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Tue, 23 Jan 1996 13:26:24 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk > I'm now working on improving the floating point equality operator. My > understanding is that doing a == b is unreasonable for floating point > numbers, and instead the test should be something like > > abs(a-b) < TOLERANCE*abs(a) Hm. If you put this in the array module it would be a reasonable expectation that it would also work for ordinary Python floats, which it doesn't. Also, the tolerance may be application dependent or dependent on the part of the application that's doing this, so it may need to be specified in the operation. I'd advise to continue using C's '==' operator for floats (which is what ordinary Python floats do). Users who know enough to write numerical code should also know not to compare for equality. (What you could do is provide a method or function for comparisons where you can pass the tolerance in as an argument.) --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 23 13:49:52 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA10957 for matrix-sig-people; Tue, 23 Jan 1996 13:49:52 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id NAA10952 for ; Tue, 23 Jan 1996 13:49:50 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA29834; Tue, 23 Jan 96 13:48:21 EST Date: Tue, 23 Jan 96 13:48:21 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601231848.AA29834@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: guido@CNRI.Reston.VA.US Cc: matrix-sig@python.org In-Reply-To: <199601231826.NAA15424@monty> (message from Guido van Rossum on Tue, 23 Jan 1996 13:26:24 -0500) Subject: Re: [PYTHON MATRIX-SIG] floating point equalities Sender: owner-matrix-sig@python.org Precedence: bulk From: Guido van Rossum > I'm now working on improving the floating point equality operator. My > understanding is that doing a == b is unreasonable for floating point > numbers, and instead the test should be something like > > abs(a-b) < TOLERANCE*abs(a) Hm. If you put this in the array module it would be a reasonable expectation that it would also work for ordinary Python floats, which it doesn't. Also, the tolerance may be application dependent or dependent on the part of the application that's doing this, so it may need to be specified in the operation. I'd advise to continue using C's '==' operator for floats (which is what ordinary Python floats do). Users who know enough to write numerical code should also know not to compare for equality. (What you could do is provide a method or function for comparisons where you can pass the tolerance in as an argument.) I'm still curious to see what the more numerically inclined folks have to say on this, but you're obviously right that this should be provided as an additional function, not the default comparision behavior. I'd like to have a method approxEqual (or some similar name) so that a.approxEqual(b) will "do the right thing". I had believed that there was some sort of consensus in the numerical programming community on what the right thing is for floating point numbers, if I'm wrong on this, I'd like to know that too. Note: part of why this came up is the fact that on my development system (1j)**2 == (-1+1.22460635382e-16j). I'd really like to be able to have some way to say approxEqual((1j)**2, -1) and have it come out true (OK, I have an ad hoc solution that works, what I want is the "right" solution if there is one out there). -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 23 14:08:28 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA11079 for matrix-sig-people; Tue, 23 Jan 1996 14:08:28 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id OAA11073 for ; Tue, 23 Jan 1996 14:08:24 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id OAA12760 (8.6.11/IDA-1.6); Tue, 23 Jan 1996 14:05:52 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA25514; Tue, 23 Jan 1996 14:05:51 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA01142; Tue, 23 Jan 1996 14:05:49 -0500 Date: Tue, 23 Jan 1996 14:05:49 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601231905.OAA01142@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: matrix-sig@python.org In-reply-to: <9601231800.AA29105@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] floating point equalities Sender: owner-matrix-sig@python.org Precedence: bulk I'm now working on improving the floating point equality operator. My understanding is that doing a == b is unreasonable for floating point numbers, and instead the test should be something like abs(a-b) < TOLERANCE*abs(a) I don't agree. First, there are cases where testing for equality makes sense (e.g. if you know that the number represents an integer, or if you want to detect an underflow by comparing to zero). Second, standard Python floats use normal equality, so arrays should behave identically. On the other hand, having a test with tolerance would be very useful and make code more readable. APL has such a feature (using a global variable for the tolerance, which for Python is not such a great idea), which I like a lot. And since the comparison "operators" for arrays are methods anyway, it is easy to have both, with an optional second argument for the tolerance. I.e. you would write a.equal(b) for a normal equality test and a.equal(b, 1e-6) for a test with a given tolerance. As for the form of a test with tolerance, I am not sure whether there is a generally accepted best one. I have seen (and used) the form you have given above, but also the more symmetric form abs(a-b) < TOLERANCE*(abs(a)+abs(b)) (although I wonder whether this is ever necessary). Hopefully the mathematicians among us can say something more definite. And now for something completely different... While rereading the discussion about the beta release, I noticed that there was no mention at all of how to deal with functions of finite rank. Although there are none in the current package, they will appear in the application-specific packages to be written later (e.g. linear algebra), so there should be a general mechanism for dealing with them. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 23 14:40:52 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA11454 for matrix-sig-people; Tue, 23 Jan 1996 14:40:52 -0500 Received: from umbc7.umbc.edu (root@f-umbc7.umbc.edu [130.85.3.7]) by python.org (8.6.12/8.6.12) with ESMTP id OAA11450 for ; Tue, 23 Jan 1996 14:40:47 -0500 Received: from [130.85.140.14] (mitsu.umbc.edu [130.85.140.14]) by umbc7.umbc.edu (8.6.12/Umbc) with SMTP id OAA11372 for ; Tue, 23 Jan 1996 14:39:24 -0500 X-Sender: strow@umbc7.umbc.edu Message-Id: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Tue, 23 Jan 1996 14:40:48 -0500 To: matrix-sig@python.org From: strow@umbc.edu (L. Larrabee Strow) Subject: Re: [PYTHON MATRIX-SIG] floating point equalities Sender: owner-matrix-sig@python.org Precedence: bulk >In preparation for the 0.30 release (still on for Friday) I've been >cleaning up the array math functions. > >I'm now working on improving the floating point equality operator. My >understanding is that doing a == b is unreasonable for floating point >numbers, and instead the test should be something like > >abs(a-b) < TOLERANCE*abs(a) > >Rather than have me try and rederive what this should look like from >my vague memories of the IEEE floating point standard, I was wondering >if anybody out there has a good chunk of code for doing this >comparision? > >Thanks, Jim > Let the user decide how to compare for equality. We are used to avoiding the a==b problem. It would probably be a mistake to "help" us too much, since we might get lazy and make even more subtle mistakes. -- L. Larrabee Strow E-mail: strow@umbc.edu Department of Physics FAX : (410) 455-1072 University of Maryland Baltimore County Phone : (410) 455-2528 5401 Wilkens Avenue Baltimore, MD 21228-5398 ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 23 14:41:54 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA11467 for matrix-sig-people; Tue, 23 Jan 1996 14:41:54 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id OAA11463 for ; Tue, 23 Jan 1996 14:41:52 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa16045; 23 Jan 96 14:38 EST Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id OAA15633; Tue, 23 Jan 1996 14:37:35 -0500 Message-Id: <199601231937.OAA15633@monty> To: James Hugunin cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] floating point equalities In-reply-to: Your message of "Tue, 23 Jan 1996 13:48:21 EST." <9601231848.AA29834@baribal.LCS.MIT.EDU.LCS.MIT.EDU> References: <9601231848.AA29834@baribal.LCS.MIT.EDU.LCS.MIT.EDU> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Tue, 23 Jan 1996 14:37:35 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk > I'd like to have a method approxEqual (or some similar name) so that > a.approxEqual(b) will "do the right thing". I had believed that there > was some sort of consensus in the numerical programming community on > what the right thing is for floating point numbers, if I'm wrong on > this, I'd like to know that too. Cool. > Note: part of why this came up is the fact that on my development > system (1j)**2 == (-1+1.22460635382e-16j). I'd really like to be able > to have some way to say approxEqual((1j)**2, -1) and have it come out > true (OK, I have an ad hoc solution that works, what I want is the > "right" solution if there is one out there). I bet this is because in x**y, Python currently coerces y, meaning you were really calculating 1.0j ** 2.0j... (Note that the (by now pathetically incomplete) Python test suite has a similar problem and solution when testing the pow() function.) --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 23 15:30:45 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA00294 for matrix-sig-people; Tue, 23 Jan 1996 15:30:45 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id PAA00290 for ; Tue, 23 Jan 1996 15:30:40 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id PAA16636 (8.6.11/IDA-1.6); Tue, 23 Jan 1996 15:29:34 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA17347; Tue, 23 Jan 1996 15:29:33 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA07104; Tue, 23 Jan 1996 15:29:31 -0500 Date: Tue, 23 Jan 1996 15:29:31 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601232029.PAA07104@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: guido@CNRI.Reston.VA.US, matrix-sig@python.org In-reply-to: <9601231848.AA29834@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] floating point equalities Sender: owner-matrix-sig@python.org Precedence: bulk Note: part of why this came up is the fact that on my development system (1j)**2 == (-1+1.22460635382e-16j). I'd really like to be able to have some way to say approxEqual((1j)**2, -1) and have it come out Are you sure that you have installed the latest version of my complex object? It should give 1j**2 == -1., exactly. The reason is that complex_power() detects that the exponent is actually an integer (no fractional or imaginary part), and then calls c_powi(), which in turn calls c_powu() because the exponent is positive and less than 100. c_powu() reduces the power calculation to an optimized sequence of multiplications. Therefore 1j**2 is exactly equivalent to 1j*1j. (And yes, I have traced this example with a debugger right now because I was a bit worried). What does 1j*1j give on your system? It ought to be exactly -1., since there are no round-off errors in this calculation. I realize that this has nothing to do with doing comparisons in general, of course... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 23 15:39:41 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA00411 for matrix-sig-people; Tue, 23 Jan 1996 15:39:41 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id PAA00407 for ; Tue, 23 Jan 1996 15:39:38 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA01063; Tue, 23 Jan 96 15:39:21 EST Date: Tue, 23 Jan 96 15:39:21 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601232039.AA01063@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: hinsenk@ere.umontreal.ca Cc: matrix-sig@python.org In-Reply-To: <199601232029.PAA07104@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Subject: Re: [PYTHON MATRIX-SIG] floating point equalities Sender: owner-matrix-sig@python.org Precedence: bulk From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Note: part of why this came up is the fact that on my development system (1j)**2 == (-1+1.22460635382e-16j). I'd really like to be able to have some way to say approxEqual((1j)**2, -1) and have it come out Are you sure that you have installed the latest version of my complex object? It should give 1j**2 == -1., exactly. The reason is that complex_power() detects that the exponent is actually an integer (no fractional or imaginary part), and then calls c_powi(), which in turn calls c_powu() because the exponent is positive and less than 100. c_powu() reduces the power calculation to an optimized sequence of multiplications. Therefore 1j**2 is exactly equivalent to 1j*1j. (And yes, I have traced this example with a debugger right now because I was a bit worried). What does 1j*1j give on your system? It ought to be exactly -1., since there are no round-off errors in this calculation. I realize that this has nothing to do with doing comparisons in general, of course... My fault, I'm still running with an old version of your patch. I'll install the new one before I do much more testing. (Still, this problem comes up in many other places). -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 23 15:41:36 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA00432 for matrix-sig-people; Tue, 23 Jan 1996 15:41:36 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id PAA00428 for ; Tue, 23 Jan 1996 15:41:32 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id PAA16955 (8.6.11/IDA-1.6); Tue, 23 Jan 1996 15:40:00 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA19238; Tue, 23 Jan 1996 15:39:59 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA07827; Tue, 23 Jan 1996 15:39:57 -0500 Date: Tue, 23 Jan 1996 15:39:57 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601232039.PAA07827@cyclone.ERE.UMontreal.CA> To: guido@CNRI.Reston.VA.US CC: jjh@goldilocks.lcs.mit.edu, matrix-sig@python.org In-reply-to: <199601231937.OAA15633@monty> (message from Guido van Rossum on Tue, 23 Jan 1996 14:37:35 -0500) Subject: Re: [PYTHON MATRIX-SIG] floating point equalities Sender: owner-matrix-sig@python.org Precedence: bulk I bet this is because in x**y, Python currently coerces y, meaning you were really calculating 1.0j ** 2.0j... (Note that the (by now pathetically incomplete) Python test suite has a similar problem and solution when testing the pow() function.) For complex numbers, 1j**2.0 is calculated as 1j**2. And with my set of patches, the same happens for real numbers. You still don't get the full efficiency of an integer exponentiation, but at least the accuracy. That is of course no argument for not fixing the pow() coercion problem... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 23 17:00:02 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA01113 for matrix-sig-people; Tue, 23 Jan 1996 17:00:02 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id QAA01108 for ; Tue, 23 Jan 1996 16:59:57 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA04093; Tue, 23 Jan 96 13:59:51 PST Received: from kristen (localhost) by kristen.llnl.gov (5.x/SMI-SVR4) id AA20246; Tue, 23 Jan 1996 13:59:48 -0800 Message-Id: <31055A53.739E@llnl.gov> Date: Tue, 23 Jan 1996 13:59:47 -0800 From: "Paul. Dubois" Organization: Lawrence Livermore National Laboratory X-Mailer: Mozilla 2.0b5 (X11; I; SunOS 5.4 sun4m) Mime-Version: 1.0 To: James Hugunin Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] floating point equalities References: <9601231848.AA29834@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk Guido is clearly right. As for how to do approxEqual(a,b), there is no one right answer. There are two ways of measuring error, relative and absolute. Relative is usually defined as abs(a-b)/max(abs(a),abs(b)). Absolute is just abs(a-b). Suppose the two numbers are 10**-15 and 10**-16. Depending on what you are doing, the difference between these two is either (a) a factor of 10, which sounds "large", or (b) about 10**-15, which sounds small. Many people use a "mixed" critera, considering two numbers equal if their relative difference is small or their absolute difference is (perhaps a different size of) small. You see tests like: abs(a-b) < rel_tol * abs(a) + abs_tol But small is still hard to be precise about, since it really depends on context. If you get a relative error of 1.e-8 solving a linear system it might be expected, but that same number might be a sign of trouble in other contexts where errors more like 1.e-14 are normal. One solution is say a class RealNumberComparer, which contains attributes maximum_relative_error, maximum_absolute_error, routines to set these (with some reasonable defaults set on creation) and an equals(a,b) to carry out the test. -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Jan 24 11:18:12 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA06356 for matrix-sig-people; Wed, 24 Jan 1996 11:18:12 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id LAA06352 for ; Wed, 24 Jan 1996 11:18:08 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id QAA24860 for matrix-sig@python.org; Wed, 24 Jan 1996 16:18:20 GMT From: "Jim Fulton, U.S. Geological Survey" Message-Id: <9601241118.ZM24858@dsdbqvarsa.er.usgs.gov> Date: Wed, 24 Jan 1996 11:18:19 -0500 X-Phone: (703) 648-5622 X-Face: (;vf"lccr&V=y; Thu, 25 Jan 1996 07:34:48 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA06874; Thu, 25 Jan 96 07:34:25 EST Date: Thu, 25 Jan 96 07:34:25 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601251234.AA06874@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: jfulton@usgs.gov Cc: matrix-sig@python.org In-Reply-To: <9601241118.ZM24858@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] Summary for main list? Sender: owner-matrix-sig@python.org Precedence: bulk There is no complete summary of the work anywhere. At this point, the best summary is going to probably be the documentation for the NumericPython extension package (completed in time for the first beta release). I'm still on the schedule I recently included in my proposals for a final set of naming conventions, etc. This includes a general beta release (advertised in the main newsgroup) on 2/12. Personally, I'd just as soon wait until then to bring the main python group "up to date". If somebody wants to post a generic message reminding people of the existence of the matrix-sig and its purpose, that might be useful. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Jan 25 08:13:27 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id IAA12777 for matrix-sig-people; Thu, 25 Jan 1996 08:13:27 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id IAA12773 for ; Thu, 25 Jan 1996 08:13:24 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA07030; Thu, 25 Jan 96 08:13:02 EST Date: Thu, 25 Jan 96 08:13:02 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601251313.AA07030@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] type coercion one more time Sender: owner-matrix-sig@python.org Precedence: bulk I've been working to design a stable version of automatic type coercion that everybody (including myself) can be happy with. The following is what I have. Much of this is in line with the C rules for type coercion (at least according to my old K&R). I will use the standard C type names (double means a python float). Only the following three automatic coercions are allowed: 1) Any integer type can be coerced to a float, a double, a complex float or a complex double. 2) A float can only be coerced to a complex float. 3) A double can only be coerced to a complex double. Explanation: The following type hierarchy is assumed: INTEGER < FLOAT < COMPLEX Within each level there are a number of possible precisions. No automatic coercion will take place between these precisions. This remains completely consistent with the existing python automatic type coercion. Examples: a_t = array([x,y,z,...], 't') # Where t is a legal typecode 1 - signed byte l - long f - float d - double F - complex float D - complex double a_l * a_d = a_d a_f * a_l = a_f a_f * a_d = TypeError Exception raised a_d * a_D = a_D a_1 * a_l = TypeError Exception raised Comments? - Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Jan 25 09:53:21 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA13174 for matrix-sig-people; Thu, 25 Jan 1996 09:53:21 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id JAA13170 for ; Thu, 25 Jan 1996 09:53:17 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id JAA15926 (8.6.11/IDA-1.6); Thu, 25 Jan 1996 09:52:03 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA10487; Thu, 25 Jan 1996 09:52:02 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA02955; Thu, 25 Jan 1996 09:52:01 -0500 Date: Thu, 25 Jan 1996 09:52:01 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601251452.JAA02955@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: matrix-sig@python.org In-reply-to: <9601251313.AA07030@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] type coercion one more time Sender: owner-matrix-sig@python.org Precedence: bulk Only the following three automatic coercions are allowed: 1) Any integer type can be coerced to a float, a double, a complex float or a complex double. 2) A float can only be coerced to a complex float. 3) A double can only be coerced to a complex double. Sounds OK for me. This remains completely consistent with the existing python automatic type coercion. Which is all I really care about... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Jan 25 10:06:33 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA13298 for matrix-sig-people; Thu, 25 Jan 1996 10:06:33 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id KAA13293 for ; Thu, 25 Jan 1996 10:06:31 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa09631; 25 Jan 96 9:59 EST Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id JAA21034; Thu, 25 Jan 1996 09:57:56 -0500 Message-Id: <199601251457.JAA21034@monty> To: James Hugunin cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] type coercion one more time In-reply-to: Your message of "Thu, 25 Jan 1996 08:13:02 EST." <9601251313.AA07030@baribal.LCS.MIT.EDU.LCS.MIT.EDU> References: <9601251313.AA07030@baribal.LCS.MIT.EDU.LCS.MIT.EDU> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Thu, 25 Jan 1996 09:57:56 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk > No automatic coercion will take place between these precisions. This surprises me (especially since C does support float -> double coercions). Would you care to explain it? --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Jan 25 10:46:42 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA13544 for matrix-sig-people; Thu, 25 Jan 1996 10:46:42 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id KAA13540 for ; Thu, 25 Jan 1996 10:46:39 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA08514; Thu, 25 Jan 96 10:45:58 EST Date: Thu, 25 Jan 96 10:45:58 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601251545.AA08514@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: guido@CNRI.Reston.VA.US Cc: matrix-sig@python.org In-Reply-To: <199601251457.JAA21034@monty> (message from Guido van Rossum on Thu, 25 Jan 1996 09:57:56 -0500) Subject: Re: [PYTHON MATRIX-SIG] type coercion one more time Sender: owner-matrix-sig@python.org Precedence: bulk From: Guido van Rossum > No automatic coercion will take place between these precisions. This surprises me (especially since C does support float -> double coercions). Would you care to explain it? Naive users of the system are only expected to use arrays of longs, doubles, and complex doubles. I wanted to make sure that all of the automatic coercion rules would work for these most frequently used types so that they'll work just like the corresponding python scalars. On the other hand, I see the other types as being used for cases where the programmer really wants to use a particular precision, ie. 2-byte integers for dealing with 16-bit sound data. In a case like this it seems decidely unfriendly to automatically coerce to another precision. One can imagine dividing each sample in a sound by 2, and then trying to play it back and hearing garbage because it got automatically converted to an array of longs during the division. I do have one other possible set of rules that I would be happy with. These are basically the same as the last post, except: 4) Coercion to a higher precision is allowed within the main groupings. 5) The precision of a python scalar is allowed to change to match the other arguments (3.14 can be either a float or a double depending on which one is "desired"). I don't like this rule, but without it I think that it is too easy to get bitten by python scalars when writing "real" code. Does this make sense? Do people prefer this revised scheme? -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Jan 25 11:10:52 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA13771 for matrix-sig-people; Thu, 25 Jan 1996 11:10:52 -0500 Received: from atr-sw.atr-sw.atr.co.jp (atr-sw.atr-sw.atr.co.jp [133.186.20.117]) by python.org (8.6.12/8.6.12) with ESMTP id LAA13767 for ; Thu, 25 Jan 1996 11:10:44 -0500 Received: from ciris21.atr-sw.atr.co.jp (ciris21.atr-sw.atr.co.jp [133.186.20.41]) by atr-sw.atr-sw.atr.co.jp (8.6.11+2.5Wb2/3.4Wbeta3/95031616ATR-SW) with ESMTP id BAA29226; Fri, 26 Jan 1996 01:09:55 +0900 Received: from localhost by ciris21.atr-sw.atr.co.jp (940816.SGI.8.6.9/6.4J.6) id BAA02646; Fri, 26 Jan 1996 01:09:53 +0900 Message-Id: <199601251609.BAA02646@ciris21.atr-sw.atr.co.jp> To: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] type coercion one more time In-reply-to: Your message of "Thu, 25 Jan 1996 08:13:02 JST" References: <9601251313.AA07030@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Date: Fri, 26 Jan 1996 01:09:53 +0900 From: "Perry A. Stoll" Sender: owner-matrix-sig@python.org Precedence: bulk >>>>> "JH" == James Hugunin writes: JH> I've been working to design a stable version of automatic type JH> coercion that everybody (including myself) can be happy with. Sounds good. I offer my support for continuing something similar to the current scheme. JH> Only the following three automatic coercions are allowed: JH> 1) Any integer type can be coerced to a float, a double, a JH> complex float or a complex double. Does 'any integer' type mean byte,short,integer? If so, OK. JH> 2) A float can only be coerced to a complex float. JH> 3) A double can only be coerced to a complex double. JH> Explanation: JH> The following type hierarchy is assumed: JH> INTEGER < FLOAT < COMPLEX JH> Within each level there are a number of possible precisions. True, and it would be unrealistic to do arbitrary coercions. JH> No automatic coercion will take place between these JH> precisions. But there is no reason not to do coercion to a greater precision. See below. JH> This remains completely consistent with the existing python JH> automatic type coercion. To some extent. But there is no access in Python to most underlying C numeric types, for good or bad. Good in most cases, bad in C/Fortran interfacing cases. It seems like we're trying to add these types to Python - this could be difficult. JH> a_l * a_d = a_d JH> a_f * a_l = a_f JH> a_f * a_d = TypeError Exception raised JH> a_d * a_D = a_D JH> a_1 * a_l = TypeError Exception raised Raising exceptions in these circumstances seems a bit arbitrary. I think the source of your problem is the difference between Python's floats (i.e. C doubles) and C floats. I don't know any way to specify a C float in Python - does anyone? It seems the idealistic approach of having only one floating point representation in Python is at odds with peoples' desire to access C floats. Also, I don't see how the scheme you outline above will solve your problem Array_f([1.0,2.0,3.0]) * 2.0 = Array_f([2.0,4.0,6.0]) since '2.0' is interpreted as a Python 'float', i.e. a C double. From the scheme you propose above, this would raise an exception. Assuming it is too late to add '2.0f' to Python (or a better type specification scheme as outlined earlier on this list), I can't think of a way to get around this without explicitly doing an ungainly Matrix_f(2.0). Of course, if it matters this much, you could always do M_f = Matrix_f and have M_f(2.0) scattered through your code. The more I think about this (and since I'm thinking as I'm writing, this is at the expense of conciseness - sorry!), the more I think your situation is outside the Python norm and should really be viewed as such. Although it may not solve your problem of C floats, I think it makes more sense to allow conversion to any "higher" type, where "higher" means higher in the list of types, i.e. b < s < i < l < f < d Also: (b,s,i,l,f) <= C (b,s,i,l,f,d,C) <= D With the exception of the last bit, isn't this how the present (0.2) release works? Also, I don't recall anyone presenting a list of 'nice' names for the Matrix functions as Konrad has been advocating. I think having these is a good idea; the type code that we have now has always seemed somehow counter to (my view of) the Python 'do-it-the-Right-Way' attitude. If I actually list them all out here, maybe this will force the issue and start some discussion. Comments on the following? Array_Char, Array_UChar Array_Short, Array_UShort Array_Integer,Array_UInteger, Array_Float, Array_Double, Array_FloatComplex, Array_DoubleComplex 'Unsigned' instead of 'U'? XArray instead of Array_X? BTW, is the C multiarray object going to support all builtin numeric C types? Right now, there is supported for both signed and unsigned bytes, but not for unsigned shorts, ints, or longs - is that good enough? -Perry Stoll Research Associate, CSRL Advanced Telecommunications Research 2-2 Hikaridai, Seika-cho Soraku-gun, Kyoto 619-02 JAPAN FINGER: stoll@yoshino.atr.co.jp EMAIL: stoll@atr-sw.atr.co.jp PGP 2.6 Key fingerprint = AF 56 5C D8 5F 78 BA FD 21 6E 2A 68 C4 55 9E B0 ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Jan 25 11:34:56 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA14018 for matrix-sig-people; Thu, 25 Jan 1996 11:34:56 -0500 Received: from atr-sw.atr-sw.atr.co.jp (atr-sw.atr-sw.atr.co.jp [133.186.20.117]) by python.org (8.6.12/8.6.12) with ESMTP id LAA14014 for ; Thu, 25 Jan 1996 11:34:50 -0500 Received: from ciris21.atr-sw.atr.co.jp (ciris21.atr-sw.atr.co.jp [133.186.20.41]) by atr-sw.atr-sw.atr.co.jp (8.6.11+2.5Wb2/3.4Wbeta3/95031616ATR-SW) with ESMTP id BAA29546; Fri, 26 Jan 1996 01:34:01 +0900 Received: from localhost by ciris21.atr-sw.atr.co.jp (940816.SGI.8.6.9/6.4J.6) id BAA02667; Fri, 26 Jan 1996 01:33:59 +0900 Message-Id: <199601251633.BAA02667@ciris21.atr-sw.atr.co.jp> To: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] type coercion one more time In-reply-to: Your message of "Thu, 25 Jan 1996 10:45:58 JST" References: <9601251545.AA08514@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Date: Fri, 26 Jan 1996 01:33:59 +0900 From: "Perry A. Stoll" Sender: owner-matrix-sig@python.org Precedence: bulk >>>>> "JH" == James Hugunin writes: JH> From: Guido van Rossum >> No automatic coercion will take place between these precisions. JH> This surprises me (especially since C does support float -> JH> double coercions). Would you care to explain it? JH> Naive users of the system are only expected to use arrays of JH> longs, doubles, and complex doubles. I wanted to make sure JH> that all of the automatic coercion rules would work for these JH> most frequently used types so that they'll work just like the JH> corresponding python scalars. JH> On the other hand, I see the other types as being used for JH> cases where the programmer really wants to use a particular JH> precision, ie. 2-byte integers for dealing with 16-bit sound JH> data. In a case like this it seems decidely unfriendly to JH> automatically coerce to another precision. JH> One can imagine dividing each sample in a sound by 2, and then JH> trying to play it back and hearing garbage because it got JH> automatically converted to an array of longs during the JH> division. JH> I do have one other possible set of rules that I would be JH> happy with. These are basically the same as the last post, JH> except: JH> 4) Coercion to a higher precision is allowed within the main JH> groupings. JH> 5) The precision of a python scalar is allowed to change to JH> match the other arguments (3.14 can be either a float or a JH> double depending on which one is "desired"). JH> Do people prefer this revised scheme? So you try to coerce a scalar to the type of the matrix, using the same idea as in 4)? Is the following right? Array_SomeIntType * AnyIntType = Array_SomeIntType Array_SomeIntType * SomeFloatType =? Array_SomeFloatType Array_SomeFloatType * AnyIntType =? Array_SomeFloatType Array_SomeFloatType * OtherFloatType = Array_SomeFloatType That seems pretty reasonable. Do the '=?' cases sound ok? It should preserve coercions with the normal Python types. Users of specialized C type multiarrays must exercise a bit of caution that they don't cause overflow, but that isn't unreasonable. -Perry ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Jan 25 11:38:57 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA14055 for matrix-sig-people; Thu, 25 Jan 1996 11:38:57 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id LAA14051 for ; Thu, 25 Jan 1996 11:38:54 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa12114; 25 Jan 96 11:28 EST Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id LAA21529; Thu, 25 Jan 1996 11:27:05 -0500 Message-Id: <199601251627.LAA21529@monty> To: James Hugunin cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] type coercion one more time In-reply-to: Your message of "Thu, 25 Jan 1996 10:45:58 EST." <9601251545.AA08514@baribal.LCS.MIT.EDU.LCS.MIT.EDU> References: <9601251545.AA08514@baribal.LCS.MIT.EDU.LCS.MIT.EDU> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Thu, 25 Jan 1996 11:27:05 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk > > No automatic coercion will take place between these precisions. > > This surprises me (especially since C does support float -> double > coercions). Would you care to explain it? > > Naive users of the system are only expected to use arrays of longs, > doubles, and complex doubles. I wanted to make sure that all of the > automatic coercion rules would work for these most frequently used > types so that they'll work just like the corresponding python scalars. > > On the other hand, I see the other types as being used for cases where > the programmer really wants to use a particular precision, ie. 2-byte > integers for dealing with 16-bit sound data. In a case like this it > seems decidely unfriendly to automatically coerce to another > precision. > > One can imagine dividing each sample in a sound by 2, and then trying > to play it back and hearing garbage because it got automatically > converted to an array of longs during the division. I see your point, but I am uneasy with the solution. Shouldn't there be a "divide in place" operation for this case that coerces constants the other way? (I haven't followed all the details -- you may have it already or you may have a good reason for not having it.) I guess that you also don't check for overflow on such operations -- suppose I multiply each sample by 2 and it doesn't fit... This is also different than Python's own numeric operations, but again I can see the reason why (and I don't object in this case). I guess I won't complain too loudly as long as you also have a way to coerce an array of shorts into an array of longs. (BTW, on 64-bit machines, there are three relevant int sizes. Do you support this?) > I do have one other possible set of rules that I would be happy with. > These are basically the same as the last post, except: > > 4) Coercion to a higher precision is allowed within the main > groupings. > > 5) The precision of a python scalar is allowed to change to match the > other arguments (3.14 can be either a float or a double depending on > which one is "desired"). Some examples, please? Where else is coercion applied? It does sound like it could be a good compromise. > I don't like this rule, but without it I think that it is too easy to > get bitten by python scalars when writing "real" code. Agreed. > Does this make sense? > > Do people prefer this revised scheme? I'm not sure. More input, please... --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Jan 25 11:42:16 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA14075 for matrix-sig-people; Thu, 25 Jan 1996 11:42:16 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id LAA14071 for ; Thu, 25 Jan 1996 11:42:14 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA09236; Thu, 25 Jan 96 11:40:49 EST Date: Thu, 25 Jan 96 11:40:49 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601251640.AA09236@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: stoll@atr-sw.atr.co.jp Cc: matrix-sig@python.org In-Reply-To: <199601251633.BAA02667@ciris21.atr-sw.atr.co.jp> (stoll@atr-sw.atr.co.jp) Subject: Re: [PYTHON MATRIX-SIG] type coercion one more time Sender: owner-matrix-sig@python.org Precedence: bulk From: "Perry A. Stoll" JH> I do have one other possible set of rules that I would be JH> happy with. These are basically the same as the last post, JH> except: JH> 4) Coercion to a higher precision is allowed within the main JH> groupings. JH> 5) The precision of a python scalar is allowed to change to JH> match the other arguments (3.14 can be either a float or a JH> double depending on which one is "desired"). JH> Do people prefer this revised scheme? So you try to coerce a scalar to the type of the matrix, using the same idea as in 4)? Is the following right? Array_SomeIntType * AnyIntType = Array_SomeIntType Array_SomeIntType * SomeFloatType =? Array_SomeFloatType Array_SomeFloatType * AnyIntType =? Array_SomeFloatType Array_SomeFloatType * OtherFloatType = Array_SomeFloatType That seems pretty reasonable. Do the '=?' cases sound ok? It should preserve coercions with the normal Python types. Users of specialized C type multiarrays must exercise a bit of caution that they don't cause overflow, but that isn't unreasonable. -Perry This is exactly what I'm proposing (change the '=?' to '='). The fact that it seems to make sense to you is a good sign that it might be reasonable. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Jan 25 11:50:41 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA14144 for matrix-sig-people; Thu, 25 Jan 1996 11:50:41 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id LAA14140 for ; Thu, 25 Jan 1996 11:50:39 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa12528; 25 Jan 96 11:47 EST Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id LAA21663; Thu, 25 Jan 1996 11:46:14 -0500 Message-Id: <199601251646.LAA21663@monty> To: James Hugunin cc: stoll@atr-sw.atr.co.jp, matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] type coercion one more time In-reply-to: Your message of "Thu, 25 Jan 1996 11:40:49 EST." <9601251640.AA09236@baribal.LCS.MIT.EDU.LCS.MIT.EDU> References: <9601251640.AA09236@baribal.LCS.MIT.EDU.LCS.MIT.EDU> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Thu, 25 Jan 1996 11:46:14 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk OK, I'm convinced too. --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Jan 25 11:54:50 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA14182 for matrix-sig-people; Thu, 25 Jan 1996 11:54:50 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id LAA14178 for ; Thu, 25 Jan 1996 11:54:47 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA09280; Thu, 25 Jan 96 11:54:06 EST Date: Thu, 25 Jan 96 11:54:06 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601251654.AA09280@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: guido@CNRI.Reston.VA.US Cc: matrix-sig@python.org In-Reply-To: <199601251627.LAA21529@monty> (message from Guido van Rossum on Thu, 25 Jan 1996 11:27:05 -0500) Subject: Re: [PYTHON MATRIX-SIG] type coercion one more time Sender: owner-matrix-sig@python.org Precedence: bulk From: Guido van Rossum > One can imagine dividing each sample in a sound by 2, and then trying > to play it back and hearing garbage because it got automatically > converted to an array of longs during the division. I see your point, but I am uneasy with the solution. Shouldn't there be a "divide in place" operation for this case that coerces constants the other way? (I haven't followed all the details -- you may have it already or you may have a good reason for not having it.) There should be a divide in place method, I just never bothered to add it. I'll try and remedy this before the beta release. I guess that you also don't check for overflow on such operations -- suppose I multiply each sample by 2 and it doesn't fit... This is also different than Python's own numeric operations, but again I can see the reason why (and I don't object in this case). There are in fact two different modules which contain array math operations. umath and fast_umath. These modules will actually define the code used for things like a*4. fast_umath will never check for overflow for the obvious speed reasons. umath is supposed to work as much like regular python math as possible. It checks for divide-by-zero and for floating point overflows. At the moment it doesn't check for integer overflows, but it SHOULD, and it will as soon as I get around to it. I think having both of these modules is a nice way to get "the best of both worlds". I guess I won't complain too loudly as long as you also have a way to coerce an array of shorts into an array of longs. Anything can be cast to anything else, including shoving an array of doubles into an array of bytes. Of course, some casts are a better idea than others... (BTW, on 64-bit machines, there are three relevant int sizes. Do you support this?) four actually, byte, short (2-byte), int (4-byte), and long (8-byte). They're all there. > I do have one other possible set of rules that I would be happy with. > These are basically the same as the last post, except: > > 4) Coercion to a higher precision is allowed within the main > groupings. > > 5) The precision of a python scalar is allowed to change to match the > other arguments (3.14 can be either a float or a double depending on > which one is "desired"). Some examples, please? Where else is coercion applied? It does sound like it could be a good compromise. See Perry's recent post for a good explanation of this proposal. Examples: array([1,2,3], 'f') * 2. -> array([1,2,3], 'f') # 2. is treated as a float here array([1,2,3], 'l') * 2. -> array([1,2,3], 'l') # 2. is treated as a double here > I don't like this rule, but without it I think that it is too easy to > get bitten by python scalars when writing "real" code. Agreed. > Does this make sense? > > Do people prefer this revised scheme? I'm not sure. More input, please... Well, I just saw your latest reply, so I don't think more input's needed. I'll send this anyway because the earlier comments should be useful. I think I'll give this thing another hour to settle down and then I'll code up the new typecasting rules (I really love how quickly discussion moves on this list). -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Jan 25 12:11:45 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA14313 for matrix-sig-people; Thu, 25 Jan 1996 12:11:45 -0500 Received: from harfang.CC.UMontreal.CA ([132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id MAA14309 for ; Thu, 25 Jan 1996 12:11:32 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id MAA22878 (8.6.11/IDA-1.6); Thu, 25 Jan 1996 12:09:20 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA17871; Thu, 25 Jan 1996 12:09:19 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA13702; Thu, 25 Jan 1996 12:09:17 -0500 Date: Thu, 25 Jan 1996 12:09:17 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601251709.MAA13702@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: guido@CNRI.Reston.VA.US, matrix-sig@python.org In-reply-to: <9601251545.AA08514@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] type coercion one more time Sender: owner-matrix-sig@python.org Precedence: bulk On the other hand, I see the other types as being used for cases where the programmer really wants to use a particular precision, ie. 2-byte integers for dealing with 16-bit sound data. In a case like this it seems decidely unfriendly to automatically coerce to another precision. I wouldn't call it "decidedly unfriendly" (you can always avoid coercion by making sure that all variables have the same type), but it makes sense. It should also be noted that one of the main reasons to have float->double coercion in C is to make it possible to call functions written for double with float arguments. This problem does not exist in Python. 4) Coercion to a higher precision is allowed within the main groupings. What are "main groupings"? 5) The precision of a python scalar is allowed to change to match the other arguments (3.14 can be either a float or a double depending on which one is "desired"). I don't like this rule for two reasons: 1) You can lose precision without noticing it. 2) It creates a subtle distinction between scalars and arrays of rank 0, which can only confuse users. On the other hand, I do see the problem you want to solve. It doesn't make any code easier to read if you have to convert every constant to a rank-0 float array. If I get an ingenious idea for solving this problem, I'll let you know immediately... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Jan 25 12:17:34 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA14371 for matrix-sig-people; Thu, 25 Jan 1996 12:17:34 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id MAA14366 for ; Thu, 25 Jan 1996 12:17:30 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id MAA23156 (8.6.11/IDA-1.6); Thu, 25 Jan 1996 12:15:09 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA19349; Thu, 25 Jan 1996 12:15:07 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA14003; Thu, 25 Jan 1996 12:15:06 -0500 Date: Thu, 25 Jan 1996 12:15:06 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601251715.MAA14003@cyclone.ERE.UMontreal.CA> To: stoll@atr-sw.atr.co.jp CC: jjh@Goldilocks.LCS.MIT.EDU, matrix-sig@python.org In-reply-to: <199601251609.BAA02646@ciris21.atr-sw.atr.co.jp> (stoll@atr-sw.atr.co.jp) Subject: Re: [PYTHON MATRIX-SIG] type coercion one more time Sender: owner-matrix-sig@python.org Precedence: bulk Also, I don't recall anyone presenting a list of 'nice' names for the Matrix functions as Konrad has been advocating. I think having these is a good idea; the type code that we have now has always seemed somehow counter to (my view of) the Python 'do-it-the-Right-Way' attitude. If I actually list them all out here, maybe this will force the issue and start some discussion. Comments on the following? Array_Char, Array_UChar Array_Short, Array_UShort Array_Integer,Array_UInteger, Array_Float, Array_Double, Array_FloatComplex, Array_DoubleComplex 'Unsigned' instead of 'U'? XArray instead of Array_X? BTW, is the C Do we have unsigned types? I proposed a similar list of names a while ago, using "XArray" instead of "Array_X". I prefer that form because it is closer to the real-life expressions "integer array", "float array" etc. But the main point is to get rid of these silly type codes! ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Jan 25 12:28:49 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA14446 for matrix-sig-people; Thu, 25 Jan 1996 12:28:49 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id MAA14440 for ; Thu, 25 Jan 1996 12:28:46 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA09628; Thu, 25 Jan 96 12:28:10 EST Date: Thu, 25 Jan 96 12:28:10 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601251728.AA09628@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: hinsenk@ere.umontreal.ca Cc: matrix-sig@python.org In-Reply-To: <199601251709.MAA13702@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA) Subject: Re: [PYTHON MATRIX-SIG] type coercion one more time Sender: owner-matrix-sig@python.org Precedence: bulk From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad) On the other hand, I see the other types as being used for cases where the programmer really wants to use a particular precision, ie. 2-byte integers for dealing with 16-bit sound data. In a case like this it seems decidely unfriendly to automatically coerce to another precision. I wouldn't call it "decidedly unfriendly" (you can always avoid coercion by making sure that all variables have the same type), but it makes sense. It should also be noted that one of the main reasons to have float->double coercion in C is to make it possible to call functions written for double with float arguments. This problem does not exist in Python. This all is irrelevant if we can agree on the part below. 4) Coercion to a higher precision is allowed within the main groupings. What are "main groupings"? int, float, complex (actually, string is also a grouping, but it doesn't really pertain). 5) The precision of a python scalar is allowed to change to match the other arguments (3.14 can be either a float or a double depending on which one is "desired"). I don't like this rule for two reasons: 1) You can lose precision without noticing it. You can also lose precision by coercing a long to a double, but nobody's complaining about that. Also, the only place you can lose precision is in a python scalar so I'm not really worried about this. Admittedly, array([1,2,3], 'f')*3.1415926535 might cause you to lose some precision in pi, but I assume that this is what most people would want. 2) It creates a subtle distinction between scalars and arrays of rank 0, which can only confuse users. This is my own objection to the idea, but I think I have a way to make this cleaner: Add three more names, "GenericInt", "GenericFloat", "GenericComplex" (or whatever). Now I can say a = array(1, "GenericFloat") and this means that in general a will behave as a double, but it can be coerced downwards to a float if required. Now, 3.14 -> array(3.14, "GenericFloat"). Does this seem conceptually clean? On the other hand, I do see the problem you want to solve. It doesn't make any code easier to read if you have to convert every constant to a rank-0 float array. If I get an ingenious idea for solving this problem, I'll let you know immediately... I'm finally starting to feel happy about this proposal, but ingenious ideas are always welcome. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Jan 25 12:31:05 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA14472 for matrix-sig-people; Thu, 25 Jan 1996 12:31:05 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id MAA14467 for ; Thu, 25 Jan 1996 12:31:02 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA09638; Thu, 25 Jan 96 12:30:24 EST Date: Thu, 25 Jan 96 12:30:24 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601251730.AA09638@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: hinsenk@ere.umontreal.ca Cc: matrix-sig@python.org In-Reply-To: <199601251715.MAA14003@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA) Subject: Re: [PYTHON MATRIX-SIG] type coercion one more time Sender: owner-matrix-sig@python.org Precedence: bulk From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad) Also, I don't recall anyone presenting a list of 'nice' names for the Matrix functions as Konrad has been advocating. I think having these is a good idea; the type code that we have now has always seemed somehow counter to (my view of) the Python 'do-it-the-Right-Way' attitude. If I actually list them all out here, maybe this will force the issue and start some discussion. Comments on the following? Array_Char, Array_UChar Array_Short, Array_UShort Array_Integer,Array_UInteger, Array_Float, Array_Double, Array_FloatComplex, Array_DoubleComplex 'Unsigned' instead of 'U'? XArray instead of Array_X? BTW, is the C Do we have unsigned types? I proposed a similar list of names a while ago, using "XArray" instead of "Array_X". I prefer that form because it is closer to the real-life expressions "integer array", "float array" etc. But the main point is to get rid of these silly type codes! Currently the types are: character unsigned byte signed byte signed short signed int signed long float double complex float complex double If you really want to add unsigned shorts, longs, and ints, I'd obviously be willing, but the whole type collection already feels a little bit large, so I'd rather wait and see if anybody ever NEEDS them. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Jan 25 13:49:17 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA15114 for matrix-sig-people; Thu, 25 Jan 1996 13:49:17 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id NAA15110 for ; Thu, 25 Jan 1996 13:49:05 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id NAA26596 (8.6.11/IDA-1.6); Thu, 25 Jan 1996 13:46:07 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA10863; Thu, 25 Jan 1996 13:46:06 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA20452; Thu, 25 Jan 1996 13:46:05 -0500 Date: Thu, 25 Jan 1996 13:46:05 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601251846.NAA20452@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: matrix-sig@python.org In-reply-to: <9601251728.AA09628@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] type coercion one more time Sender: owner-matrix-sig@python.org Precedence: bulk You can also lose precision by coercing a long to a double, but nobody's complaining about that. Also, the only place you can lose That's not exactly the same problem. More importantly, it is a much less frequent problem. precision is in a python scalar so I'm not really worried about this. Admittedly, array([1,2,3], 'f')*3.1415926535 might cause you to lose some precision in pi, but I assume that this is what most people would want. In this case, with an explicit constant, I agree that most people would probably want a float result. But suppose you are using a library function that does float calculations and returns a float array as a result. You call this function and multiply its result with a double that you have obtained from some complicated high-precision calculation. Wouldn't you be both surprised and annoyed if the result were a float array? Add three more names, "GenericInt", "GenericFloat", "GenericComplex" (or whatever). Now I can say a = array(1, "GenericFloat") and this means that in general a will behave as a double, but it can be coerced downwards to a float if required. Why not make it float from the beginning? If you can't rely on the extra precision being conserved, you could just as well forget about it. I don't see much use for "unknown precision" numbers. Now, 3.14 -> array(3.14, "GenericFloat"). Does this seem conceptually clean? Conceptually clean, yes. But it is a somewhat weird concept. Let me start another approach to the problem: do you see a need for "downcasting" anything else than constants? In that case one could think about some special treatment for them. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Jan 25 13:50:27 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA15137 for matrix-sig-people; Thu, 25 Jan 1996 13:50:27 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id NAA15133 for ; Thu, 25 Jan 1996 13:50:23 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id NAA26773 (8.6.11/IDA-1.6); Thu, 25 Jan 1996 13:48:54 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA11410; Thu, 25 Jan 1996 13:48:53 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA20609; Thu, 25 Jan 1996 13:48:52 -0500 Date: Thu, 25 Jan 1996 13:48:52 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601251848.NAA20609@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: matrix-sig@python.org In-reply-to: <9601251730.AA09638@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] type coercion one more time Sender: owner-matrix-sig@python.org Precedence: bulk If you really want to add unsigned shorts, longs, and ints, I'd I didn't mean to suggest that we need more unsigned types! I just wanted an inventory of existing types for the purpose of finding names. obviously be willing, but the whole type collection already feels a little bit large, so I'd rather wait and see if anybody ever NEEDS them. Definitely! ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Jan 25 14:02:12 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA15223 for matrix-sig-people; Thu, 25 Jan 1996 14:02:12 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id OAA15219 for ; Thu, 25 Jan 1996 14:02:09 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA10739; Thu, 25 Jan 96 14:01:34 EST Date: Thu, 25 Jan 96 14:01:34 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601251901.AA10739@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: hinsenk@ere.umontreal.ca Cc: matrix-sig@python.org In-Reply-To: <199601251846.NAA20452@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Subject: Re: [PYTHON MATRIX-SIG] type coercion one more time Sender: owner-matrix-sig@python.org Precedence: bulk From: hinsenk@ere.umontreal.ca (Hinsen Konrad) precision is in a python scalar so I'm not really worried about this. Admittedly, array([1,2,3], 'f')*3.1415926535 might cause you to lose some precision in pi, but I assume that this is what most people would want. In this case, with an explicit constant, I agree that most people would probably want a float result. But suppose you are using a library function that does float calculations and returns a float array as a result. You call this function and multiply its result with a double that you have obtained from some complicated high-precision calculation. Wouldn't you be both surprised and annoyed if the result were a float array? I'd be tempted to say that if you have a high-precision double that you're really concerned about then you'll just have to say something like pi = array(3.1415926535, 'd'). In general, library functions should be expected to only return longs, doubles and complex doubles unless they are somehow "told otherwise". This is just my opinion. Add three more names, "GenericInt", "GenericFloat", "GenericComplex" (or whatever). Now I can say a = array(1, "GenericFloat") and this means that in general a will behave as a double, but it can be coerced downwards to a float if required. Why not make it float from the beginning? If you can't rely on the extra precision being conserved, you could just as well forget about it. I don't see much use for "unknown precision" numbers. The reason to not just make it float is that 3. * Array_l should produce an array of doubles, not of floats. Also, if a function can accept either floats or doubles, 3. should be treated as a double. Now, 3.14 -> array(3.14, "GenericFloat"). Does this seem conceptually clean? Conceptually clean, yes. But it is a somewhat weird concept. Weird I can agree with. Let me start another approach to the problem: do you see a need for "downcasting" anything else than constants? In that case one could think about some special treatment for them. I don't really see much difference between python scalars and constants. The problem with both of them is that they have double precision no matter what. I think that it's important to allow these types to be downcast to a lower precision. (Or don't allow any automatic coercion of precision, but that original proposal seems to be an extremely unpopular idea based on some private responses I've gotten). -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Jan 25 14:43:53 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA00185 for matrix-sig-people; Thu, 25 Jan 1996 14:43:53 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id OAA00181 for ; Thu, 25 Jan 1996 14:43:48 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA09984; Thu, 25 Jan 96 11:43:34 PST Received: from kristen (localhost) by kristen.llnl.gov (5.x/SMI-SVR4) id AA03373; Thu, 25 Jan 1996 11:43:31 -0800 Message-Id: <3107DD63.7CAB@llnl.gov> Date: Thu, 25 Jan 1996 11:43:31 -0800 From: "Paul. Dubois" Organization: Lawrence Livermore National Laboratory X-Mailer: Mozilla 2.0b5 (X11; I; SunOS 5.4 sun4m) Mime-Version: 1.0 To: James Hugunin Cc: matrix-SIG@python.org Subject: Re: [PYTHON MATRIX-SIG] type coercion one more time References: <9601251759.AA10036@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk I have been busy giving talks and attending meetings, etc., and had a hard time plowing through all this carefully. If I understand the latest idea with respect to precision, I think I can live with it but I don't like it. It would violate the principle of least surprise. It essentially changes the precision of the scalars in Python iff they get involved with doing something with Array_f's. I don't like exceptions in languages. Exceptions lead to weird behavior. For example, we all agree it is a Good Thing that x[i] is a scalar not an array of size 1. But in fact this means that (1./3.) * x[i] is not ((1./3.) * x)[i] if x is Array_f. I agree it is nice to go fast but let's not go too fast. I thought the "generic" stuff was off track. Everybody who writes anything that handles Arrays has enough trouble without one more type to look for. Jim, what you need to do in your own work can be pretty much solved with a little literal function: def lit(d): return Array_f(d) so that you just always say lit(2.) rather than 2. You have to train yourself to do this in the places where it matters, true, but otherwise you are just going to make the rest of us miserable. -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Jan 25 16:27:04 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA00860 for matrix-sig-people; Thu, 25 Jan 1996 16:27:04 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id QAA00856 for ; Thu, 25 Jan 1996 16:27:00 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA12494; Thu, 25 Jan 96 16:26:37 EST Date: Thu, 25 Jan 96 16:26:37 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601252126.AA12494@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: dubois1@llnl.gov Cc: matrix-SIG@python.org In-Reply-To: <3107DD63.7CAB@llnl.gov> (dubois1@llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] type coercion one more time Sender: owner-matrix-sig@python.org Precedence: bulk Paul's and Konrad's arguments make me think that this whole type coercion issue needs to be reevaluated before mistakenly commiting to a bad strategy. I still want to release a 0.30 version tommorrow, hopefully with stable naming conventions and API. The only things I have left to do are packaging and tests on our alphas and pentiums for some semblance of portability. Unfortunately, this release will have an interim API that is highly unlikely to be the final solution, oh well. Sender: dubois@llnl.gov I have been busy giving talks and attending meetings, etc., and had a hard time plowing through all this carefully. If I understand the latest idea with respect to precision, I think I can live with it but I don't like it. It would violate the principle of least surprise. I agree with this completely. This is why I proposed the awkward Array_f*3.14 -> Exception solution. For me having this go to an Array_d is not what I am expecting. It essentially changes the precision of the scalars in Python iff they get involved with doing something with Array_f's. I don't like exceptions in languages. Exceptions lead to weird behavior. For example, we all agree it is a Good Thing that x[i] is a scalar not an array of size 1. But in fact this means that (1./3.) * x[i] is not ((1./3.) * x)[i] if x is Array_f. This is a great example of where my newer proposal has problems. I agree it is nice to go fast but let's not go too fast. I thought the "generic" stuff was off track. Everybody who writes anything that handles Arrays has enough trouble without one more type to look for. I don't think this will actually impact people much beyond the sort of problems you mention above. It's mainly just to keep things reasonably clean conceptually. Jim, what you need to do in your own work can be pretty much solved with a little literal function: def lit(d): return Array_f(d) so that you just always say lit(2.) rather than 2. You have to train yourself to do this in the places where it matters, true, but otherwise you are just going to make the rest of us miserable. Well, the last thing I want to do is to make anybody miserable, but I still think that I have a reasonable point here. With the current (0.20) type coercion semantics, python scalars are completely useless in an equation unless you want to work with longs, doubles, or complex doubles. The new proposal, while slightly strange, at least gives a reasonable meaning to python scalars for other types. Just to make things concrete, I picked a reasonable simple function that I use a lot of the time. Do you find this readable? def hamming(x): return lit(0.54)-lit(0.46)*cos(lit(2*pi)*x/lit(len(x)-1)) vs. def hamming(x): return 0.54-0.46*cos(2*pi*x/(len(x)-1)) -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Jan 25 16:35:18 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA00948 for matrix-sig-people; Thu, 25 Jan 1996 16:35:18 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id QAA00940 for ; Thu, 25 Jan 1996 16:35:15 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id OAA28042 (8.6.11/IDA-1.6); Thu, 25 Jan 1996 14:18:00 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA18211; Thu, 25 Jan 1996 14:17:59 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA22980; Thu, 25 Jan 1996 14:17:58 -0500 Date: Thu, 25 Jan 1996 14:17:58 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601251917.OAA22980@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: matrix-sig@python.org In-reply-to: <9601251901.AA10739@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] type coercion one more time Sender: owner-matrix-sig@python.org Precedence: bulk like pi = array(3.1415926535, 'd'). In general, library functions should be expected to only return longs, doubles and complex doubles unless they are somehow "told otherwise". This is just my opinion. Library routines might need floats for the same reason they were introduced: efficiency. And converting the result to double just to conform to a specification is a bad idea if the caller wants to go on using floats. The reason to not just make it float is that 3. * Array_l should produce an array of doubles, not of floats. Also, if a function can accept either floats or doubles, 3. should be treated as a double. I understand that, but don't see the point. Either I want double precision, and then I want it under circumstances, or I am happy with float, and then I start with float. For what application might I possible want a number of unknown precision that is as expensive as double, but can't be trusted to be more than float? That's the worst of both types. I don't really see much difference between python scalars and constants. The problem with both of them is that they have double Not in implementation, but in use. Most constants occurring in real programs can be represented by a float, but that is not true for an arbitrary Python scalar that might be the result of a calculation. So suppose (in principle; I don't want to do this!) that we add a type "float" to Python and make all constants that do not exceed the precision of "float" float constants (with automatic coercion to double if the need arises), would you say that your problem is solved? In fact, now that I think of it, this idea is not completely unrealistic. A new type "float" could be seen as an implementation alternative to "double" and wouldn't even have to be visible to the programmer. I'll have to think a bit about this. types to be downcast to a lower precision. (Or don't allow any automatic coercion of precision, but that original proposal seems to be an extremely unpopular idea based on some private responses I've gotten). Given the choice between the two proposals, I very much prefer the first one. An occasional exception (whis is easy to fix) is a lot better than a loss in accuracy without warning. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 26 02:38:34 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id CAA03984 for matrix-sig-people; Fri, 26 Jan 1996 02:38:34 -0500 Received: from atr-sw.atr-sw.atr.co.jp (atr-sw.atr-sw.atr.co.jp [133.186.20.117]) by python.org (8.6.12/8.6.12) with ESMTP id CAA03980 for ; Fri, 26 Jan 1996 02:38:20 -0500 Received: from ciris21.atr-sw.atr.co.jp (ciris21.atr-sw.atr.co.jp [133.186.20.41]) by atr-sw.atr-sw.atr.co.jp (8.6.11+2.5Wb2/3.4Wbeta3/95031616ATR-SW) with ESMTP id QAA21800; Fri, 26 Jan 1996 16:37:40 +0900 Received: from localhost by ciris21.atr-sw.atr.co.jp (940816.SGI.8.6.9/6.4J.6) id QAA03317; Fri, 26 Jan 1996 16:37:39 +0900 Message-Id: <199601260737.QAA03317@ciris21.atr-sw.atr.co.jp> To: matrix-sig@python.org cc: jjh@Goldilocks.LCS.MIT.EDU Subject: [PYTHON MATRIX-SIG] Type coercion two more times Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Date: Fri, 26 Jan 1996 16:37:38 +0900 From: "Perry A. Stoll" Sender: owner-matrix-sig@python.org Precedence: bulk I'm about 12 hours behind all of you in North America, so I come to work in the morning and read all the activity of the previous day. Most of the debate is over by the time I read it, so I have to spend the day working instead ;-) Here's the situation now: - There is support for three scalar numerical types in Python: PyInteger ( C long ) PyFloat ( C double ) PyComplex ( C double complex ) - The multiarray object is introducing (providing access to) operations for many more C numerical types. The question is how do we handle operations that mix C numerical types not in Python (i.e. C float, C short) with Python supported C types (C long,C double)? In the following, I'm going to use Jim's term of "3 Classes" of numerical types ( Integer, Float, Complex) to mean the set of corresponding C types: Integer = any of the C byte,short,int, or long Float = either C float or C double Complex = either C float complex or C double complex -------------------------------------------------- The Proposal: We define an optional attribute for multiarrays, say, oh, I don't know: array.__maintain_ctype__, or array.__avoid_coercion__, or array.__do_no_coerce_me__ (?!?) - Coercions within a Class (Integer<->Integer,Float<->Float) are governed by this attribute, namely, If this attribute exists and is TRUE: then the result will be the same type as the multiarray. If the attribute does not exists or is FALSE, then NORMAL Python coercion rules apply, and the result will always be a PyInteger, PyFloat, or PyComplex. - Coercions to a higher Class (i.e. Integer-> Float -> Complex) always occur if necessary. The result will be the normal Python type for that Class, namely PyInteger, PyFloat, or PyComplex. - If both operands have this attribute set but the array types disagree, raise an exception. Operations involving two multiarrays can be dealt with this way. - Correspondingly, add a keyword to the Array creation function, i.e. >>> myshortarray = Array.array([1,2,3,4],'s',nocoercion=1) -------------------------------------------------- The Explanation Normally, this attribute would not be set, so operations will follow normal Python rules. That makes it consistant with the rest of Python for new and old users. But when Jim reads in a C float file, he sets the "__do_not_coerce_me__" attribute, letting the C multiarray code know he expects operations involving this array and other Floats to result in C float arrays. -------------------------------------------------- Comments? -Perry ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 26 09:57:01 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA05663 for matrix-sig-people; Fri, 26 Jan 1996 09:57:01 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id JAA05658 for ; Fri, 26 Jan 1996 09:56:55 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id JAA25268 (8.6.11/IDA-1.6); Fri, 26 Jan 1996 09:54:11 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA19089; Fri, 26 Jan 1996 09:54:09 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA23121; Fri, 26 Jan 1996 09:54:08 -0500 Date: Fri, 26 Jan 1996 09:54:08 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601261454.JAA23121@cyclone.ERE.UMontreal.CA> To: stoll@atr-sw.atr.co.jp CC: matrix-sig@python.org, jjh@Goldilocks.LCS.MIT.EDU In-reply-to: <199601260737.QAA03317@ciris21.atr-sw.atr.co.jp> (stoll@atr-sw.atr.co.jp) Subject: Re: [PYTHON MATRIX-SIG] Type coercion two more times Sender: owner-matrix-sig@python.org Precedence: bulk I'm about 12 hours behind all of you in North America, so I come to work in the morning and read all the activity of the previous On the contrary, you are 14 hours ahead of us (us on the East coast at least), so you could complete a whole discussion ready for us every morning ;-) - Coercions within a Class (Integer<->Integer,Float<->Float) are governed by this attribute, namely, If this attribute exists and is TRUE: then the result will be the same type as the multiarray. If the attribute does not exists or is FALSE, then NORMAL Python coercion rules apply, and the result will always be a PyInteger, PyFloat, or PyComplex. Basically that means having two array types with different behaviour, and hiding that distinction in an attribute. Which means most users won't be aware of its existence and will be surprised the first time they get something else than they expect. Besides, it doesn't solve Jim's problem of mixing C float arrays with Python float scalars. If you consider scalars to be of "coercible" type, then you run into the same problem as with Jim's second proposal. If not, then the problem is the same as with full automatic coercion. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 26 10:36:04 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA05860 for matrix-sig-people; Fri, 26 Jan 1996 10:36:04 -0500 Received: from atr-sw.atr-sw.atr.co.jp (atr-sw.atr-sw.atr.co.jp [133.186.20.117]) by python.org (8.6.12/8.6.12) with ESMTP id KAA05856 for ; Fri, 26 Jan 1996 10:35:53 -0500 Received: from ciris21.atr-sw.atr.co.jp (ciris21.atr-sw.atr.co.jp [133.186.20.41]) by atr-sw.atr-sw.atr.co.jp (8.6.11+2.5Wb2/3.4Wbeta3/95031616ATR-SW) with ESMTP id AAA04435; Sat, 27 Jan 1996 00:35:14 +0900 Received: from localhost by ciris21.atr-sw.atr.co.jp (940816.SGI.8.6.9/6.4J.6) id AAA03838; Sat, 27 Jan 1996 00:35:12 +0900 Message-Id: <199601261535.AAA03838@ciris21.atr-sw.atr.co.jp> To: hinsenk@ere.umontreal.ca (Hinsen Konrad) CC: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Type coercion two more times In-reply-to: Your message of "Fri, 26 Jan 1996 09:54:08 JST" References: <199601261454.JAA23121@cyclone.ERE.UMontreal.CA> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Date: Sat, 27 Jan 1996 00:35:12 +0900 From: "Perry A. Stoll" Sender: owner-matrix-sig@python.org Precedence: bulk >>>>> "HK" == Hinsen Konrad writes: HK> On the contrary, you are 14 hours ahead of us (us on the East Yes, I realized that this afternoon as I left work for the weekend. BTW, have a good day at work/school today :-) But don't ask why I'm logged in after midnight to read this mail :-) HK> Basically that means having two array types with different HK> behaviour, and hiding that distinction in an attribute. Which Maybe, but it seems like we're trying to satisfy two conflicting constraints, namely, 1) providing a nice, consistant array library usable for the mathematically oriented (potential) Python user, and 2) providing easy access to custom, relatively low level number crunching and control of C types, i.e. 16 bit speech data, 8 bit images, etc. HK> means most users won't be aware of its existence and will be HK> surprised the first time they get something else than they HK> expect. Most users won't care about its existance unless they fall into group 2) above. Then they must care about data type conversion and this provides some degree of control. I'm not saying it's good; this is a real problem. HK> Besides, it doesn't solve Jim's problem of mixing C float HK> arrays with Python float scalars. I thought it did - can you explain this a bit more? I'm probably missing something. HK> If you consider scalars to be of "coercible" type, then you HK> run into the same problem as with Jim's second proposal. I see. Yes, the question of whether or not to to convert scalars is pivotal. If someone is using a type other than a Python numeric type, then doesn't that mean they are willing to accept the loss of precision? If you have an array_f, do you expect calculations with PyFloats to occur at C double precision? I say by using an array_f, you say "I want C float precision!". -Perry ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 26 10:53:26 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA05979 for matrix-sig-people; Fri, 26 Jan 1996 10:53:26 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id KAA05975 for ; Fri, 26 Jan 1996 10:53:23 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id KAA27432 (8.6.11/IDA-1.6); Fri, 26 Jan 1996 10:51:17 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id KAA06090; Fri, 26 Jan 1996 10:51:16 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id KAA28001; Fri, 26 Jan 1996 10:51:15 -0500 Date: Fri, 26 Jan 1996 10:51:15 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601261551.KAA28001@cyclone.ERE.UMontreal.CA> To: stoll@atr-sw.atr.co.jp CC: matrix-sig@python.org In-reply-to: <199601261535.AAA03838@ciris21.atr-sw.atr.co.jp> (stoll@atr-sw.atr.co.jp) Subject: Re: [PYTHON MATRIX-SIG] Type coercion two more times Sender: owner-matrix-sig@python.org Precedence: bulk Maybe, but it seems like we're trying to satisfy two conflicting constraints, namely, 1) providing a nice, consistant array library usable for the mathematically oriented (potential) Python user, and 2) providing easy access to custom, relatively low level number crunching and control of C types, i.e. 16 bit speech data, 8 bit images, etc. You seem to imply that the additional types will be used only by people in group 2. But I expect that C floats will also be used by ordinary mathematically oriented Python users, whenever memory or CPU time are more important than accuracy. Most of my numerical programs (mostly in Fortran) use both precisions, so why should Python code be different? I see. Yes, the question of whether or not to to convert scalars is pivotal. If someone is using a type other than a Python numeric type, then doesn't that mean they are willing to accept the loss of precision? If you have an array_f, do you expect calculations with Sometimes. There are two situations where I expect problems: 1) Someone uses a library written by someone else, and this library uses C floats. The user of this library may not be aware of this, or might not care. Until he experiences a mysterious loss of precision in his code... 2) Someone wants to use both precisions, and either forgets about the coercion problems, or expects them to be like in all other languages (which coerce float->double, but not the reverse). Therefore I consider Jim's first proposal (no coercion between float and double) still the best: those who want both precisions and do everything correctly get what they want, and those who make a mistake or have misunderstood the rules get an exception. I'd rather be surprised by an exception than by loss of precision. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 26 11:24:05 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA06253 for matrix-sig-people; Fri, 26 Jan 1996 11:24:05 -0500 Received: from atr-sw.atr-sw.atr.co.jp (atr-sw.atr-sw.atr.co.jp [133.186.20.117]) by python.org (8.6.12/8.6.12) with ESMTP id LAA06248 for ; Fri, 26 Jan 1996 11:23:55 -0500 Received: from ciris21.atr-sw.atr.co.jp (ciris21.atr-sw.atr.co.jp [133.186.20.41]) by atr-sw.atr-sw.atr.co.jp (8.6.11+2.5Wb2/3.4Wbeta3/95031616ATR-SW) with ESMTP id BAA05442; Sat, 27 Jan 1996 01:22:00 +0900 Received: from localhost by ciris21.atr-sw.atr.co.jp (940816.SGI.8.6.9/6.4J.6) id BAA03904; Sat, 27 Jan 1996 01:21:57 +0900 Message-Id: <199601261621.BAA03904@ciris21.atr-sw.atr.co.jp> To: hinsenk@ere.umontreal.ca (Hinsen Konrad) CC: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Type coercion two more times In-reply-to: Your message of "Fri, 26 Jan 1996 10:51:15 JST" References: <199601261551.KAA28001@cyclone.ERE.UMontreal.CA> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Date: Sat, 27 Jan 1996 01:21:57 +0900 From: "Perry A. Stoll" Sender: owner-matrix-sig@python.org Precedence: bulk You seem to imply that the additional types will be used only by people in group 2. I had been implicitly assuming that. I was basing this on the precendent set by Python numerical types. Therefore I consider Jim's first proposal (no coercion between float and double) still the best: those who want both precisions and do everything correctly get what they want, and those who make a mistake or have misunderstood the rules get an exception. Ok, I'm starting to be convinced. My last hurdle to saying yes is having to explain the following to potential Python array users: >>> pi = omath.arctan(1.0)*4.0 >>> m = Array_f([1.0,2.0,3.0]) >>> m * pi Traceback (innermost last): File "", line 1, in ? TypeError: inconsistant floating point types -Perry ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 26 11:28:43 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA06298 for matrix-sig-people; Fri, 26 Jan 1996 11:28:43 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id LAA06294 for ; Fri, 26 Jan 1996 11:28:41 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA00464; Fri, 26 Jan 96 11:28:26 EST Date: Fri, 26 Jan 96 11:28:26 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601261628.AA00464@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Type naming conventions Sender: owner-matrix-sig@python.org Precedence: bulk Returning one last time to the issue of naming conventions. (I have some time on my hands until a type coercion strategy is decided on). I'd like to propose adding the following functions to Numeric.py. Character Integer Float Complex On a typical system, Integer(16) would return "s" (the typecode for an int). Integer(64) would raise an exception on most systems, but would return "l" on an alpha. Thus you could write things like array([1,2,3], Integer(32)). Things like Integer(minimumSize=16) would also be possible if we agreed they made sense. You'd also want to have Integer() just return the typecode used by python ints. This could be integrated with the standard array print function to also have more pleasing outputs. I know that Paul DuBois had some good comments on this previously in regards to FORTRAN 90. I'd be curious to know if this new proposal is at all useful or reasonable. Opinions? -Jim PS - Thanks to Mike McLay for the note that made me think of this again. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 26 11:29:06 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA06312 for matrix-sig-people; Fri, 26 Jan 1996 11:29:06 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id LAA06308 for ; Fri, 26 Jan 1996 11:29:02 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id LAA28925 (8.6.11/IDA-1.6); Fri, 26 Jan 1996 11:26:40 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id LAA16466; Fri, 26 Jan 1996 11:26:39 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id LAA01120; Fri, 26 Jan 1996 11:26:38 -0500 Date: Fri, 26 Jan 1996 11:26:38 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601261626.LAA01120@cyclone.ERE.UMontreal.CA> To: stoll@atr-sw.atr.co.jp CC: matrix-sig@python.org In-reply-to: <199601261621.BAA03904@ciris21.atr-sw.atr.co.jp> (stoll@atr-sw.atr.co.jp) Subject: Re: [PYTHON MATRIX-SIG] Type coercion two more times Sender: owner-matrix-sig@python.org Precedence: bulk Ok, I'm starting to be convinced. My last hurdle to saying yes is having to explain the following to potential Python array users: >>> pi = omath.arctan(1.0)*4.0 >>> m = Array_f([1.0,2.0,3.0]) >>> m * pi Traceback (innermost last): File "", line 1, in ? TypeError: inconsistant floating point types It could be made a bit easier by changing the text of the error message: TypeError: no automatic coercion between precisions Yet I agree, this is not a desirable feature. It is just the most acceptable among the evils we have to choose from. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 26 11:36:23 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA06407 for matrix-sig-people; Fri, 26 Jan 1996 11:36:23 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id LAA06403 for ; Fri, 26 Jan 1996 11:36:21 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA00493; Fri, 26 Jan 96 11:35:47 EST Date: Fri, 26 Jan 96 11:35:47 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601261635.AA00493@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: hinsenk@ere.umontreal.ca Cc: stoll@atr-sw.atr.co.jp, matrix-sig@python.org In-Reply-To: <199601261626.LAA01120@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Subject: Re: [PYTHON MATRIX-SIG] Type coercion two more times Sender: owner-matrix-sig@python.org Precedence: bulk From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Ok, I'm starting to be convinced. My last hurdle to saying yes is having to explain the following to potential Python array users: >>> pi = omath.arctan(1.0)*4.0 >>> m = Array_f([1.0,2.0,3.0]) >>> m * pi Traceback (innermost last): File "", line 1, in ? TypeError: inconsistant floating point types It could be made a bit easier by changing the text of the error message: TypeError: no automatic coercion between precisions Yet I agree, this is not a desirable feature. It is just the most acceptable among the evils we have to choose from. An important thing to remember here is that a naive user should never write m = array([1.0,2.0,3.0], 'f') (the new syntax). They should just write m = array([1.0,2.0,3.0]) which will produce an array of doubles and should allow everybody to live happily ever after. Not that I particularly like the type error either... -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 26 11:39:40 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA06434 for matrix-sig-people; Fri, 26 Jan 1996 11:39:40 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id LAA06430 for ; Fri, 26 Jan 1996 11:39:37 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa09999; 26 Jan 96 11:34 EST Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) for id LAA25356; Fri, 26 Jan 1996 11:32:46 -0500 Message-Id: <199601261632.LAA25356@monty> To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Type coercion two more times In-reply-to: Your message of "Sat, 27 Jan 1996 01:21:57 +0900." <199601261621.BAA03904@ciris21.atr-sw.atr.co.jp> References: <199601261551.KAA28001@cyclone.ERE.UMontreal.CA> <199601261621.BAA03904@ciris21.atr-sw.atr.co.jp> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Fri, 26 Jan 1996 11:32:45 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk Re confusing Python users: how confusing is Python's use of the words "float" for "double" and "int" for "long"? Is it more important to be consistent with C or with Python? This may affect the names chosen for the array types (Array_float, etc.)... --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 26 11:58:25 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA06613 for matrix-sig-people; Fri, 26 Jan 1996 11:58:25 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id LAA06604 for ; Fri, 26 Jan 1996 11:58:22 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA12076; Fri, 26 Jan 96 11:52:44 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) for matrix-sig@python.org id MAA03624; Fri, 26 Jan 1996 12:03:30 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199601261703.MAA03624@maigret> Subject: Re: [PYTHON MATRIX-SIG] Type coercion two more times To: matrix-sig@python.org Date: Fri, 26 Jan 1996 12:03:29 -0500 (EST) In-Reply-To: <9601261635.AA00493@baribal.LCS.MIT.EDU.LCS.MIT.EDU> from "James Hugunin" at Jan 26, 96 11:35:47 am X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 967 Sender: owner-matrix-sig@python.org Precedence: bulk > having to explain the following to potential Python array users: > >>> pi = omath.arctan(1.0)*4.0 > >>> m = Array_f([1.0,2.0,3.0]) > >>> m * pi > Traceback (innermost last): > File "", line 1, in ? > TypeError: inconsistant floating point types > > It could be made a bit easier by changing the text of the > error message: > TypeError: no automatic coercion between precisions I have no idea whether this is possible or not, but one area where I think Python needs help is the detail of the exceptions. For example, would it be possible at all to have at least the two uncoercable types listed ("first argument is a single precision float, second argument is a double precision float")? --david PS: This doesn't have to be in the next release, just something to keep in mind. A certain set of users at least will want to use the array tool in an interactive context, where these things matter a lot. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 26 12:12:01 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA06724 for matrix-sig-people; Fri, 26 Jan 1996 12:12:01 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id MAA06720 for ; Fri, 26 Jan 1996 12:11:58 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id MAA00881 (8.6.11/IDA-1.6); Fri, 26 Jan 1996 12:08:15 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA28447; Fri, 26 Jan 1996 12:08:14 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA05505; Fri, 26 Jan 1996 12:08:12 -0500 Date: Fri, 26 Jan 1996 12:08:12 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601261708.MAA05505@cyclone.ERE.UMontreal.CA> To: guido@CNRI.Reston.VA.US CC: matrix-sig@python.org In-reply-to: <199601261632.LAA25356@monty> (message from Guido van Rossum on Fri, 26 Jan 1996 11:32:45 -0500) Subject: Re: [PYTHON MATRIX-SIG] Type coercion two more times Sender: owner-matrix-sig@python.org Precedence: bulk Re confusing Python users: how confusing is Python's use of the words "float" for "double" and "int" for "long"? Is it more important to be consistent with C or with Python? This may affect the names chosen for the array types (Array_float, etc.)... Internal consistency is more important. I don't think of Python as a dialect of C, and many potential numerical users won't know C at all. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 26 14:26:52 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA07612 for matrix-sig-people; Fri, 26 Jan 1996 14:26:52 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id OAA07600 for ; Fri, 26 Jan 1996 14:26:46 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id MAA00801 (8.6.11/IDA-1.6); Fri, 26 Jan 1996 12:05:47 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA27744; Fri, 26 Jan 1996 12:05:47 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA05321; Fri, 26 Jan 1996 12:05:45 -0500 Date: Fri, 26 Jan 1996 12:05:45 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601261705.MAA05321@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: matrix-sig@python.org In-reply-to: <9601261628.AA00464@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] Type naming conventions Sender: owner-matrix-sig@python.org Precedence: bulk I'd like to propose adding the following functions to Numeric.py. Character Integer Float Complex I like the idea, but would prefer names like CharacterType, IntType etc. "Integer" looks too much like a cast to integer type. Things like Integer(minimumSize=16) would also be possible if we agreed they made sense. You'd also want to have Integer() just return the typecode used by python ints. The latter makes more sense to me. To be precise, I'd like these functions to return a typecode for a type that is at least as big as indicated, not necessarily exactly as big. For integers this is not so important (the sizes are always 8/16/32/64 on modern machines), but there is more variation for float types. In fact the FloatType() function should perhaps have two arguments, one specifying precision and the other range. Then FloatType(10, 50) would specify a type with an accuracy of 10 or more decimal digits and a range of at least 10**50 (which would be "double" on most machines, but "float" on a Cray). ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Jan 26 18:40:03 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA01049 for matrix-sig-people; Fri, 26 Jan 1996 18:40:03 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id SAA01044 for ; Fri, 26 Jan 1996 18:39:59 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA04290; Fri, 26 Jan 96 18:39:56 EST Date: Fri, 26 Jan 96 18:39:56 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601262339.AA04290@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Final pre-beta (0.30) of NumericalPython is available! Sender: owner-matrix-sig@python.org Precedence: bulk Release 0.30 of the numerical extension to python is available. Now that the design appears to be final, I'd like as many people as possible to install this system and play with it. Once it is released as 1.0BETA1, any changes to the fundamental design will be very unlikely to happen, so if you want to have a final system that you'll be happy with, now's the time to speak up. You can find the code at: ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericalPython-0.30.tar.gz If somebody (Tom are you there?) would like to place this on a site in Europe and post the location, I'd appreciate it. It's frustrating just how hard it can be to cross the Atlantic these days. The easiest way to install it is to untar the file in your top level python directory and follow the instructions in INSTALL.NumPy. Please do read this file as it contains important information (unlike the currently vacuous README). The naming conventions and C API of this version should be final. If this release goes well, the next release will be 1.0BETA1 to the general python community. I intend to produce a new bug-fixed release every week until I manage to go a week without any bug reports, and then I'll release 1.0BETA, so PLEASE report any bugs you find so that they can be squashed quickly. The code is running reasonably stably right now, and I've compilied it without a problem under the following configurations: alpha DECOS cc Sun SUNOS4 gcc P90 NextStep3 cc If you do grab the code and try to compile it, please send me mail (hugunin@mit.edu) telling me the machine and OS you used, and whether or not you could get it to compile, and whether or not you could run the TestSuite (ArrayTest/TestSuite.py). I'd also like to know the time reported by the Benchmark in the test suite just out of general curiosity. You should also check out TUTORIAL.NumPy for a VERY brief summary of how to use the object. Enjoy - Jim Things that probably will change before the BETA release: -1 The print function sucks! There is a function in Numeric.py called print_array that is the function that gets invoked whenever an array is printed. Currently it displays a very ugly representation. I really hope that somebody will write a nice pretty printing function to go here (you get to code it in python instead of C!). -2 The exact contents of Numeric.py, I'm very interested in comments -3 The type coercion scheme. If you only work with longs, doubles, and complex doubles (the basic python types) you can consider the coercion scheme fixed and stable. If you use other types, this is the current scheme: Only the following three automatic coercions are allowed: 1) Any integer type can be coerced to a float, a double, a complex float or a complex double. 2) A float can only be coerced to a complex float. 3) A double can only be coerced to a complex double. Explanation: The following type hierarchy is assumed: INTEGER < FLOAT < COMPLEX Within each level there are a number of possible precisions. No automatic coercion will take place between these precisions. This remains completely consistent with the existing python automatic type coercion. This scheme seems to make the most people the least unhappy, but I'm still interested in better ideas. -4 The Precision.py stuff I just coded in the last half hour, so obviously it might change in the near future. I am convinced that something like this is the right approach. -5 If Jim Fulton implements a nice dynamically loadable version using CObjects and I can easily install it on my SUNs (I'm usually clueless about dynamic loading issues) then this will be used. -6 Now that the design seems stable its time to write documentation. I'm not enough of a fool to widely release something without decent documentation. I've heard rumors that Paul DuBois might be helping out with this. Anybody else who'd like to contribute? ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Jan 27 00:11:37 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id AAA02831 for matrix-sig-people; Sat, 27 Jan 1996 00:11:37 -0500 Received: from atr-sw.atr-sw.atr.co.jp (atr-sw.atr-sw.atr.co.jp [133.186.20.117]) by python.org (8.6.12/8.6.12) with ESMTP id AAA02824 for ; Sat, 27 Jan 1996 00:11:05 -0500 Received: from ciris21.atr-sw.atr.co.jp (ciris21.atr-sw.atr.co.jp [133.186.20.41]) by atr-sw.atr-sw.atr.co.jp (8.6.11+2.5Wb2/3.4Wbeta3/95031616ATR-SW) with ESMTP id OAA20397; Sat, 27 Jan 1996 14:09:13 +0900 Received: from localhost by ciris21.atr-sw.atr.co.jp (940816.SGI.8.6.9/6.4J.6) id OAA04528; Sat, 27 Jan 1996 14:09:13 +0900 Message-Id: <199601270509.OAA04528@ciris21.atr-sw.atr.co.jp> To: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Final pre-beta (0.30) of NumericalPython is available! In-reply-to: Your message of "Fri, 26 Jan 1996 18:39:56 JST" References: <9601262339.AA04290@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Date: Sat, 27 Jan 1996 14:09:12 +0900 From: "Perry A. Stoll" Sender: owner-matrix-sig@python.org Precedence: bulk Being 14 hours ahead may have given me the honor of being the first to download and try the code. As usual, being on the cutting edge means you can get cut, to wit: the file Modules/arraytypes.c is NOT IN THE DISTRIBUTION!!! I'm trying to use the old matrixtypes.c (converting matrix -> array). It compiles, but I'll let you know if it works or not... -Perry ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Jan 27 11:49:28 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA04498 for matrix-sig-people; Sat, 27 Jan 1996 11:49:28 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id LAA04494 for ; Sat, 27 Jan 1996 11:49:26 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA17704; Sat, 27 Jan 96 11:49:14 EST Date: Sat, 27 Jan 96 11:49:14 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601271649.AA17704@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Oops! Problems with 0.30 distribution Sender: owner-matrix-sig@python.org Precedence: bulk My apologies to all (and thanks to Perry and Konrad for being the guinea pigs), but I left a file out of the distribution. This has been fixed now, and the file NumericalPython-0.30.tar.gz should now contain all of the needed files for a successful compile. If you've already downloaded the distribution, you can just get the file arraytypes.c and put it in your Modules directory. This is the file I left out. I hope this solves the problem - Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Jan 28 13:21:52 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA09637 for matrix-sig-people; Sun, 28 Jan 1996 13:21:52 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id NAA09632 for ; Sun, 28 Jan 1996 13:21:47 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA06247; Sun, 28 Jan 96 13:21:22 EST Date: Sun, 28 Jan 96 13:21:22 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601281821.AA06247@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] New release of NumericPython 0.31 Sender: owner-matrix-sig@python.org Precedence: bulk There turned out to be a small bug in the test suite pertaining to complex numbers, so I produced a new release that fixes this. When untarring the NumericPython release, as well as patches.tar (Konrad's) patches to the python core), you should use tar -xvfm. The m tells tar to set the modification times on the files to the current time, and this will ensure that a new build will not use old object files. It was forgetting to do this myself that led to my latest build not including Konrad's latest patch version, and the bug in the test suite. For those who don't want to download a complete new version, you just need to pick up TestCreation.py and put it in ArrayTest. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Jan 28 21:38:33 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id VAA11903 for matrix-sig-people; Sun, 28 Jan 1996 21:38:33 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id VAA11899 for ; Sun, 28 Jan 1996 21:38:28 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id VAA25790 (8.6.11/IDA-1.6 for ); Sun, 28 Jan 1996 21:37:01 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id VAA16766; Sun, 28 Jan 1996 21:37:00 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id VAA05601; Sun, 28 Jan 1996 21:37:00 -0500 Date: Sun, 28 Jan 1996 21:37:00 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601290237.VAA05601@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org In-reply-to: <9601281821.AA06247@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] New release of NumericPython 0.31 Sender: owner-matrix-sig@python.org Precedence: bulk When untarring the NumericPython release, as well as patches.tar (Konrad's) patches to the python core), you should use tar -xvfm. The m tells tar to set the modification times on the files to the current time, and this will ensure that a new build will not use old object files. Also make sure that you remove the old files Lib/Matrix.py and Lib/MLab.py, otherwise they will be loaded instead of their new replacements in the directory "numeric". And don't worry if "TestPrecision" reports bad results; that's just a bug in the test suite. I got the whole test suite running except for the final speed benchmark, which uses a function "sum" that no longer exists (should it?). ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Jan 28 22:06:37 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id WAA12095 for matrix-sig-people; Sun, 28 Jan 1996 22:06:37 -0500 Received: from atr-sw.atr-sw.atr.co.jp (atr-sw.atr-sw.atr.co.jp [133.186.20.117]) by python.org (8.6.12/8.6.12) with ESMTP id WAA12070 for ; Sun, 28 Jan 1996 22:04:16 -0500 Received: from ciris21.atr-sw.atr.co.jp (ciris21.atr-sw.atr.co.jp [133.186.20.41]) by atr-sw.atr-sw.atr.co.jp (8.6.11+2.5Wb2/3.4Wbeta3/95031616ATR-SW) with ESMTP id MAA18887; Mon, 29 Jan 1996 12:03:11 +0900 Received: from localhost by ciris21.atr-sw.atr.co.jp (940816.SGI.8.6.9/6.4J.6) id MAA08416; Mon, 29 Jan 1996 12:03:10 +0900 Message-Id: <199601290303.MAA08416@ciris21.atr-sw.atr.co.jp> To: hinsenk@ere.umontreal.ca (Hinsen Konrad) CC: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] New release of NumericPython 0.31 In-reply-to: Your message of "Sun, 28 Jan 1996 21:37:00 JST" References: <199601290237.VAA05601@cyclone.ERE.UMontreal.CA> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Date: Mon, 29 Jan 1996 12:03:10 +0900 From: "Perry A. Stoll" Sender: owner-matrix-sig@python.org Precedence: bulk > I got the whole test suite running except for the final speed > benchmark, which uses a function "sum" that no longer exists > (should it?) Yes, it exists! It's in MLab.py. You have to add 'from MLab import *' to the top of the Benchmark.py file. This means Jim has removed the MLab support from the default Numeric library. Is this ok with people? I don't mind, but we're going to have to let people Can I make a suggestions: can we avoid the 'from X import *' construction in the numeric package where it's not speed critical (??)- it makes finding the source of definitions a pain. -Perry ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Jan 29 08:31:50 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id IAA14927 for matrix-sig-people; Mon, 29 Jan 1996 08:31:50 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id IAA14923 for ; Mon, 29 Jan 1996 08:31:47 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id IAA04402 (8.6.11/IDA-1.6); Mon, 29 Jan 1996 08:23:38 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id IAA27234; Mon, 29 Jan 1996 08:23:38 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id IAA00117; Mon, 29 Jan 1996 08:23:36 -0500 Date: Mon, 29 Jan 1996 08:23:36 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601291323.IAA00117@cyclone.ERE.UMontreal.CA> To: stoll@atr-sw.atr.co.jp CC: matrix-sig@python.org In-reply-to: <199601290303.MAA08416@ciris21.atr-sw.atr.co.jp> (stoll@atr-sw.atr.co.jp) Subject: Re: [PYTHON MATRIX-SIG] New release of NumericPython 0.31 Sender: owner-matrix-sig@python.org Precedence: bulk Yes, it exists! It's in MLab.py. You have to add 'from MLab import *' to the top of the Benchmark.py file. Right. I also had to add "from fast_umath import *" to MLab.py, otherwise it wouldn't know how add.reduce. Note that "from umath import *" wouldn't work with the benchmark because it produces an overflow: Traceback (innermost last): File "Benchmark.py", line 226, in ? test2(10) File "Benchmark.py", line 46, in test2 esc= escout(zt, akap, srcfun) File "Benchmark.py", line 115, in escout exf= exp(outer(multiply,-1./mu,tau)) OverflowError: math range error This is of course something the benchmark author should worry about. This means Jim has removed the MLab support from the default Numeric library. Is this ok with people? I don't mind, but we're going to have to let people I'd have done the same. I see the Matlab compatibility module as something distinct from the standard array/matrix function, because the differ in some conventions. Mixing this up would only generate confusion. Can I make a suggestions: can we avoid the 'from X import *' construction in the numeric package where it's not speed critical (??)- it makes finding the source of definitions a pain. This is not a matter of speed; the import takes hardly any time. It is a matter of typing convenience. It might indeed be a good idea to to replace this with an explicit list (e.g. "from umath import acos, asin, atan, ...") in the standard modules, but that should be done only in the final version, to avoid updating problems. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Jan 29 09:44:41 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA15366 for matrix-sig-people; Mon, 29 Jan 1996 09:44:41 -0500 Received: from rnet.rose.rsoc.rockwell.com (rose.rsoc.rockwell.com [161.40.39.100]) by python.org (8.6.12/8.6.12) with SMTP id JAA15362 for ; Mon, 29 Jan 1996 09:44:36 -0500 Received: by rnet.rose.rsoc.rockwell.com (4.1/SMI-4.1) id AA02975; Mon, 29 Jan 96 08:43:58 CST Received: from sunrise(161.40.50.11) by rnet via smap (V1.3) id sma002971; Mon Jan 29 08:43:54 1996 Received: from darwin.rsoc.rockwell.com by rose.rsoc.rockwell.com (SMI-8.6/SMI-SVR4) id IAA07949; Mon, 29 Jan 1996 08:43:51 -0600 Received: by darwin.rsoc.rockwell.com (SMI-8.6/SMI-SVR4) id IAA02517; Mon, 29 Jan 1996 08:43:45 -0600 Date: Mon, 29 Jan 1996 08:43:45 -0600 From: friedric@rose.rsoc.rockwell.com (Robin Friedrich) Message-Id: <199601291443.IAA02517@darwin.rsoc.rockwell.com> To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] New release of NumericPython 0.31 X-Sun-Charset: US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk Having a few problems with the Numeric release. I built 0.30 with tar xvfm as instructed. (I'm not using .31 yet but I don't think it relates to this problem.) >From the tutorial: 1) First problem is with the types.py file. For some reason when it's imported vars(__builtins__) is an illegal argument while it works fine from the prompt. Traceback (innermost last): File "", line 1, in ? File "/local2/tools/lib/python/numeric/Numeric.py", line 9, in ? import string, types, math File "/local2/tools/lib/python/types.py", line 13, in ? if vars(__builtins__).has_key('complex'): TypeError: vars() argument must have __dict__ attribute 2) The following basic function is worrisome as well... >>> m = Matrix([[1,2,3], [4,5,6], [7,8,9]]) >>> m*m Traceback (innermost last): File "", line 1, in ? File "/local2/tools/lib/python/numeric/Matrix.py", line 74, in __mul__ NameError: MathError >>> ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Jan 29 10:28:54 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA15772 for matrix-sig-people; Mon, 29 Jan 1996 10:28:54 -0500 Received: from atr-sw.atr-sw.atr.co.jp (atr-sw.atr-sw.atr.co.jp [133.186.20.117]) by python.org (8.6.12/8.6.12) with ESMTP id KAA15768 for ; Mon, 29 Jan 1996 10:28:49 -0500 Received: from ciris21.atr-sw.atr.co.jp (ciris21.atr-sw.atr.co.jp [133.186.20.41]) by atr-sw.atr-sw.atr.co.jp (8.6.11+2.5Wb2/3.4Wbeta3/95031616ATR-SW) with ESMTP id AAA10793; Tue, 30 Jan 1996 00:27:47 +0900 Received: from localhost by ciris21.atr-sw.atr.co.jp (940816.SGI.8.6.9/6.4J.6) id AAA10895; Tue, 30 Jan 1996 00:27:45 +0900 Message-Id: <199601291527.AAA10895@ciris21.atr-sw.atr.co.jp> To: friedric@rose.rsoc.rockwell.com (Robin Friedrich) CC: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] New release of NumericPython 0.31 In-reply-to: Your message of "Mon, 29 Jan 1996 08:43:45 JST" References: <199601291443.IAA02517@darwin.rsoc.rockwell.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Date: Tue, 30 Jan 1996 00:27:44 +0900 From: "Perry A. Stoll" Sender: owner-matrix-sig@python.org Precedence: bulk 1) Problem with: if vars(__builtins__).has_key('complex'): TypeError: vars() argument must have __dict__ attribute yep, I had the same problem. The easiest way to get around this and yet have the same functionality is to comment out the following lines in $PYLIB/types.py: #if vars(__builtins__).has_key('complex'): # ComplexType = type(complex(0,1)) And add: try: ComplexType = type(complex(0,1)) except: pass I know how __builtins__ is becoming a dictioary instead of a module (in Python/ceval.c), but i don't know why. Anyone? Guido? I don't know about problem 2), although I just tried the same thing , namely a matrix multiply, and had got errors. Hey, what would a prebeta be without a few some problems? :-) -Perry ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Jan 29 10:55:39 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA15944 for matrix-sig-people; Mon, 29 Jan 1996 10:55:39 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id KAA15940 for ; Mon, 29 Jan 1996 10:55:35 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id KAA11792 (8.6.11/IDA-1.6); Mon, 29 Jan 1996 10:53:37 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id KAA07511; Mon, 29 Jan 1996 10:53:35 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id KAA09076; Mon, 29 Jan 1996 10:53:34 -0500 Date: Mon, 29 Jan 1996 10:53:34 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601291553.KAA09076@cyclone.ERE.UMontreal.CA> To: friedric@rose.rsoc.rockwell.com CC: matrix-sig@python.org In-reply-to: <199601291443.IAA02517@darwin.rsoc.rockwell.com> (friedric@rose.rsoc.rockwell.com) Subject: Re: [PYTHON MATRIX-SIG] New release of NumericPython 0.31 Sender: owner-matrix-sig@python.org Precedence: bulk >From the tutorial: 1) First problem is with the types.py file. For some reason when it's imported vars(__builtins__) is an illegal argument while it works fine from the prompt. You can solve this temporarily by removing the call to vars(). It seems that inside a module definition, __builtins__ is not a module, but a dictionary. I suspect this is a bug in Python, until someone can explain me the reason... 2) The following basic function is worrisome as well... >>> m = Matrix([[1,2,3], [4,5,6], [7,8,9]]) >>> m*m Traceback (innermost last): File "", line 1, in ? File "/local2/tools/lib/python/numeric/Matrix.py", line 74, in __mul__ NameError: MathError >>> When I try this here, it is a bit different: Traceback (innermost last): File "test.py", line 3, in ? m*m File "/usr/people/hinsenk/lib/python/numeric/Matrix.py", line 5, in __mul__ return self.__return_constructor__(self.array.matrixMultiply(other)) AttributeError: __int__ This indicates that some function can't deal with an int matrix. What puzzles me in your error message is the indication "line 74". Matrix.py is rather short! ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Jan 29 11:22:09 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA16205 for matrix-sig-people; Mon, 29 Jan 1996 11:22:09 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id LAA16190 for ; Mon, 29 Jan 1996 11:22:04 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA18040; Mon, 29 Jan 96 11:21:27 EST Date: Mon, 29 Jan 96 11:21:27 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601291621.AA18040@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org In-Reply-To: <199601291553.KAA09076@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Subject: [PYTHON MATRIX-SIG] And onto release 0.32 Sender: owner-matrix-sig@python.org Precedence: bulk The good news is that most of the bugs found so far are extremely minor, the bad news is that so many small bugs slipped by me on Friday. I have a new release that fixes all of the bugs reported so far Numeric.py lacked some definitions UserArray and Matrix had a serious bug small warning in ofuncobject small bugs in TestSuite the bug in types.py (which strangely doesn't show up on my system) ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericPython-0.32.tar.gz I'd also like to remind people of Konrad's warning to remove any previous versions of Matrix.py that might be in your python library path. Similarly, anybody interested in using the new Matrix.py (for linear algebra style matrices) should take a good long look at Matrix.py. At the moment it does almost nothing. I have no intentions to use this class personally (a.matrixMultiply(b) works fine for me). If somebody sends me a better version of this class I'd be happy to add it to the distribution. I know that people probably don't want to keep downloading and installing a completely new version of the distribution each time some simple bugs are fixed (the diff's between 0.30 and 0.32 are about 100 lines). Embarassingly enough, I don't know an easy way to take these two tar files and produce a diff between them that will be appropriate for the patch program to be applied to the in-place distribution. I've included the file "diffs-0.30-0.32" that I made between untarred directories created from the distribution, if somebody knows how to take this file and update from 0.30 to 0.32, please share your knowledge. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Jan 29 12:14:32 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA16717 for matrix-sig-people; Mon, 29 Jan 1996 12:14:32 -0500 Received: from bartel.jhuapl.edu (bartel.jhuapl.edu [128.244.146.143]) by python.org (8.6.12/8.6.12) with SMTP id MAA16713 for ; Mon, 29 Jan 1996 12:14:30 -0500 Message-Id: <199601291714.MAA16713@python.org> Received: by bartel.jhuapl.edu (1.38.193.4/16.2) id AA23108; Mon, 29 Jan 1996 12:20:22 -0500 Date: Mon, 29 Jan 1996 12:20:22 -0500 From: Chris Chase S1A Cc: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Final pre-beta (0.30) of NumericalPython is available! In-Reply-To: <9601262339.AA04290@baribal.LCS.MIT.EDU.LCS.MIT.EDU> References: <9601262339.AA04290@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk >>>>> "JH" == James Hugunin writes: JH> Release 0.30 of the numerical extension to python is available. I got it and I am checking it out. Jim, you announced with the 2.0 release that you would be adding the slice notation patches, i.e. in the 2.0 release post: JH> Things I plan to fix ASAP JH> 1) Integrate Chris Chase's grammar patches to replace JH> Slice(None,None,2) with ::2. JH> 2) Make matrixMultiply work on matrices larger than 2d (this JH> requires a notion of rank). The ":" patch is missing. I haven't checked yet for extension of matrix multiplication to larger dimensional arrays. Are these to be added? Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Jan 29 12:35:31 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA16910 for matrix-sig-people; Mon, 29 Jan 1996 12:35:31 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id MAA16906 for ; Mon, 29 Jan 1996 12:35:28 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA18476; Mon, 29 Jan 96 12:34:47 EST Date: Mon, 29 Jan 96 12:34:47 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601291734.AA18476@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: chris.chase@jhuapl.edu Cc: matrix-sig@python.org In-Reply-To: <9601291713.AA25467@goldilocks> (message from Chris Chase S1A on Mon, 29 Jan 1996 12:20:22 -0500) Subject: [PYTHON MATRIX-SIG] Slice notation and function ranks Sender: owner-matrix-sig@python.org Precedence: bulk From: Chris Chase S1A JH> Things I plan to fix ASAP JH> 1) Integrate Chris Chase's grammar patches to replace JH> Slice(None,None,2) with ::2. JH> 2) Make matrixMultiply work on matrices larger than 2d (this JH> requires a notion of rank). The ":" patch is missing. I haven't checked yet for extension of matrix multiplication to larger dimensional arrays. Are these to be added? I'd still like to do these ASAP, but that "as possible" part is really getting in my way. I'm eager to produce a stable, robust, useful version of the array object that Guido will be happy to include as part of the standard distribution as quickly as possible. I think that it would be a very GOOD THING to have these array objects as a standard module in Python-1.4. In the interests of moving this along, I'm unfortunately neglecting two things that I'd also really like to see done. 1) Better multidimensional indexing (a[:-1, ::2] as well as a[1, .., 2]) Why not? I don't feel like I have the energy to do the necessary lobbying to convince the python community/Guido that these changes to the python core are needed. I'm using Konrad's patches because he packaged them up nicely and negociated with Guido for their inclusion in the next release of the language core. If somebody (Chris?) can produce a similar set of patches for slices (okay you've done this part) AND get Guido to agree to add them to the python core, then I'd love to incorporate them with the NumericPython package. 2) A solid notion of rank for both bounded and unbounded rank functions Why not? Personally I have no pressing need for this capability (my own needs for ranks are satisfied by pseudo-indices), and I figure it would probably take me two-weeks of serious work to do this right and make sure it's bug free. Also, this particular feature is an extension to the basic array system, so it can always be added later without causing any incompatible changes (I took reasonable care designing the ofuncobject with such an extension in mind). This makes me feel more comfortable with waiting on it until after a solid release 1.0 is out. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Jan 29 13:01:37 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA17071 for matrix-sig-people; Mon, 29 Jan 1996 13:01:37 -0500 Received: from GS151.SP.CS.CMU.EDU (GS151.SP.CS.CMU.EDU [128.2.203.163]) by python.org (8.6.12/8.6.12) with SMTP id NAA17066 for ; Mon, 29 Jan 1996 13:01:34 -0500 Received: from localhost by GS151.SP.CS.CMU.EDU id aa19163; 29 Jan 96 13:00 EST To: matrix-sig@python.ORG Subject: [PYTHON MATRIX-SIG] Documentation Reply-To: dredish@CS.cmu.edu Date: Mon, 29 Jan 1996 13:00:13 -0500 Message-ID: <19160.822938413@GS151.SP.CS.CMU.EDU> From: David Redish Sender: owner-matrix-sig@python.ORG Precedence: bulk Is there some documentation available on the Matrix implementation? Although it seems fast and complete, I've had trouble implementing even relatively basic operations in v0.2 (such as dot product). In v0.3 there is a file TUTORIAL.NumPy but it is rather meager (to say the least). Thanks David Redish Computer Science Department CMU graduate student Neural Processes in Cognition Training Program Center for the Neural Basis of Cognition (CNBC) http://www.cs.cmu.edu/Web/People/dredish/home.html ------------------------------------------------------------ maintainer, CNBC website: http://www.cs.cmu.edu/Web/Groups/CNBC maintainer, Cog-Neuro Sites on the Internet, courtesy CNBC: http://www.cs.cmu.edu/Web/Groups/CNBC/other maintainer, NIPS*96 website: http://www.cs.cmu.edu/Web/Groups/NIPS ------------------------------------------------------------ ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Jan 29 13:21:43 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA17342 for matrix-sig-people; Mon, 29 Jan 1996 13:21:43 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id NAA17338 for ; Mon, 29 Jan 1996 13:21:40 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA18632; Mon, 29 Jan 96 13:20:43 EST Date: Mon, 29 Jan 96 13:20:43 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601291820.AA18632@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: dredish@CS.cmu.edu Cc: matrix-sig@python.ORG In-Reply-To: <19160.822938413@GS151.SP.CS.CMU.EDU> (message from David Redish on Mon, 29 Jan 1996 13:00:13 -0500) Subject: Re: [PYTHON MATRIX-SIG] Documentation Sender: owner-matrix-sig@python.ORG Precedence: bulk Short answer: Not yet. If you are confused about how to do something with the array object, this is the perfect place to ask questions. I'll try and collect the answers to these questions into the Tutorial (and then when the BETA release comes out those folks will have a real tutorial). There are two ways to take the dot product of two vectors a and b. The first is close to the standard form for writing a dot product: > add.reduce(a*b) add.reduce(m) is equivalent to the python form reduce(add, m), but it will run several orders of magnitude faster for large arrays. The module Numeric.py also defines the function sum, which is equivalent to add.reduce, so this could also be written as: > sum(a*b) The second approach is to realize that matrixMultiply when applied to two vectors is equivalent to taking their dot product, so the following will also work (and will run the fastest): a.matrixMultiply(b) -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Jan 29 13:57:13 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA17708 for matrix-sig-people; Mon, 29 Jan 1996 13:57:13 -0500 Received: from bartel.jhuapl.edu (bartel.jhuapl.edu [128.244.146.143]) by python.org (8.6.12/8.6.12) with SMTP id NAA17704 for ; Mon, 29 Jan 1996 13:57:10 -0500 Message-Id: <199601291857.NAA17704@python.org> Received: by bartel.jhuapl.edu (1.38.193.4/16.2) id AA23369; Mon, 29 Jan 1996 14:02:42 -0500 Date: Mon, 29 Jan 1996 14:02:42 -0500 From: Chris Chase S1A To: matrix-sig@python.org Cc: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Subject: [PYTHON MATRIX-SIG] Problem compiling ofuncobject.c In-Reply-To: <199601291553.KAA09076@cyclone.ERE.UMontreal.CA> References: <199601291443.IAA02517@darwin.rsoc.rockwell.com> <199601291553.KAA09076@cyclone.ERE.UMontreal.CA> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk While compiling the 0.32 release of the NumericPython I ran into an error. Compiling on SGI with IRIX 5.2 and gcc 2.6.0. I received the following output: gcc -O -I./../Include -I.. -DHAVE_CONFIG_H -c ./ofuncobject.c ./ofuncobject.c: In function `PyOFunc_GenericReduction': ./ofuncobject.c:490: wrong type argument to unary exclamation mark ./ofuncobject.c: In function `PyOFunc_GenericReduceAt': ./ofuncobject.c:590: wrong type argument to unary exclamation mark *** Error code 1 (bu21) In ofuncobject.c, lines 490 and 590 have the following: if (self->check_return) TRY(check_array(ret)); With TRY defined as: #define TRY(E) if(! (E)) return NULL The problem is check_array() has return type void. I took out the TRY() on those two lines as no action is taken on the result. The rest of the compile was successful. Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Jan 29 14:01:12 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA17738 for matrix-sig-people; Mon, 29 Jan 1996 14:01:12 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id OAA17734 for ; Mon, 29 Jan 1996 14:01:08 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA20003; Mon, 29 Jan 96 11:00:26 PST Received: from kristen (localhost) by kristen.llnl.gov (5.x/SMI-SVR4) id AA13352; Mon, 29 Jan 1996 11:00:22 -0800 Message-Id: <310D1946.7FF5@llnl.gov> Date: Mon, 29 Jan 1996 11:00:22 -0800 From: "Paul. Dubois" Organization: Lawrence Livermore National Laboratory X-Mailer: Mozilla 2.0b5 (X11; I; SunOS 5.4 sun4m) Mime-Version: 1.0 To: James Hugunin Cc: dredish@cs.cmu.edu, matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Documentation References: <9601291820.AA18632@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk I have a deadline of Feb. 15 for my article for Computers in Physics in which I will give the basic documentation for the numerical extension. It will exist, trust me. I've been a little late getting it started because of my work connecting PDB to Python, which is a bottleneck here for our local work and I just had to push it. In recent developments, we used Python/Numeric combined with an extension that can read PDB files and one that knows how to talk to a fancy graphics package we got from our sister laboratory in France, and were able to make 3d pictures of the data in the files. (They haven't made a general release of this package so I won't bother to describe it). -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Jan 29 14:07:56 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA17812 for matrix-sig-people; Mon, 29 Jan 1996 14:07:56 -0500 Received: from GS151.SP.CS.CMU.EDU (GS151.SP.CS.CMU.EDU [128.2.203.163]) by python.org (8.6.12/8.6.12) with SMTP id OAA17808 for ; Mon, 29 Jan 1996 14:07:53 -0500 Received: from localhost by GS151.SP.CS.CMU.EDU id aa19317; 29 Jan 96 14:06 EST To: matrix-sig@python.ORG cc: dredish@CS.cmu.edu Subject: [PYTHON MATRIX-SIG] Compiling on HPUX with gcc Reply-To: dredish@CS.cmu.edu Date: Mon, 29 Jan 1996 14:06:51 -0500 Message-ID: <19315.822942411@GS151.SP.CS.CMU.EDU> From: David Redish Sender: owner-matrix-sig@python.ORG Precedence: bulk When trying to compile in HPUX with gcc, I get errors on lines 490 and 590 of ofuncobject.c. The problem is that check_array is of type void, but TRY() tries to test whether it's 0 or not. By changing from if (self->check_return) TRY(check_array(ret)); to if (self->check_return) check_array(ret); I can get it to compile. PS. I see someone just posted this in reference to the SGI. However, I get a ld error when I try to load it saying unsatisfied symbol: "finite". Am I missing a file? I'm using NumericPython-0.32.tar.gz. (I also untarred patches.tar.) Thanks David Redish Computer Science Department CMU graduate student Neural Processes in Cognition Training Program Center for the Neural Basis of Cognition (CNBC) http://www.cs.cmu.edu/Web/People/dredish/home.html ------------------------------------------------------------ maintainer, CNBC website: http://www.cs.cmu.edu/Web/Groups/CNBC maintainer, Cog-Neuro Sites on the Internet, courtesy CNBC: http://www.cs.cmu.edu/Web/Groups/CNBC/other maintainer, NIPS*96 website: http://www.cs.cmu.edu/Web/Groups/NIPS ------------------------------------------------------------ ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Jan 29 14:08:22 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA17831 for matrix-sig-people; Mon, 29 Jan 1996 14:08:22 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id OAA17825 for ; Mon, 29 Jan 1996 14:08:18 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA18816; Mon, 29 Jan 96 14:07:37 EST Date: Mon, 29 Jan 96 14:07:37 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601291907.AA18816@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: chris.chase@jhuapl.edu Cc: matrix-sig@python.org In-Reply-To: <9601291856.AA26104@goldilocks> (message from Chris Chase S1A on Mon, 29 Jan 1996 14:02:42 -0500) Subject: [PYTHON MATRIX-SIG] Another bug in release 0.32 Sender: owner-matrix-sig@python.org Precedence: bulk I should have known better than to mess around with ofuncobject.c in the middle of getting out a new release... For those who haven't been bitten yet, the following patch needs to be applied to Modules/ofuncobject.c Basically, on lines 490, and 590, you should remove the TRY from around the check_array function call. I have no idea why this didn't show up as an error under gcc. *** 487,493 **** /* Cleanup the returned matrices so that scalars will be returned as python scalars */ ! if (self->check_return) check_array(ret); return PyArray_Return(ret); } --- 487,493 ---- /* Cleanup the returned matrices so that scalars will be returned as python scalars */ ! if (self->check_return) TRY(check_array(ret)); return PyArray_Return(ret); } *************** *** 587,593 **** Py_DECREF(indices); /* Cleanup the returned matrices so that scalars will be returned as python scalars */ ! if (self->check_return) check_array(ret); return PyArray_Return(ret); } --- 587,593 ---- Py_DECREF(indices); /* Cleanup the returned matrices so that scalars will be returned as python scalars */ ! if (self->check_return) TRY(check_array(ret)); return PyArray_Return(ret); } *************** ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Jan 29 14:23:22 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA18031 for matrix-sig-people; Mon, 29 Jan 1996 14:23:22 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id OAA18025 for ; Mon, 29 Jan 1996 14:23:19 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA18868; Mon, 29 Jan 96 14:22:36 EST Date: Mon, 29 Jan 96 14:22:36 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601291922.AA18868@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: dredish@CS.cmu.edu Cc: matrix-sig@python.ORG, dredish@CS.cmu.edu In-Reply-To: <19315.822942411@GS151.SP.CS.CMU.EDU> (message from David Redish on Mon, 29 Jan 1996 14:06:51 -0500) Subject: Re: [PYTHON MATRIX-SIG] Compiling on HPUX with gcc Sender: owner-matrix-sig@python.ORG Precedence: bulk From: David Redish stuff about embarassing error covered elsewhere However, I get a ld error when I try to load it saying unsatisfied symbol: "finite". Am I missing a file? You're not missing a file, you're running into one of the unfortunate system dependent issues in IEEE libraries. On most systems, the standard library includes the function "finite" which returns true iff a floating point value is not +-Inf, and not NaN. This function is only referenced in ofuncobject.c in the macro CHECK which is defined at the top of the file. A quick fix is to just define CHECK(x) as nothing. This will mean that the umathmodule will not check for floating point overflows in calculations as it should. A better solution would be for you to figure out how to get this desired behavior on HPUX and send me the patches. A good check to see if you have a working implementation is to do the following: >>> import umath >>> umath.multiply(1.0e200, 1.0e200) This should produce OverflowError If instead it returns Infinity, you know that your routine is not doing the needed checks. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Jan 29 14:51:54 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA18263 for matrix-sig-people; Mon, 29 Jan 1996 14:51:54 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id OAA18258 for ; Mon, 29 Jan 1996 14:51:48 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id OAA22374 (8.6.11/IDA-1.6); Mon, 29 Jan 1996 14:49:24 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA08481; Mon, 29 Jan 1996 14:49:22 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA01699; Mon, 29 Jan 1996 14:49:21 -0500 Date: Mon, 29 Jan 1996 14:49:21 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601291949.OAA01699@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: matrix-sig@python.ORG Subject: [PYTHON MATRIX-SIG] Bug in matrixMultiply, and once more names Sender: owner-matrix-sig@python.ORG Precedence: bulk Here's a bug (or two) that I just found: >>> from Numeric import * >>> x = array([1,2,3.]) >>> add.reduce(x*x) 14.0 >>> x.matrixMultiply(x) array(14.0, 'd') >>> 2.*x.matrixMultiply(x) Segmentation fault The segmentation fault is the obvious bug, but I am also surprised that x.matrixMultiply(x) returns an array of rank 0 instead of a scalar. This raises the question of how rank-0 arrays are treated in general. Although it may seem reasonable to convert them to scalars, it also makes sense to keep them as arrays if they have one of the new types (e.g. C float). About names: >From the recent discussion I remember that there are supposed to be two array types: a low-level type "array" and a high-level type "Array". The latter may have become unnecessary now that "array"s have automatic coercion again. But should the remaining one be called "array" or "Array"? I don't really care, but certainly we should have either "array" and "matrix" or "Array" and "Matrix", not "array" and "Matrix" as it is now. Also, I have some more objections against the name of the module UserArray. I don't like the "UserXxx" names at all (because they don't relate to the user any more than any other module), but that's not my point now. UserList is a Python class that is equivalent to the built-in list type, UserDict similarly wraps dictionaries. So one should conclude that UserArray is an analogous wrapping of the old array type, but that is not true. The new arrays have different functionality, so they should have another name. One possibility would be "NumericArray", since the main new feature is numerical operations. On the other hand, there are no numerical operations on e.g. character arrays. Maybe someone else finds a better name. And now back to testing... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Jan 29 15:09:18 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA18425 for matrix-sig-people; Mon, 29 Jan 1996 15:09:18 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id PAA18421 for ; Mon, 29 Jan 1996 15:09:15 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA19173; Mon, 29 Jan 96 15:08:35 EST Date: Mon, 29 Jan 96 15:08:35 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601292008.AA19173@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: hinsenk@ere.umontreal.ca Cc: matrix-sig@python.ORG In-Reply-To: <199601291949.OAA01699@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA) Subject: [PYTHON MATRIX-SIG] Re: Bug in matrixMultiply, and once more names Sender: owner-matrix-sig@python.ORG Precedence: bulk From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad) Here's a bug (or two) that I just found: >>> from Numeric import * >>> x = array([1,2,3.]) >>> add.reduce(x*x) 14.0 I assume this isn't a bug. >>> x.matrixMultiply(x) array(14.0, 'd') This is a bug all right. The standing agreement is that all array functions will return a python scalar instead of a 0d array. Perhaps you're right and this should only happen for returned arrays of type long, double and double complex. It's all in one function so this would be an easy change, the final decision on the right thing to do here depends critically on the decisions made regarding type coercions. I just forgot to call this function for matrixMultiply. >>> 2.*x.matrixMultiply(x) Segmentation fault I'll fix this too. (once I figure it out) About names: >From the recent discussion I remember that there are supposed to be two array types: a low-level type "array" and a high-level type "Array". The latter may have become unnecessary now that "array"s have automatic coercion again. But should the remaining one be called "array" or "Array"? I don't really care, but certainly we should have either "array" and "matrix" or "Array" and "Matrix", not "array" and "Matrix" as it is now. I actually rather like the way that things stand now. Here's what we have: A low level type named array that is what I expect 90% of the people who use this system to use. This is entirely implemented in C. A wrapper class called UserArray that provides a wrapper around this new built-in array type. The theory here is that so long as we don't break any old code, we are free to ignore the old array type as obsolete. A subclass of UserArray called Matrix that implements matrixMultiply instead of array multiply for the "*" operator. This is mainly provided for people who do entirely linear algebra work. Personally I think that using a.matrixMultiply(b) is a better solution for people who want to work with both arrays and matrices. Also, I have some more objections against the name of the module UserArray. I don't like the "UserXxx" names at all (because they don't relate to the user any more than any other module), but that's not my point now. UserList is a Python class that is equivalent to the built-in list type, UserDict similarly wraps dictionaries. So one should conclude that UserArray is an analogous wrapping of the old array type, but that is not true. The new arrays have different functionality, so they should have another name. One possibility would be "NumericArray", since the main new feature is numerical operations. On the other hand, there are no numerical operations on e.g. character arrays. Maybe someone else finds a better name. To reiterate, UserArray is a wrapper class around the new C level type of array. There is a little bit of baggage in this wrapper involved in the fact that arrays are a fairly complicated class. In fact, I was able to substitute UserArrays into Benchmark.py with about 10 lines of code as a replacement for arrays, I used this for testing the speed of the wrapper class. And now back to testing... Keep those bugs coming! -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Jan 29 18:31:56 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA20112 for matrix-sig-people; Mon, 29 Jan 1996 18:31:56 -0500 Received: from muecke.appl-math.tu-muenchen.de (muecke.appl-math.tu-muenchen.de [129.187.242.1]) by python.org (8.6.12/8.6.12) with ESMTP id SAA20104 for ; Mon, 29 Jan 1996 18:31:43 -0500 Received: from hamster.appl-math.tu-muenchen.de by muecke.appl-math.tu-muenchen.de via ESMTP (940816.SGI.8.6.9/911001.SGI) for id AAA24166; Tue, 30 Jan 1996 00:30:28 +0100 Received: by hamster.appl-math.tu-muenchen.de (940816.SGI.8.6.9/911001.SGI) for matrix-sig@python.org id AAA28268; Tue, 30 Jan 1996 00:30:26 +0100 From: "Thomas Schwaller" Message-Id: <9601300030.ZM28266@hamster.appl-math.tu-muenchen.de> Date: Tue, 30 Jan 1996 00:30:20 +0100 X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] NumericPython-0.32.tar.gz (Europe) + PDE Proposal Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-matrix-sig@python.org Precedence: bulk Yeah. I'm still there... For people in Europe: You can fetch the matrixmodule at ftp://ftp.appl-math.tu-muenchen.de/pub/et/NumericPython-0.32.tar.gz I was quite happy not to get it more than once (takes really very long). By the way. Is FDL (or FIDL ?) working with the latest matrixmodule. (This is rally a question for Jim Fulton ! ;-)) Or asked the other way: is there some work done binding the usual FORTRAN Libraries (*PACK) to this extension? (To get the state of Matlab, SciLab, Octave). Well, let's look one step further. After several experiences with integrating C++ classes in Python, I'm not that enthousiastic anymore about this approach. One year earlier I would have tried to do it with Diffpack (C++ classes for the solution of PDE, which ist one of the best public packages for that purpose if you like C++). But some native Python Module for that purpose would be much more efficient, now that we have a common base for matrix extensions. What about a package for the abstract specification of PDE's? 1) High level commands for building stiffness matrices out of the week form of a PDE. This is for expert users. I saw such an abstract matrix language (I don't remember the name....) at the International Congress of Applied Mathematics. 2) Linear Algebra stuff related to that (Multigrid, Mutlilevel, domain decomposition) 3) For non expert users the possibility to solve nonlinear systems of PDE's by combining predefinied equations of type 1) (Iteration, Linearization and the like) 4) Using the C++ Visual Toolkit as a starting point for the visualisation of the results (OpenGL). I had no time to rework my Python Binding for it, which is very necessary because it has some bugs. Loading the C++ module is very slow so I would prefer to have a native Python/C module using the C++ Code (where is code reusing gone ??? :-( ) Unfortunately a lot of work in this direction will be done for Java by SGI (read their announcments http://www.sgi.com/Products/cosmo). Some comments? Anybody written some ODE stuff yet. I took a look at the matlab ODE suite recently. What a hack... When we present such an example with the official release of the matrix module, people will like it I think. I'll polish some Delaunay-Triangulation modules for public release as soon as I can. They work nice with the matrixmodule, but I have to pythonize the error handling of the C-Code it uses. For me it's ok, but you will certainly agree with me, that it's not nice to see the Python interpreter crashing with wrong geometrical data. Back to first tests with the new matrix version.... Tom ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Jan 29 19:20:25 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id TAA20551 for matrix-sig-people; Mon, 29 Jan 1996 19:20:25 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id TAA20547 for ; Mon, 29 Jan 1996 19:20:21 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id AAA03157; Tue, 30 Jan 1996 00:19:26 GMT From: "Jim Fulton, U.S. Geological Survey" Message-Id: <9601291919.ZM3155@dsdbqvarsa.er.usgs.gov> Date: Mon, 29 Jan 1996 19:19:25 -0500 In-Reply-To: "Thomas Schwaller" "[PYTHON MATRIX-SIG] NumericPython-0.32.tar.gz (Europe) + PDE Proposal" (Jan 30, 12:30am) References: <9601300030.ZM28266@hamster.appl-math.tu-muenchen.de> X-Phone: (703) 648-5622 X-Face: (;vf"lccr&V=y, matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] NumericPython-0.32.tar.gz (Europe) + PDE Proposal Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-matrix-sig@python.org Precedence: bulk On Jan 30, 12:30am, Thomas Schwaller wrote: > Subject: [PYTHON MATRIX-SIG] NumericPython-0.32.tar.gz (Europe) + PDE Prop > Yeah. I'm still there... > For people in Europe: > You can fetch the matrixmodule at > ftp://ftp.appl-math.tu-muenchen.de/pub/et/NumericPython-0.32.tar.gz > > I was quite happy not to get it more than once (takes really very long). > > By the way. Is FDL (or FIDL ?) working with the latest matrixmodule. > (This is rally a question for Jim Fulton ! ;-)) Not yet. I've been waiting for the matrix implementation to settle down. I still have to jump through some hoops before I can release FIDL. > Or asked the other way: is there some work done binding the usual FORTRAN > Libraries (*PACK) to this extension? (To get the state of Matlab, SciLab, > Octave). > > Well, let's look one step further. After several experiences with integrating > C++ classes in Python, I'm not that enthousiastic anymore about this approach. > One year earlier I would have tried to do it with Diffpack (C++ classes for > the solution of PDE, which ist one of the best public packages for that > purpose if you like C++). But some native Python Module for that purpose > would be much more efficient, now that we have a common base for matrix > extensions. My a "native Python module", do you mean a module written in Python, using the matrix extension? Or do you mean a custom built extension? In what way would this approach be more efficient? A tool like FIDL allows you to reuse existing libraries with minimal effort. A FIDL spec for a routine typically only requires a line or two, not counting comments. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Jan 29 19:52:09 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id TAA20695 for matrix-sig-people; Mon, 29 Jan 1996 19:52:09 -0500 Received: from bartel.jhuapl.edu (bartel.jhuapl.edu [128.244.146.143]) by python.org (8.6.12/8.6.12) with SMTP id TAA20689 for ; Mon, 29 Jan 1996 19:52:07 -0500 Message-Id: <199601300052.TAA20689@python.org> Received: by bartel.jhuapl.edu (1.38.193.4/16.2) id AA24311; Mon, 29 Jan 1996 19:57:54 -0500 Date: Mon, 29 Jan 1996 19:57:54 -0500 From: Chris Chase S1A To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Compiling on HPUX with gcc In-Reply-To: <9601291922.AA18868@baribal.LCS.MIT.EDU.LCS.MIT.EDU> References: <19315.822942411@GS151.SP.CS.CMU.EDU> <9601291922.AA18868@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk From: David Redish stuff about embarassing error covered elsewhere However, I get a ld error when I try to load it saying unsatisfied symbol: "finite". Am I missing a file? HPUX series 700/800 at Release 9 contain two versions of the math library. The precision architecture (PA) libm version 1.1 contains the IEEE functions. It is located in /lib/pa1.1. By default gcc links against version 1.0 (in /lib) whereas HPUX cc links against version 1.1. If you have the pa1.1 libraries then use the --with-libm=-L/lib/pa1.1 option to configure to get the pa1.1 version of libm when using gcc. Using this I was able to compile successfully. An aside: /usr/lib/pa1.1 for the library path when using FORTRAN or PASCAL or the vector library. Of possible future interest, the vector library on the HP series 700 systems is very fast and contains a number of operations that Numeric module could use for many of its basic operations. Another note: I have been unable to get Python to compile with the HP C compiler with or without ANSI mode. Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Jan 29 21:59:01 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id VAA21364 for matrix-sig-people; Mon, 29 Jan 1996 21:59:01 -0500 Received: from atr-sw.atr-sw.atr.co.jp (atr-sw.atr-sw.atr.co.jp [133.186.20.117]) by python.org (8.6.12/8.6.12) with ESMTP id VAA21359 for ; Mon, 29 Jan 1996 21:58:40 -0500 Received: from ciris21.atr-sw.atr.co.jp (ciris21.atr-sw.atr.co.jp [133.186.20.41]) by atr-sw.atr-sw.atr.co.jp (8.6.11+2.5Wb2/3.4Wbeta3/95031616ATR-SW) with ESMTP id LAA25599; Tue, 30 Jan 1996 11:57:25 +0900 Received: from localhost by ciris21.atr-sw.atr.co.jp (940816.SGI.8.6.9/6.4J.6) id LAA12363; Tue, 30 Jan 1996 11:57:24 +0900 Message-Id: <199601300257.LAA12363@ciris21.atr-sw.atr.co.jp> To: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) cc: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] A fix for mrange Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Date: Tue, 30 Jan 1996 11:57:23 +0900 From: "Perry A. Stoll" Sender: owner-matrix-sig@python.org Precedence: bulk >>> Numeric.mrange(1,5,1,'i') Traceback (innermost last): File "", line 1, in ? File "/home/stoll/Python/1.3/lib/python/numeric/Numeric.py", line 64, in mrange return m*step + start TypeError: cannot perform this operation on these types >>> Numeric.mrange(1,5,0.5,'f') Traceback (innermost last): File "", line 1, in ? File "/home/stoll/Python/1.3/lib/python/numeric/Numeric.py", line 64, in mrange return m*step + start TypeError: cannot perform this operation on these types >>> Numeric.mrange(1,5,1.0,'l') array([1.0,2.0,3.0,4.0], 'd') Shouldn't mrange honor the typecode? How about this as a replacement for mrange: # ???default typecode to long int to match builtin range()??? def mrange(start, stop=None, step=1, typecode='l'): """Just like range() except it returns an array whose type can be specfied by the keyword argument typecode. """ if (stop == None): stop = start start = 0 n = int(math.ceil(float(stop-start)/step)) m = array(range(n), 'l')*step + start if (m.typecode() != typecode): m = m.cast(typecode) return m Now I get this: >>> Numeric.mrange(1,5,1,'i') array([1,2,3,4], 'i') >>> Numeric.mrange(1,5,1.0,'l') array([1,2,3,4], 'l') >>> Numeric.mrange(200,300,10,'b') array([200,210,220,230,240,250,4,14,24,34], 'b') >>> Numeric.mrange(1,5,0.25,'l') array([1,1,1,1,2,2,2,2,3,3,3,3,4,4,4,4], 'l') The last is a little weird, but mrange always honors the typecode, which I think is very important. When I specify a type, I expect an array of that type. To coerce or not to coerce, that is the question... -Perry ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 30 08:40:53 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id IAA24114 for matrix-sig-people; Tue, 30 Jan 1996 08:40:53 -0500 Received: from harfang.CC.UMontreal.CA ([132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id IAA24110 for ; Tue, 30 Jan 1996 08:40:48 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id IAA19800 (8.6.11/IDA-1.6 for ); Tue, 30 Jan 1996 08:39:06 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id IAA19219; Tue, 30 Jan 1996 08:39:05 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id IAA22977; Tue, 30 Jan 1996 08:39:04 -0500 Date: Tue, 30 Jan 1996 08:39:04 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601301339.IAA22977@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] First experiences Sender: owner-matrix-sig@python.org Precedence: bulk Yesterday I replaced a rather awkward combination of a shell script and a Matlab program by a Python program that is shorter, easier to read, and faster than the old solution. First the most important and good news: it works, and it produces the same results as the Matlab code. Of course I also noticed a few problems/strange features/possible improvements, and here they are for general discussion: "mrange" should be called "arange" now. Or maybe something clearer, e.g. "rangeArray"? Why add.reduce(...) but outer(add, ...)? I have typed add.outer(...) more than once, and that makes more sense to me. There is a function sum() that is almost equivalent to add.reduce() - almost because it doesn't allow a second argument to specify the axis. Why? The more I use it (and see it on my screen), the pseudo-index None seems strange to me. As a test, I have shown my code to a colleague who is an active Matlab user. I told him what x[2,3] means and asked him what he expects x[All, None] to mean. His immediate answer was: "everything along the first axis, nothing along the second axis." Then after a pause: "but that doesn't make sense." So once I again I propose to change the name to "NewAxis" or "New" or anything else indicating that this index *creates* a new dimension. A related observation: I have used the combination [All, None] very often, but no other index expression involving All or None. This is of course a feature of my application, but I wouldn't be surprised if appending a new axis were the most frequent use of these pseudo-indices. In that case it would be worthwhile providing an equivalent convenience function that is shorter (i.e. less visually obtrusive). Currently Numeric.py imports from fast_umath; in my opinion the default should be umath. Those who want speed can always overwrite the definition by importing from fast_umath. A minor cosmetic point: when I print a ufunc, is still says 'ofunc'. Another cosmetic point: "Array.error" should be "ArrayError". I don't know how much effort this would be, but I'd really like reshape() to accept new shapes that lead to a different size. From my APL/J experience, I found that such reshaping is one of the most useful methods to construct non-standard matrices. Some examples: - an NxN array with all elements 37: reshape(37, (N, N)) - a checkerboard matrix with 0s and 1s: reshape([1,0],(8,9))[All, 0..7] We could then eliminate special case constructors like zeros(). Another important function that seems to be missing is appending, i.e. the equivalent of + for lists. Of course for arrays it has to work along any axis. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 30 10:26:20 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA24817 for matrix-sig-people; Tue, 30 Jan 1996 10:26:20 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id KAA24813 for ; Tue, 30 Jan 1996 10:26:18 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA27316; Tue, 30 Jan 96 10:19:51 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) id KAA18459; Tue, 30 Jan 1996 10:30:51 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199601301530.KAA18459@maigret> Subject: Re: [PYTHON MATRIX-SIG] None, All, etc. To: hinsenk@ere.umontreal.ca (Hinsen Konrad) Date: Tue, 30 Jan 1996 10:30:50 -0500 (EST) Cc: matrix-sig@python.org In-Reply-To: <199601301339.IAA22977@cyclone.ERE.UMontreal.CA> from "Hinsen Konrad" at Jan 30, 96 08:39:04 am X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 365 Sender: owner-matrix-sig@python.org Precedence: bulk After just reading Hinsen's post, and remembering Jim H's, comments about 'this is the perfect place to ask', I'd like to ask one question which I'm sure many novices will have. A good answer to this one should be in the Tutorial. [Stealing from APL/J docs is a possible way to go] What are these pseudo-indices (None, All, etc.) and how do they work? --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 30 11:14:18 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA25434 for matrix-sig-people; Tue, 30 Jan 1996 11:14:18 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id LAA25430 for ; Tue, 30 Jan 1996 11:14:13 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA03369; Tue, 30 Jan 96 11:12:04 EST Date: Tue, 30 Jan 96 11:12:04 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601301612.AA03369@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: stoll@atr-sw.atr.co.jp Cc: matrix-sig@python.org In-Reply-To: <199601300257.LAA12363@ciris21.atr-sw.atr.co.jp> (stoll@atr-sw.atr.co.jp) Subject: [PYTHON MATRIX-SIG] Re: A fix for mrange Sender: owner-matrix-sig@python.org Precedence: bulk Shouldn't mrange honor the typecode? Obviously, I'll switch over to using your new version. However, I think that it's important to make a small change for determining the default typecode. If any of the arguments is a python float, than the range returned should be an array of doubles, not longs. ie. arange(1, 5, 1.0) should be array([1.,2.,3.,4.], 'd') I'll add some checks to the code to get this behavior in the case of no explicit typecode given. BTW - the typecode argument is really intended to be used as a keyword argument, as in arange(1, 5, 1.0, typecode=Integer()). This keeps things uniform for the case of arange(10, typecode=Integer()) which wouldn't work without the keyword argument. The last is a little weird, but mrange always honors the typecode, which I think is very important. When I specify a type, I expect an array of that type. Agreed! To coerce or not to coerce, that is the question... Now that question needs to be addressed seperately. Thanks for the fix - Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 30 11:36:18 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA25637 for matrix-sig-people; Tue, 30 Jan 1996 11:36:18 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id LAA25633 for ; Tue, 30 Jan 1996 11:36:15 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA03443; Tue, 30 Jan 96 11:35:08 EST Date: Tue, 30 Jan 96 11:35:08 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601301635.AA03443@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: hinsenk@ere.umontreal.ca Cc: matrix-sig@python.org In-Reply-To: <199601301339.IAA22977@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Subject: Re: [PYTHON MATRIX-SIG] First experiences Sender: owner-matrix-sig@python.org Precedence: bulk From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Yesterday I replaced a rather awkward combination of a shell script and a Matlab program by a Python program that is shorter, easier to read, and faster than the old solution. First the most important and good news: it works, and it produces the same results as the Matlab code. Great news! This is of course one of the main purposes for this code. Of course I also noticed a few problems/strange features/possible improvements, and here they are for general discussion: "mrange" should be called "arange" now. Or maybe something clearer, e.g. "rangeArray"? Definately "arange" (or something clearer but equally short). I use arange far to much to be willing to put up with a nice clear name like rangeArray. Why add.reduce(...) but outer(add, ...)? I have typed add.outer(...) more than once, and that makes more sense to me. Writing outer product in python code was trivial, writing it in C would have taken some time, so I cut some corners while I was still unsure about the permanent status of pseudo-indices in the system. I'll remove outer from the standard functions, and add a new method to the ofuncs (ufuncs now?). There is a function sum() that is almost equivalent to add.reduce() - almost because it doesn't allow a second argument to specify the axis. Why? I don't know whether or not sum even belongs in the set of standard functions. On the one hand I'd rather make people use add.reduce because it's much more in the spirit of the array object. On the other hand, sum is such a friendly little function. If it remains in the system, then I agree it should get the second argument. The more I use it (and see it on my screen), the pseudo-index None seems strange to me. As a test, I have shown my code to a colleague who is an active Matlab user. I told him what x[2,3] means and asked him what he expects x[All, None] to mean. His immediate answer was: "everything along the first axis, nothing along the second axis." Then after a pause: "but that doesn't make sense." So once I again I propose to change the name to "NewAxis" or "New" or anything else indicating that this index *creates* a new dimension. I agree with you here. Clearly there is another naming issue to be resolved. See my next post. A related observation: I have used the combination [All, None] very often, but no other index expression involving All or None. This is of course a feature of my application, but I wouldn't be surprised if appending a new axis were the most frequent use of these pseudo-indices. In that case it would be worthwhile providing an equivalent convenience function that is shorter (i.e. less visually obtrusive). Well, I find that All is used a lot of the time, but I agree with you that None is almost always use to append a new axis. I'll address this also in my next post. Currently Numeric.py imports from fast_umath; in my opinion the default should be umath. Those who want speed can always overwrite the definition by importing from fast_umath. And vice versa. This is mainly a matter of personal taste, and since my taste favors fast_umath, that's going to be the default version for now. If I find that many people on this sig disagree with me then I'll change it. A minor cosmetic point: when I print a ufunc, is still says 'ofunc'. That's because ufunc's are still called ofunc's. This is another one of those great renaming sorts of things, and it should be done. Another cosmetic point: "Array.error" should be "ArrayError". Easy to fix. I don't know how much effort this would be, but I'd really like reshape() to accept new shapes that lead to a different size. From my APL/J experience, I found that such reshaping is one of the most useful methods to construct non-standard matrices. Some examples: - an NxN array with all elements 37: reshape(37, (N, N)) - a checkerboard matrix with 0s and 1s: reshape([1,0],(8,9))[All, 0..7] We could then eliminate special case constructors like zeros(). What you suggest would be very easy to code, but I don't want it included in the behavior for reshape. I like having reshape error check my dimensions for me, and I like the fact that reshape doesn't copy the whole array, but just returns a new pointer to it. However, I have no objections to adding another method which does what you're asking for. What would you like it to be called? Another important function that seems to be missing is appending, i.e. the equivalent of + for lists. Of course for arrays it has to work along any axis. Try this: a = array([1,2,3]) a.concat([4,5], [7,8]) -> array([1,2,3,4,5,7,8]) I assume that this is what you're looking for? It should probably work along any axis, but I don't have a terribly good way to specify the axis and still keep the nice feature of being able to concat together an arbitrary number of arrays (very important for efficiency). It would also be a bit of a hassle to write in C the way you're suggesting, would it be enough to add in a standard python function with the desired behavior? -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 30 12:43:47 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA26202 for matrix-sig-people; Tue, 30 Jan 1996 12:43:47 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id MAA26198 for ; Tue, 30 Jan 1996 12:43:42 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA03714; Tue, 30 Jan 96 12:42:49 EST Date: Tue, 30 Jan 96 12:42:49 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601301742.AA03714@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: da@maigret.cog.brown.edu Cc: hinsenk@ere.umontreal.ca, matrix-sig@python.org In-Reply-To: <199601301530.KAA18459@maigret> (da@maigret.cog.brown.edu) Subject: [PYTHON MATRIX-SIG] Pseudo Indices Sender: owner-matrix-sig@python.org Precedence: bulk Well, here's a fairly long and slightly confusing response, but what do you expect from a first draft? The idea of pseudo indices comes from the language Yorick created by Dave Munro at LLNL, not from APL/J. Everybody expects this to work: a = array([1,2,3]) a+2 -> array([3,4,5]) 2 is 0-dimensional array with shape [], and a has shape [3]. The rule says that any non-existent dimension is "broadcast" to match a higher dimensional array as required. The idea is to extend this notion of broadcasting to any dimension of length one. So the following will also work: a+array([2]) -> array([3,4,5]) array([2]) has shape [1], and a has shape [3]. The [1] is broadcast to match the [3]. The classic example of where this is useful is in taking an outer product. b = array([10, 20]) a * b.reshape(2, 1) -> array([[11, 12, 13], [21, 22, 23]]) a has shape [3], and b.reshape(2,1) has shape [2, 1]. The result of a binary operator applied to them has shape [2,3]. In order to make such code easier to read, Yorick introduces the notion of a pseudo index that can be added to any multidimensional index and means, add a new dimension here. ie. b[2, PseudoIndex] <--> b.reshape(2, 1) "All" is not in fact related to Yorick pseudo indices at all, but is related to Slices. c = array([[1,2,3], [11,12,13]]) I'd really like to be able to say c[:, 3] -> array([3,13]). Python syntax doesn't currently allow this, so until somebody convinces Guido to make the change, the following objects are used instead. c[:, 1:2] --> c[Slice(), Slice(1,2)] All = Slice() One more piece is needed to handle multidimensional indexing properly. Let's say that I want to extract the third element from the last dimension of an array. In Yorick that would look like a[.., 3]. Since Python doesn't yet have this rubber index, I use the following: a[RubberIndex, 3]. So, the following special symbols are needed to properly treat multidimensional indexing: Slice PseudoIndex RubberIndex My current feeling on the best way to do this is to keep Slice's as they are (until somebody lobbys Guido for a better system). Define string constants NewAxis = "NewAxis" Ellipses = "Ellipses" NewAxis is equivalent to the current "None" and indicates a PseudoIndex Ellipses is equivalent to the current "RubberIndex" and indicates standard ellipses as in, fill in what should naturally go here. Comments? -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 30 12:55:31 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA26315 for matrix-sig-people; Tue, 30 Jan 1996 12:55:31 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id MAA26310 for ; Tue, 30 Jan 1996 12:55:28 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA03750; Tue, 30 Jan 96 12:54:22 EST Date: Tue, 30 Jan 96 12:54:22 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601301754.AA03750@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: hinsenk@ere.umontreal.ca Cc: matrix-sig@python.org In-Reply-To: <199601301726.MAA16169@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA) Subject: Re: [PYTHON MATRIX-SIG] First experiences Sender: owner-matrix-sig@python.org Precedence: bulk From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad) I don't know whether or not sum even belongs in the set of standard functions. On the one hand I'd rather make people use add.reduce because it's much more in the spirit of the array object. On the other hand, sum is such a friendly little function. If it remains in the system, then I agree it should get the second argument. In the spirit of "no needless redundancies", it would be better not to have this function. Maybe we should add an (optional) module with standard abbreviations, where we would simply write sum = add.reduce. That would also be a good way to have clearer standard names for things like arange() while still having short alternatives that are available on every system. I think I like this idea, I'll play with it and see what I come up with. What you suggest would be very easy to code, but I don't want it included in the behavior for reshape. I like having reshape error check my dimensions for me, and I like the fact that reshape doesn't copy the whole array, but just returns a new pointer to it. It could still return a new pointer if the size is the same, and make a copy only if the size changes. Personally I don't care This I REALLY don't want. I firmly believe that whether a function returns a pointer or a copy should not be dependent on its arguments. about dimension checking; in all my APL code that has never been a problem. If the size remains the same, then typically the new dimensions are specified as an expression involving the old dimensions. I have never had an error there. Try this: a = array([1,2,3]) a.concat([4,5], [7,8]) -> array([1,2,3,4,5,7,8]) I assume that this is what you're looking for? Not really. First of all, in most cases I need a non-destructive version, similar to sequence addition. Second, I really need it for all dimensions. In my application, I have two 1d arrays of equal length and have to construct a 2d array from them. What I used is c = array([tuple(a), tuple(b)]) but there ought to be a much cleaner and more efficient way. Well, you should use c = array([a, b]) for that. This seems fairly clean to me, and will run quite efficiently if a and b are already arrays of the same type. What do you mean by non-destructive? a.concat(b) does not modify a, but instead returns a new array that is b concatenated with a. This is almost exactly equivalent to a+b if a and b were lists. BTW, we also ought to have a way to convert arrays to nested lists and maybe tuples. Oddly enough, Python has no built-in function list(), although it does have tuple(). I agree, I'll add this when I get the time. I'll probablyu call it toList() to go along with toString and toFile. It should probably work along any axis, but I don't have a terribly good way to specify the axis and still keep the nice feature of being able to concat together an arbitrary number of arrays (very important One argument could be a tuple of all arrays to be concatenated. This doesn't work nicely if you only have one array. for efficiency). It would also be a bit of a hassle to write in C the way you're suggesting, would it be enough to add in a standard python function with the desired behavior? That's just a matter of efficiency and has to be tried. In my APL code this function is used a lot, and often on small arrays. Again, what would you call this new method? -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 30 13:06:27 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA26419 for matrix-sig-people; Tue, 30 Jan 1996 13:06:27 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id NAA26415 for ; Tue, 30 Jan 1996 13:06:24 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA02599; Tue, 30 Jan 96 10:05:33 PST Received: from kristen (localhost) by kristen.llnl.gov (5.x/SMI-SVR4) id AA20024; Tue, 30 Jan 1996 10:05:23 -0800 Message-Id: <310E5DE2.5E40@llnl.gov> Date: Tue, 30 Jan 1996 10:05:22 -0800 From: "Paul. Dubois" Organization: Lawrence Livermore National Laboratory X-Mailer: Mozilla 2.0b5 (X11; I; SunOS 5.4 sun4m) Mime-Version: 1.0 To: equipe-basis@lists.llnl.gov, matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Uniform random number generator v. 2.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk On ftp-icf.llnl.gov, directory /pub/basis, lies: URNG-2.0.tar.gz which contains a C extension and Python wrapper for a random number generator equivalent to the one on Cray systems. This version works with the Numeric extension 0.32. (It contains a feature for returning a random sample or randomly-filled array of arbitrary shape, as well as a more pedestrian "ranf()".) URNG can create separate generators that act independently for use in different modules. Note to Jim Hugunin: I can't remember how I "delivered" this to your ftp directory before... -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 30 13:12:46 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA26488 for matrix-sig-people; Tue, 30 Jan 1996 13:12:46 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id NAA26483 for ; Tue, 30 Jan 1996 13:12:44 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa12302; 30 Jan 96 13:11 EST Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id NAA08602; Tue, 30 Jan 1996 13:09:42 -0500 Message-Id: <199601301809.NAA08602@monty> To: James Hugunin cc: da@maigret.cog.brown.edu, hinsenk@ere.umontreal.ca, matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Pseudo Indices In-reply-to: Your message of "Tue, 30 Jan 1996 12:42:49 EST." <9601301742.AA03714@baribal.LCS.MIT.EDU.LCS.MIT.EDU> References: <9601301742.AA03714@baribal.LCS.MIT.EDU.LCS.MIT.EDU> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Tue, 30 Jan 1996 13:09:42 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk > ie. b[2, PseudoIndex] <--> b.reshape(2, 1) I'm not sure I understand reshape completely, but isn't this a rather drastic reinterpretation of the index arguments when a pseudo index is present? In general, if no pseudo index is present, single indices take away that dimension, don't they? (While slice indices can reduce the length in that dimension but otherwise leave it intact.) But in your example, the '2' becomes a shorthand for a slice. I don't like this very much... > I'd really like to be able to say c[:, 3] -> array([3,13]). Python > syntax doesn't currently allow this, so until somebody convinces Guido > to make the change, the following objects are used instead. Same comment -- I don't like the '3' index to turn into a slice by context, if I understand this right. (Bear with me, I'm not even sure that array([3, 13]) really is a 3x13 matrix.) > c[:, 1:2] --> c[Slice(), Slice(1,2)] This is fine (and what I thought you were proposing all the time). I would consider a patch if the existing 1-dim sequence and mapping types are not affected. > a[RubberIndex, 3]. You could sneak this into the abovementioned patch as "a[::, 3]". > (until somebody lobbys Guido for a better system). Ain't I nice today? ;-) --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 30 13:14:02 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA26516 for matrix-sig-people; Tue, 30 Jan 1996 13:14:02 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id NAA26512 for ; Tue, 30 Jan 1996 13:13:59 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA03881; Tue, 30 Jan 96 13:13:06 EST Date: Tue, 30 Jan 96 13:13:06 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601301813.AA03881@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: dubois1@llnl.gov Cc: matrix-sig@python.org In-Reply-To: <310E5DE2.5E40@llnl.gov> (dubois1@llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] Uniform random number generator v. 2.0 Sender: owner-matrix-sig@python.org Precedence: bulk Note to Jim Hugunin: I can't remember how I "delivered" this to your ftp directory before... Our sysadmins plugged a security hole that had made it so convenient for people to upload code like this to my ftp directory in the past. Since I still like the idea of keeping things like this in one place, I'll take it on myself to download code like this to sls-ftp.lcs.mit.edu into directory /pub/jjh. This is already done for Paul's URNG package. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 30 13:36:05 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA26709 for matrix-sig-people; Tue, 30 Jan 1996 13:36:05 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id NAA26705 for ; Tue, 30 Jan 1996 13:36:02 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA03955; Tue, 30 Jan 96 13:34:54 EST Date: Tue, 30 Jan 96 13:34:54 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601301834.AA03955@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: guido@CNRI.Reston.VA.US Cc: matrix-sig@python.org In-Reply-To: <199601301809.NAA08602@monty> (message from Guido van Rossum on Tue, 30 Jan 1996 13:09:42 -0500) Subject: Re: [PYTHON MATRIX-SIG] Pseudo Indices Sender: owner-matrix-sig@python.org Precedence: bulk From: Guido van Rossum > ie. b[2, PseudoIndex] <--> b.reshape(2, 1) I'm not sure I understand reshape completely, but isn't this a rather drastic reinterpretation of the index arguments when a pseudo index is present? In general, if no pseudo index is present, single indices take away that dimension, don't they? (While slice indices can reduce the length in that dimension but otherwise leave it intact.) But in your example, the '2' becomes a shorthand for a slice. I don't like this very much... I'm sorry, that was a type-o on my part. What I should have said was b[All, PseudoIndex] <--> b.reshape(2,1). > I'd really like to be able to say c[:, 3] -> array([3,13]). Python > syntax doesn't currently allow this, so until somebody convinces Guido > to make the change, the following objects are used instead. Same comment -- I don't like the '3' index to turn into a slice by context, if I understand this right. (Bear with me, I'm not even sure that array([3, 13]) really is a 3x13 matrix.) Here you are simply not understanding my (admittedly poor) description. array([3,13]) is a one dimensional array with a shape of [2] which contains the numeric elements 3 and 13. So the 3 index is not returning a slice by context. The returned array has a dimension one less that the initial array because it has the index 3 instead of a slice. > c[:, 1:2] --> c[Slice(), Slice(1,2)] This is fine (and what I thought you were proposing all the time). I would consider a patch if the existing 1-dim sequence and mapping types are not affected. Great! I'll get on with the process of coordinating this patch. Currently the best way we've come up to implement this patch is to add a new object type to python called a sliceobject. c[:, 1:2] then gets converted to c[ (slice(), slice(1,2)) ] in the interpreter and then passed to the standard getitem mapping method. Special consideration is payed to c[:] so as not to break existing code. I'm a little bit troubled by the need to add a new object (admittedly extremely small) just to get this behavior. Does this solution seem reasonable to you? The only other approach I can think of is to add a new protocol for multidimensional arrays, on equal footing with number, mapping, and sequence. This seems like it would be overkill. > a[RubberIndex, 3]. You could sneak this into the abovementioned patch as "a[::, 3]". Slices have an additional feature which is a third index specifying a stride. ie. Slice(None,None,2) will take all the even elements along a dimension. Slice(None,None,-1) will reverse a dimension. This arrangement corresponds very nicely with the internals of array objects. Unfortunately, this means that things like ::-1 will be reasonably common, and so using :: as a RubberIndex might be confusing. However, I do like this syntax for the RubberIndex as it really captures what it does which is to apply : to every unused dimension. Maybe there's a similar proposal that will work. > (until somebody lobbys Guido for a better system). Ain't I nice today? ;-) Gee, now that we've got Guido in such a friendly mood, what else should we ask for? A new operator just for matrix multiplication? ;) -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 30 14:23:22 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA27177 for matrix-sig-people; Tue, 30 Jan 1996 14:23:22 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id OAA27172 for ; Tue, 30 Jan 1996 14:23:11 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id OAA07345 (8.6.11/IDA-1.6); Tue, 30 Jan 1996 14:21:13 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA26983; Tue, 30 Jan 1996 14:21:12 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA23872; Tue, 30 Jan 1996 14:21:11 -0500 Date: Tue, 30 Jan 1996 14:21:11 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601301921.OAA23872@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: da@maigret.cog.brown.edu, matrix-sig@python.org In-reply-to: <9601301742.AA03714@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: [PYTHON MATRIX-SIG] Re: Pseudo Indices Sender: owner-matrix-sig@python.org Precedence: bulk In order to make such code easier to read, Yorick introduces the notion of a pseudo index that can be added to any multidimensional index and means, add a new dimension here. ie. b[2, PseudoIndex] <--> b.reshape(2, 1) Not quite. "2" means "the third element". "All" is not in fact related to Yorick pseudo indices at all, but is related to Slices. But "All" is the equivalent of the Yorick pseudoindex *, which of course is also a special form of a slice. c = array([[1,2,3], [11,12,13]]) I'd really like to be able to say c[:, 3] -> array([3,13]). Python Small correction: c[:, 2]. In Yorick that would look like a[.., 3]. Since Python doesn't yet have this rubber index, I use the following: a[RubberIndex, 3]. That should also get a better name... I think I already made some proposals, I just have to dig that message out again... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 30 14:35:43 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA27268 for matrix-sig-people; Tue, 30 Jan 1996 14:35:43 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id OAA27264 for ; Tue, 30 Jan 1996 14:35:35 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id OAA08204 (8.6.11/IDA-1.6); Tue, 30 Jan 1996 14:33:43 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA02896; Tue, 30 Jan 1996 14:33:42 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA24990; Tue, 30 Jan 1996 14:33:41 -0500 Date: Tue, 30 Jan 1996 14:33:41 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601301933.OAA24990@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: matrix-sig@python.org In-reply-to: <9601301754.AA03750@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] First experiences Sender: owner-matrix-sig@python.org Precedence: bulk It could still return a new pointer if the size is the same, and make a copy only if the size changes. Personally I don't care This I REALLY don't want. I firmly believe that whether a function returns a pointer or a copy should not be dependent on its arguments. You are right. I had forgotten that returning a pointer or returning a copy makes a real difference here; I am probably too much infected by J, in which arrays are immutable objects. And the more I think about it, the more I like immutable arrays! But that's another point. Well, you should use c = array([a, b]) for that. This seems fairly clean to me, and will run quite efficiently if a and b are already arrays of the same type. I didn't know this was possible (of course I should have tried...). But still that is a solution only if I want to create a new first axis. What if I don't want a new axis, or if the new axis is not the first one? What do you mean by non-destructive? a.concat(b) does not modify a, but instead returns a new array that is b concatenated with a. This is almost exactly equivalent to a+b if a and b were lists. OK, I misunderstood this. One argument could be a tuple of all arrays to be concatenated. This doesn't work nicely if you only have one array. Not as long as it is a method. But concat() could be made a normal function. Actually, that is another slightly unclear point in the current array module. Some operations are functions (e.g. reshape()), others are methods (like concat()), but that seems to be completely arbitrary. That's just a matter of efficiency and has to be tried. In my APL code this function is used a lot, and often on small arrays. Again, what would you call this new method? Ehh... Which one? Concatenation? Well, I still hope to find a solution that incorporates all applications into one, which can then be called concat(). We should try to keep the number of functions/methods to a minimum. So how about this: function concat((a,b,c...), n=0) concatenates arrays a,b,c,... along the nth axis and returns the result. The shape along the other axes must agree. This doesn't take care of new axes, but they can be added easily with the pseudo index. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 30 14:51:45 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA27453 for matrix-sig-people; Tue, 30 Jan 1996 14:51:45 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id OAA27449 for ; Tue, 30 Jan 1996 14:51:42 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA04262; Tue, 30 Jan 96 14:50:41 EST Date: Tue, 30 Jan 96 14:50:41 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601301950.AA04262@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: hinsenk@ere.umontreal.ca Cc: matrix-sig@python.org In-Reply-To: <199601301933.OAA24990@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA) Subject: Re: [PYTHON MATRIX-SIG] First experiences Sender: owner-matrix-sig@python.org Precedence: bulk From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad) You are right. I had forgotten that returning a pointer or returning a copy makes a real difference here; I am probably too much infected by J, in which arrays are immutable objects. And the more I think about it, the more I like immutable arrays! But that's another point. I rather like immutable arrays myself, but I was convinced that mutable arrays were the right choice sometime in the middle of this project, and I'm unlikely to change again. current array module. Some operations are functions (e.g. reshape()), others are methods (like concat()), but that seems to be completely arbitrary. Actually, reshape is now an array method, and the old reshape function is only around for "historical reasons". I should probably remove it. That's just a matter of efficiency and has to be tried. In my APL code this function is used a lot, and often on small arrays. Again, what would you call this new method? Ehh... Which one? Concatenation? Well, I still hope to find a solution that incorporates all applications into one, which can then be called concat(). We should try to keep the number of functions/methods to a minimum. So how about this: function concat((a,b,c...), n=0) concatenates arrays a,b,c,... along the nth axis and returns the result. The shape along the other axes must agree. This doesn't take care of new axes, but they can be added easily with the pseudo index. This can easily be added to the standard set of functions and implemented on top of the existing transpose and concat methods. You might want to do this as an exercise ;). I was actually asking for a new name for the method that will do a reshape allowing you to make copies of the array in the process. I'm fairly convinced that I don't want this to be the default behavior of the reshape method. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 30 15:03:39 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA27553 for matrix-sig-people; Tue, 30 Jan 1996 15:03:39 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id PAA27549 for ; Tue, 30 Jan 1996 15:03:35 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id PAA09026 (8.6.11/IDA-1.6); Tue, 30 Jan 1996 15:01:02 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA11051; Tue, 30 Jan 1996 15:01:01 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA26746; Tue, 30 Jan 1996 15:00:59 -0500 Date: Tue, 30 Jan 1996 15:00:59 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601302000.PAA26746@cyclone.ERE.UMontreal.CA> To: guido@CNRI.Reston.VA.US CC: jjh@goldilocks.lcs.mit.edu, da@maigret.cog.brown.edu, matrix-sig@python.org In-reply-to: <199601301809.NAA08602@monty> (message from Guido van Rossum on Tue, 30 Jan 1996 13:09:42 -0500) Subject: Re: [PYTHON MATRIX-SIG] Pseudo Indices Sender: owner-matrix-sig@python.org Precedence: bulk > c[:, 1:2] --> c[Slice(), Slice(1,2)] This is fine (and what I thought you were proposing all the time). I would consider a patch if the existing 1-dim sequence and mapping types are not affected. What do you mean by "not affected"? That their semantics stays the same (no problem) or that their implementation needn't be changed? I am not sure the latter is possible for C-types. > a[RubberIndex, 3]. You could sneak this into the abovementioned patch as "a[::, 3]". I like that idea... If I had enough time, I'd look at this, but that might not happen for a while. Any other volunteers? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 30 15:27:42 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA27755 for matrix-sig-people; Tue, 30 Jan 1996 15:27:42 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id PAA27751 for ; Tue, 30 Jan 1996 15:27:40 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA04461; Tue, 30 Jan 96 15:26:30 EST Date: Tue, 30 Jan 96 15:26:30 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601302026.AA04461@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: hinsenk@ere.umontreal.ca Cc: guido@CNRI.Reston.VA.US, da@maigret.cog.brown.edu, matrix-sig@python.org In-Reply-To: <199601302000.PAA26746@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA) Subject: Re: [PYTHON MATRIX-SIG] Pseudo Indices Sender: owner-matrix-sig@python.org Precedence: bulk From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad) > c[:, 1:2] --> c[Slice(), Slice(1,2)] This is fine (and what I thought you were proposing all the time). I would consider a patch if the existing 1-dim sequence and mapping types are not affected. What do you mean by "not affected"? That their semantics stays the same (no problem) or that their implementation needn't be changed? I am not sure the latter is possible for C-types. Chris Chase actually has a prototype implementation that doesn't alter the implementation for C types. Basically it special cases these calls. > a[RubberIndex, 3]. You could sneak this into the abovementioned patch as "a[::, 3]". I like that idea... If I had enough time, I'd look at this, but that might not happen for a while. Any other volunteers? Again, Chris Chase has a good starting point for an implementation of this. I'll probably polish it up myself (unless I can talk him into spending some more time on it) and add it to the NumericPython distribution for testing. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 30 15:48:22 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA27933 for matrix-sig-people; Tue, 30 Jan 1996 15:48:22 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id PAA27927 for ; Tue, 30 Jan 1996 15:48:16 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id PAA11508 (8.6.11/IDA-1.6); Tue, 30 Jan 1996 15:43:24 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA20666; Tue, 30 Jan 1996 15:43:24 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA29453; Tue, 30 Jan 1996 15:43:22 -0500 Date: Tue, 30 Jan 1996 15:43:22 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601302043.PAA29453@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: matrix-sig@python.org In-reply-to: <9601301950.AA04262@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] First experiences Sender: owner-matrix-sig@python.org Precedence: bulk Actually, reshape is now an array method, and the old reshape function is only around for "historical reasons". I should probably remove it. One or the other. I am not sure that "everything as a method" is the right strategy. For example, operations that have two symmetric operands look a bit odd as methods. There's also a difference in behaviour, since array functions accept nested lists instead of arrays, but methods can be called only on previously existing array objects. In the case of the "copying reshape", the array argument would often be a constant and it would make perfect sense to allow nested lists there. A useful way to make use of the function/method distinction would be to use methods for operations that modify an array and functions for operations that don't. The modified array would then be the one on which the method is called, and it makes sense to require that this actually is an array. There are not too many in-place operations right now, but there will be once topics like linear algebra are dealt with. For example, it would be nice to have m.invert() for an in-place inversion and inverse(m) for a function that returns the inverse without changing m. This can easily be added to the standard set of functions and implemented on top of the existing transpose and concat methods. You might want to do this as an exercise ;). No problem, but that would certainly be slow! I was actually asking for a new name for the method that will do a reshape allowing you to make copies of the array in the process. I'm I see. Now that's difficult. I have been calling this operation "reshape" since my high school days, so it is not easy to come up with something else! Give me some time... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 30 15:54:13 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA27991 for matrix-sig-people; Tue, 30 Jan 1996 15:54:13 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id PAA27985 for ; Tue, 30 Jan 1996 15:54:08 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id MAA03339 (8.6.11/IDA-1.6); Tue, 30 Jan 1996 12:26:13 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA12374; Tue, 30 Jan 1996 12:26:12 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA16169; Tue, 30 Jan 1996 12:26:11 -0500 Date: Tue, 30 Jan 1996 12:26:11 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199601301726.MAA16169@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: matrix-sig@python.org In-reply-to: <9601301635.AA03443@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] First experiences Sender: owner-matrix-sig@python.org Precedence: bulk I don't know whether or not sum even belongs in the set of standard functions. On the one hand I'd rather make people use add.reduce because it's much more in the spirit of the array object. On the other hand, sum is such a friendly little function. If it remains in the system, then I agree it should get the second argument. In the spirit of "no needless redundancies", it would be better not to have this function. Maybe we should add an (optional) module with standard abbreviations, where we would simply write sum = add.reduce. That would also be a good way to have clearer standard names for things like arange() while still having short alternatives that are available on every system. And vice versa. This is mainly a matter of personal taste, and since my taste favors fast_umath, that's going to be the default version for Its not personal taste, but consistency. I have replaced "fast_umath" by "umath" in my installation, but then immediately added "from fast_umath import *" to my only application. I expect to use mostly the fast version for debugged code, but still I think that the default should obey the principle of least surprise, and that is maximum similarity to Python scalar behaviour. What you suggest would be very easy to code, but I don't want it included in the behavior for reshape. I like having reshape error check my dimensions for me, and I like the fact that reshape doesn't copy the whole array, but just returns a new pointer to it. It could still return a new pointer if the size is the same, and make a copy only if the size changes. Personally I don't care about dimension checking; in all my APL code that has never been a problem. If the size remains the same, then typically the new dimensions are specified as an expression involving the old dimensions. I have never had an error there. Try this: a = array([1,2,3]) a.concat([4,5], [7,8]) -> array([1,2,3,4,5,7,8]) I assume that this is what you're looking for? Not really. First of all, in most cases I need a non-destructive version, similar to sequence addition. Second, I really need it for all dimensions. In my application, I have two 1d arrays of equal length and have to construct a 2d array from them. What I used is c = array([tuple(a), tuple(b)]) but there ought to be a much cleaner and more efficient way. BTW, we also ought to have a way to convert arrays to nested lists and maybe tuples. Oddly enough, Python has no built-in function list(), although it does have tuple(). It should probably work along any axis, but I don't have a terribly good way to specify the axis and still keep the nice feature of being able to concat together an arbitrary number of arrays (very important One argument could be a tuple of all arrays to be concatenated. for efficiency). It would also be a bit of a hassle to write in C the way you're suggesting, would it be enough to add in a standard python function with the desired behavior? That's just a matter of efficiency and has to be tried. In my APL code this function is used a lot, and often on small arrays. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Jan 30 17:48:30 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA28768 for matrix-sig-people; Tue, 30 Jan 1996 17:48:30 -0500 Received: from bartel.jhuapl.edu (bartel.jhuapl.edu [128.244.146.143]) by python.org (8.6.12/8.6.12) with SMTP id RAA28764 for ; Tue, 30 Jan 1996 17:48:28 -0500 Message-Id: <199601302248.RAA28764@python.org> Received: by bartel.jhuapl.edu (1.38.193.4/16.2) id AA26428; Tue, 30 Jan 1996 17:53:56 -0500 Date: Tue, 30 Jan 1996 17:53:56 -0500 From: Chris Chase S1A Cc: Guido van Rossum To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Pseudo Indices In-Reply-To: <199601301809.NAA08602@monty> References: <9601301742.AA03714@baribal.LCS.MIT.EDU.LCS.MIT.EDU> <199601301809.NAA08602@monty> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk >> a[RubberIndex, 3]. Guido> You could sneak this into the abovementioned patch as "a[::, 3]". Actually, All is equivalent to "::" and it corresponds to the nil index syntax of Yorick (which is indicated by whitespace between the commas) and the * index syntax of Matlab or IDL. Rubber indexes are different. Yorick has two types of rubber indexes, one has syntax "*" and the other is ".." and neither type has the same behavior as "::". Both types act as if the entire length of the missing intervening dimensions were selected. "*" collapses the missing intervening dimensions into 1 dimension. ".." does not collapse the intervening dimensions (it is like using All for the missing dimensions). Currently the Numeric module RubberIndex is like the Yorick ".." rubber index. I imagine that the collapsing type rubber index could easily be added by applying a shape change to the current RubberIndex behavior. What would be a possible name for this type of rubber index? CollapseIndex? Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Jan 31 10:57:35 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA01650 for matrix-sig-people; Wed, 31 Jan 1996 10:57:35 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id KAA01646 for ; Wed, 31 Jan 1996 10:57:32 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA19776; Wed, 31 Jan 96 10:56:04 EST Date: Wed, 31 Jan 96 10:56:04 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9601311556.AA19776@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: chris.chase@jhuapl.edu In-Reply-To: <199601302248.RAA28764@python.org> (message from Chris Chase S1A on Tue, 30 Jan 1996 17:53:56 -0500) Cc: jjh, matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Slices as : Sender: owner-matrix-sig@python.org Precedence: bulk The following is mainly directed to Chris, but I thought that people on the list might be interested, so I decided to include the CC. Well, it looks like your slices patch is going to get itself added to python after all. Guido turned out to be a softer touch than I'd expected. If you'd been at the workshop when I started talking about rubber indices, you'd better understand my fears. I'd like to see a couple of changes to your slices patch to make my life easier for adding it into the NumericPython distribution. Could you prepare a version of your patches that include both 1::2 -> slice(1,None,2) and ::: or .. (you pick, but it might get changed by popular vote) -> Py_Ellipses (a special object just like Py_None) I'd also prefer to see 1:2:3:4 -> Syntax Error rather than a runtime error. I know that a[:] is unchanged by your patches for backwards compatibilty. It would be nice if a[::-1] would be turned into a[slice(None,None,-1) so that it could be passed to the mapping protocol rather than being a syntax error. I don't think that pseudo indices should have special notation as these are particular to the array object. Konrad's name of Numeric.NewAxis strikes me as a good idea. I also don't think that it is necessary to add special notation for the compressing rubber index as this is done easily enough with reshape when needed. Does this all sound reasonable to you? Could you prepare this as a patch on top of the 0.32 distribution (on top of Konrad's existing patches)? As soon as you get this patch to me I'll add it into my working version and to the next distribution. I understand other commitments, but the sooner you could get this to me the better. Thanks for your help - Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 1 19:36:43 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id TAA13588 for matrix-sig-people; Thu, 1 Feb 1996 19:36:43 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id TAA13584 for ; Thu, 1 Feb 1996 19:36:38 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA17731; Thu, 1 Feb 96 19:29:36 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) for matrix-sig@python.org id TAA28070; Thu, 1 Feb 1996 19:40:46 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199602020040.TAA28070@maigret> Subject: [PYTHON MATRIX-SIG] lists vs. tuples To: matrix-sig@python.org Date: Thu, 1 Feb 1996 19:40:45 -0500 (EST) X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 438 Sender: owner-matrix-sig@python.org Precedence: bulk Just a thought. I believe that most existing python code, when returning collections, return tuples. The matrix code as implemented returns lists (e.g. shape). Shape strikes me as the kind of thing which might be better off being a tuple than a list. That way people doing: a = array(...) s = a.shape() s[0] = 3 would be stopped by the interpreter (I'm assuming this isn't working and isn't likely to work). Thoughts? --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 1 20:03:17 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id UAA13685 for matrix-sig-people; Thu, 1 Feb 1996 20:03:17 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id UAA13680 for ; Thu, 1 Feb 1996 20:03:13 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id UAA00340 (8.6.11/IDA-1.6); Thu, 1 Feb 1996 20:00:33 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id UAA00432; Thu, 1 Feb 1996 20:00:30 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id UAA08987; Thu, 1 Feb 1996 20:00:29 -0500 Date: Thu, 1 Feb 1996 20:00:29 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199602020100.UAA08987@cyclone.ERE.UMontreal.CA> To: da@maigret.cog.brown.edu CC: matrix-sig@python.org In-reply-to: <199602020040.TAA28070@maigret> (da@maigret.cog.brown.edu) Subject: Re: [PYTHON MATRIX-SIG] lists vs. tuples Sender: owner-matrix-sig@python.org Precedence: bulk I believe that most existing python code, when returning collections, return tuples. I'd say it's mostly lists, but that's just my personal experience. The matrix code as implemented returns lists (e.g. shape). Shape Well, my installation returns tuples for shapes! Are we talking about the same thing? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Feb 2 10:33:01 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA17812 for matrix-sig-people; Fri, 2 Feb 1996 10:33:01 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id KAA17808 for ; Fri, 2 Feb 1996 10:32:56 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA01424; Fri, 2 Feb 96 10:31:26 EST Date: Fri, 2 Feb 96 10:31:26 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9602021531.AA01424@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: da@maigret.cog.brown.edu Cc: matrix-sig@python.org In-Reply-To: <199602020040.TAA28070@maigret> (da@maigret.cog.brown.edu) Subject: Re: [PYTHON MATRIX-SIG] lists vs. tuples Sender: owner-matrix-sig@python.org Precedence: bulk From: da@maigret.cog.brown.edu (David Ascher) Just a thought. I believe that most existing python code, when returning collections, return tuples. The matrix code as implemented returns lists (e.g. shape). Shape strikes me as the kind of thing which might be better off being a tuple than a list. That way people doing: a = array(...) s = a.shape() s[0] = 3 would be stopped by the interpreter (I'm assuming this isn't working and isn't likely to work). Your code fragment won't work for two reasons. First, a.shape is a data member, not a function (this is because you can do a.shape = new_shape). And the data member a.shape is a tuple. You are allowed to assign to a.shape from any sequence type, but it can only be accessed as a tuple. Are you thinking of some other problem? -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Feb 2 10:38:27 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA17857 for matrix-sig-people; Fri, 2 Feb 1996 10:38:27 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id KAA17853 for ; Fri, 2 Feb 1996 10:38:24 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA18167; Fri, 2 Feb 96 10:31:11 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) for matrix-sig@python.org id KAA29265; Fri, 2 Feb 1996 10:42:23 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199602021542.KAA29265@maigret> Subject: Re: [PYTHON MATRIX-SIG] lists vs. tuples To: jjh@goldilocks.lcs.mit.edu (James Hugunin) Date: Fri, 2 Feb 1996 10:42:07 -0500 (EST) In-Reply-To: <9602021531.AA01424@baribal.LCS.MIT.EDU.LCS.MIT.EDU> from "James Hugunin" at Feb 2, 96 10:31:26 am X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 624 Sender: owner-matrix-sig@python.org Precedence: bulk > And the data member a.shape is a tuple. You are allowed > to assign to a.shape from any sequence type, but it can only be > accessed as a tuple. > >> The matrix code as implemented returns lists (e.g. shape). Shape > Well, my installation returns tuples for shapes! Are we talking about > the same thing? Hmm. Sorry. I was remembering Jim's description of the pseudoindices etc., where he says things like "array([3,13]) is a one dimensional array with a shape of [2]". Clearly the implemenation is doing the right thing. Sorry for decreasing the signal-to-noise ratio... --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Feb 2 13:05:08 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA18927 for matrix-sig-people; Fri, 2 Feb 1996 13:05:08 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id NAA18922 for ; Fri, 2 Feb 1996 13:05:04 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA02697; Fri, 2 Feb 96 13:03:31 EST Date: Fri, 2 Feb 96 13:03:31 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9602021803.AA02697@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Bug-Fix Release 0.33 available Sender: owner-matrix-sig@python.org Precedence: bulk Bug-Fix Release 0.33 of the numerical extension to python is available. New in this release: Several small BUG fixes. Hopefully this release will compile "out of the box". Konrad contributed a first cut at a decent print function for arrays An expanded TUTORIAL.NumPy mrange is now arange and honors the typecode (thanks Perry) the function reshape is gone, use the method instead A new function copyAndReshape is available that does what APL folks expect from reshape (I think). A better name would be appreciated. You can try picking up ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumPy.patch-0.32-0.33.gz If you copy this to the root python directory and type patch < patch-0.32-0.33 You might have managed to pick up the latest release. After remaking your system, rerun the test suite, and if the matrices are printed in a pretty fashion, you'll know that the patch worked. This is the first time that I've tried distributing a patch, so please let me know whether or not this works for you. If you're having problems, or don't trust patch, the complete tar file is also available. ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericalPython-0.33.tar.gz Enjoy - Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Feb 2 14:54:24 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA19667 for matrix-sig-people; Fri, 2 Feb 1996 14:54:24 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id OAA19663 for ; Fri, 2 Feb 1996 14:54:18 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id OAA02535 (8.6.11/IDA-1.6); Fri, 2 Feb 1996 14:51:10 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA18761; Fri, 2 Feb 1996 14:51:09 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA07396; Fri, 2 Feb 1996 14:51:08 -0500 Date: Fri, 2 Feb 1996 14:51:08 -0500 From: hinsenk@ere.umontreal.ca (Hinsen Konrad) Message-Id: <199602021951.OAA07396@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: matrix-sig@python.org In-reply-to: <9602021803.AA02697@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] Bug-Fix Release 0.33 available Sender: owner-matrix-sig@python.org Precedence: bulk This is the first time that I've tried distributing a patch, so please let me know whether or not this works for you. No. Patch kept asking me which file it should patch (as if I could know that!), and I preferred downloading the complete tar file to ruining my installation. Otherwise the new version seems to work; at least the test suite runs without problems. The new function copyAndReshape doesn't work at all - it references an undefined name "ArrayType". I replaced that by type(array([0])), but then I got another error later on, telling me that the "dimensions are not legal". That of course should never happen if the function does what it is supposed to do. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Feb 2 16:16:22 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA20183 for matrix-sig-people; Fri, 2 Feb 1996 16:16:22 -0500 Received: from hurlbut. (hurlbut.jhuapl.edu [128.244.146.28]) by python.org (8.6.12/8.6.12) with SMTP id QAA20179 for ; Fri, 2 Feb 1996 16:16:13 -0500 Received: by hurlbut. (5.0/SMI-SVR4) id AA04256; Fri, 2 Feb 1996 15:54:20 +0500 Date: Fri, 2 Feb 1996 15:54:20 +0500 Message-Id: <9602022054.AA04256@hurlbut.> From: Chris Chase S1A To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Bug-Fix Release 0.33 available In-Reply-To: <199602021951.OAA07396@cyclone.ERE.UMontreal.CA> References: <9602021803.AA02697@baribal.LCS.MIT.EDU.LCS.MIT.EDU> <199602021951.OAA07396@cyclone.ERE.UMontreal.CA> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII content-length: 0 Sender: owner-matrix-sig@python.org Precedence: bulk Hinsen> No. Patch kept asking me which file it should patch (as if I could Hinsen> know that!), and I preferred downloading the complete tar file Hinsen> to ruining my installation. I got the patch to work. After patching the first few files, my patch program also started asking what file to patch. I just copied and pasted the file name shown from the patch header, i.e., when patch showed: diff -c -r OldVersion/Modules/ofuncobject.c ./Modules/ofuncobject.c *** OldVersion/Modules/ofuncobject.c Mon Jan 29 10:29:30 1996 --- ./Modules/ofuncobject.c Thu Feb 1 16:57:23 1996 *************** And asked what file to patch I copied and pasted to the prompt ./Modules/ofuncobject.c I don't know why patch needed the file names after being succesful the first few patches. Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Feb 2 18:14:53 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA21023 for matrix-sig-people; Fri, 2 Feb 1996 18:14:53 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id SAA21019 for ; Fri, 2 Feb 1996 18:14:48 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id SAA16552 (8.6.11/IDA-1.6 for ); Fri, 2 Feb 1996 18:11:59 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id SAA05278; Fri, 2 Feb 1996 18:11:59 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id SAA23116; Fri, 2 Feb 1996 18:11:57 -0500 Date: Fri, 2 Feb 1996 18:11:57 -0500 Message-Id: <199602022311.SAA23116@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] New pretty printer From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk My pretty printer has made some progress, and I'll include the latest version below. Just put it into your copy of Numeric.py. Line wrapping is still missing. (BTW, what would be a good default line width? In the VT100 days that would have been 80 characters, but does this still make sense?) I can't say that I am satisfied with float formatting, but anything better than now will be difficult to do and, most of all, slow. The two problems are 1) Trailing zeros in non-exponential format. Right now there are always eight digits after the decimal point. It would be better to replace trailing zeros by spaces (easy) and reduce the width of the output fields to match the longest remaining number. But that requires a scan of the string representations of all the elements in an array, which is an expensive operation. 2) In exponential notation, numbers with three-digit exponents will not line up correctly with others. It would be necessary to have three-digit exponents for all numbers when at least one element requires it, but there is no formatting option to specify the number of exponent digits. The only solution would be cosmetic surgery on the final string, but that again is slow. There are currently some problems with complex number output due to a remaining bug in the array module. Normally it will only lead to a wrong format selection, so it's not fatal. And now the code: def arrayToString(a, max_line_length = 80): if a.contiguous(): data = a.reshape(None) else: data = a.copy().reshape(None) type = a.typecode() items_per_line = a.shape[-1] if type == 'b' or type == '1' or type == 's' or type == 'i' \ or type == 'l': max_str_len = max(len(str(maximum.reduce(data))), len(str(minimum.reduce(data)))) format = '%' + str(max_str_len) + 'd' type = 'l' item_length = max_str_len+1 elif type == 'f' or type == 'd': format, item_length = _floatFormat(data) type = 'd' elif type == 'F' or type == 'D': real_format, real_item_length = _floatFormat(data.real, sign=0) imag_format, imag_item_length = _floatFormat(data.imaginary, sign=1) format = '(' + real_format + imag_format + 'j)' item_length = real_item_length + imag_item_length + 2 type = 'D' elif type == 'c': format = '%s' item_length = 1 else: # nothing for 'O' return str(a) line_length = item_length*items_per_line - (type != 'c') if line_length > max_line_length: pass return _arrayToString(a, type, format, len(a.shape), line_length)[:-1] def _floatFormat(data, sign = 0): exp_format = 0 non_zero = abs(compress(data.notEqual(0), data)) print str(data), str(data.notEqual(0)), str(compress(data.notEqual(0), data)) if len(non_zero) == 0: max_val = 0. min_val = 0. else: max_val = maximum.reduce(non_zero) min_val = minimum.reduce(non_zero) if max_val >= 1.e12 or min_val < 0.0001 or max_val/min_val > 1000.: exp_format = 1 if exp_format: max_str_len = 16 + (0 < min_val < 1e-99 or max_val >= 1e100) if sign: format = '%+' else: format = '%' format = format + str(max_str_len) + '.8e' item_length = max_str_len + 1 else: max_str_len = len(str(int(max_val))) + 10 if sign: format = '%+' else: format = '%' format = format + str(max_str_len) + '.8f' item_length = max_str_len + 1 return (format, item_length) def _arrayToString(a, type, format, rank, line_length): if rank == 0: return str(a[0]) elif rank == 1: s = '' if type == 'c': for i in range(a.shape[0]): s = s + (format % a[i]) s = s + '\n' elif type == 'D': for i in range(a.shape[0]): s = s + (format % (a[i].real, a[i].imag)) + ' ' s = s[:-1] + '\n' else: for i in range(a.shape[0]): s = s + (format % a[i]) + ' ' s = s[:-1] + '\n' else: s = '' for i in range(a.shape[0]-1): s = s + _arrayToString(a[i], type, format, rank-1, line_length) if rank == 3: s = s + '\n' elif rank > 3: s = s + (rank-3)*(line_length*'-'+'\n') s = s + _arrayToString(a[a.shape[0]-1], type, format, rank-1, line_length) return s ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Feb 2 19:10:30 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id TAA21324 for matrix-sig-people; Fri, 2 Feb 1996 19:10:30 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id TAA21320 for ; Fri, 2 Feb 1996 19:10:25 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa16725; 2 Feb 96 19:08 EST Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id TAA10516; Fri, 2 Feb 1996 19:06:43 -0500 Message-Id: <199602030006.TAA10516@monty> To: Konrad HINSEN cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] New pretty printer In-reply-to: Your message of "Fri, 02 Feb 1996 18:11:57 EST." <199602022311.SAA23116@cyclone.ERE.UMontreal.CA> References: <199602022311.SAA23116@cyclone.ERE.UMontreal.CA> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Fri, 02 Feb 1996 19:06:42 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk > My pretty printer has made some progress, and I'll include the latest > version below. Just put it into your copy of Numeric.py. Line wrapping > is still missing. (BTW, what would be a good default line width? In the > VT100 days that would have been 80 characters, but does this still > make sense?) Yes! I'd even say 77, so at least one level of email quoting won't cause it to wrap. I keep my Emacs religiously at 80 columns wide when programming -- this way I can fit two columns of windows plus some icons on my screen. Also, the default wrap columns for printing text seems to be 80 still. > I can't say that I am satisfied with float formatting, > but anything better than now will be difficult to do and, most of > all, slow. The two problems are > > 1) Trailing zeros in non-exponential format. Right now there are > always eight digits after the decimal point. It would be better > to replace trailing zeros by spaces (easy) and reduce the width > of the output fields to match the longest remaining number. > But that requires a scan of the string representations of all > the elements in an array, which is an expensive operation. > > 2) In exponential notation, numbers with three-digit exponents will > not line up correctly with others. It would be necessary to > have three-digit exponents for all numbers when at least > one element requires it, but there is no formatting option > to specify the number of exponent digits. The only solution > would be cosmetic surgery on the final string, but that again > is slow. (For both case:) Surely, if you're printing a reasonably sized array, it will be fast enough, and if it is truly huge, nobody will really look at the numbers... I'd say make it look as good as you can. Also, perhaps a regular expression can help you do the cosmetic surgery efficiently. --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Feb 2 19:10:58 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id TAA21324 for matrix-sig-people; Fri, 2 Feb 1996 19:10:30 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id TAA21320 for ; Fri, 2 Feb 1996 19:10:25 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa16725; 2 Feb 96 19:08 EST Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id TAA10516; Fri, 2 Feb 1996 19:06:43 -0500 Message-Id: <199602030006.TAA10516@monty> To: Konrad HINSEN cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] New pretty printer In-reply-to: Your message of "Fri, 02 Feb 1996 18:11:57 EST." <199602022311.SAA23116@cyclone.ERE.UMontreal.CA> References: <199602022311.SAA23116@cyclone.ERE.UMontreal.CA> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Fri, 02 Feb 1996 19:06:42 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk > My pretty printer has made some progress, and I'll include the latest > version below. Just put it into your copy of Numeric.py. Line wrapping > is still missing. (BTW, what would be a good default line width? In the > VT100 days that would have been 80 characters, but does this still > make sense?) Yes! I'd even say 77, so at least one level of email quoting won't cause it to wrap. I keep my Emacs religiously at 80 columns wide when programming -- this way I can fit two columns of windows plus some icons on my screen. Also, the default wrap columns for printing text seems to be 80 still. > I can't say that I am satisfied with float formatting, > but anything better than now will be difficult to do and, most of > all, slow. The two problems are > > 1) Trailing zeros in non-exponential format. Right now there are > always eight digits after the decimal point. It would be better > to replace trailing zeros by spaces (easy) and reduce the width > of the output fields to match the longest remaining number. > But that requires a scan of the string representations of all > the elements in an array, which is an expensive operation. > > 2) In exponential notation, numbers with three-digit exponents will > not line up correctly with others. It would be necessary to > have three-digit exponents for all numbers when at least > one element requires it, but there is no formatting option > to specify the number of exponent digits. The only solution > would be cosmetic surgery on the final string, but that again > is slow. (For both case:) Surely, if you're printing a reasonably sized array, it will be fast enough, and if it is truly huge, nobody will really look at the numbers... I'd say make it look as good as you can. Also, perhaps a regular expression can help you do the cosmetic surgery efficiently. --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Feb 4 17:51:17 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA00333 for matrix-sig-people; Sun, 4 Feb 1996 17:51:17 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id RAA00329 for ; Sun, 4 Feb 1996 17:51:14 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA25730; Sun, 4 Feb 96 17:43:26 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) for matrix-sig@python.org id RAA08528; Sun, 4 Feb 1996 17:54:46 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199602042254.RAA08528@maigret> Subject: [PYTHON MATRIX-SIG] slicing rank-1 arrays To: matrix-sig@python.org Date: Sun, 4 Feb 1996 17:54:46 -0500 (EST) X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 437 Sender: owner-matrix-sig@python.org Precedence: bulk Given the following, which seems fine: >>> a 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 >>> a[All,All] 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 why doesn't the following work? >>> b 0 1 2 3 4 >>> b[All] Traceback (innermost last): File "", line 1, in ? AttributeError: __len__ >>> b[Reverse] Traceback (innermost last): File "", line 1, in ? AttributeError: __len__ [Using 0.33 on an SGI] --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Feb 4 17:56:57 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA00365 for matrix-sig-people; Sun, 4 Feb 1996 17:56:57 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id RAA00361 for ; Sun, 4 Feb 1996 17:56:54 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA25748; Sun, 4 Feb 96 17:49:06 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) for matrix-sig@python.org id SAA08594; Sun, 4 Feb 1996 18:00:27 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199602042300.SAA08594@maigret> Subject: [PYTHON MATRIX-SIG] New Tutorial / Some Questions To: matrix-sig@python.org Date: Sun, 4 Feb 1996 18:00:26 -0500 (EST) X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 1984 Sender: owner-matrix-sig@python.org Precedence: bulk Given that I'm starting a course on Python on Tuesday for a bunch of neural net folks, I had to take a good hard look at the Numeric package. Jim H. has gotten more mail from me than he probably wanted. The good thing is that I didn't see myself teaching them about a feature that had no documentation. So, I started a more detailed tutorial than that Jim shipped in 0.33. I aimed it at people who have very little python experience, and no prior experience w/ matlab, basis, yorick or anything like it (i.e. undergraduates =). It's obviously too basic for the folks in this SIG, but I'd appreciate it if you could give it a once-over and make sure I'm not lying outright. It's available at http://maigret.cog.brown.edu/python/arraytut.html for now. It is very much under construction (I haven't explained rubber indices and pseudo indices, for example, mostly because I'm not sure I understand them yet). I came up with a few questions in the process. Some have gone to jim h. and hinsen, but I thought I'd spread the load: 1. I can't seem to find the counterpart to the toFile() method. I've tried just the array constructor w/ a file object, which I thought might work, but I get: >>> a 0 0 1 0 0 0 1 2 3 >>> f = open('myarray', 'w') >>> a.toFile(f) >>> f.close() >>> f = open('myarray', 'r') >>> b = array(f) >>> b Traceback (innermost last): File "", line 1, in ? File "/usr/local/lib/python/numeric/Numeric.py", line 37, in arrayToString items_per_line = a.shape[-1] IndexError: tuple index out of range 2. The choose() and take() methods are a little fuzzy for me. Can someone give me a description of what they do, and more useful even examples of why they're useful? 3. Shouldn't typecodes look more like real types than strings, sort of like exceptions? >>> b.typecode() 'l' vs. >>> b.typecode() LongInteger or whatever. That's all for now. Let me know what you think of the tutorial. --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Feb 5 11:13:50 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA00156 for matrix-sig-people; Mon, 5 Feb 1996 11:13:50 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id LAA00152 for ; Mon, 5 Feb 1996 11:13:45 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA26208; Mon, 5 Feb 96 11:13:43 EST Date: Mon, 5 Feb 96 11:13:43 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9602051613.AA26208@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: da@maigret.cog.brown.edu Cc: matrix-sig@python.org In-Reply-To: <199602042254.RAA08528@maigret> (da@maigret.cog.brown.edu) Subject: Re: [PYTHON MATRIX-SIG] slicing rank-1 arrays Sender: owner-matrix-sig@python.org Precedence: bulk From: da@maigret.cog.brown.edu (David Ascher) Given the following, which seems fine: >>> a 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 >>> a[All,All] 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 why doesn't the following work? >>> b 0 1 2 3 4 >>> b[All] Traceback (innermost last): File "", line 1, in ? AttributeError: __len__ >>> b[Reverse] Traceback (innermost last): File "", line 1, in ? AttributeError: __len__ [Using 0.33 on an SGI] This is a bug. I'll fix it. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Feb 5 11:26:39 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA00290 for matrix-sig-people; Mon, 5 Feb 1996 11:26:39 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id LAA00286 for ; Mon, 5 Feb 1996 11:26:36 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA26139; Mon, 5 Feb 96 10:56:03 EST Date: Mon, 5 Feb 96 10:56:03 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9602051556.AA26139@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: da@maigret.cog.brown.edu Cc: matrix-sig@python.org In-Reply-To: <199602042300.SAA08594@maigret> (da@maigret.cog.brown.edu) Subject: Re: [PYTHON MATRIX-SIG] New Tutorial / Some Questions Sender: owner-matrix-sig@python.org Precedence: bulk From: da@maigret.cog.brown.edu (David Ascher) Given that I'm starting a course on Python on Tuesday for a bunch of neural net folks, I had to take a good hard look at the Numeric package. Jim H. has gotten more mail from me than he probably wanted. The good Actually, I love the mail. This is the best way to really shake down the package. I came up with a few questions in the process. Some have gone to jim h. and hinsen, but I thought I'd spread the load: Well, since nobody else has answered yet (and since I'd LOVE to see your course succeed). 1. I can't seem to find the counterpart to the toFile() method. There used to be such a function in Numeric.py. The reason it's not there any more is to strongly encourage people to use pickling of matrices and better yet of objects containing matrices instead of the raw toFile and fromFile methods. This is the method I use to store the fairly complicated GaussianClassifier objects that I use for speech recognition. This approach will gain you a certain degree of architecture independence (though 64-bit longs on alpha's give it trouble) and will cost almost nothing in performance terms. Note, you can still read in an array from a file if you need to for compatibility reasons as follows: a = fromString(fp.read()) 2. The choose() and take() methods are a little fuzzy for me. Can someone give me a description of what they do, and more useful even examples of why they're useful? Actually, Konrad or Paul could probably do a better job of this than me, but here are my two favorite examples. static char doc_choose[] = "m.choose(m_0, ..., m_(n-1)). m is a array of integers from 0 to n-1. Each entry in m will be taken from the corresponding entry in m_i, where i is the value of m. Frequently used with exactly two arguments for a boolean valued m, ie. m.choose(m_false, m_true)."; Assume a is an array that you want to "clip" so that no values are greater than 100.0. (a.greater(100.0)).choose(a, 100.0) Everywhere that a.greater(100.0) is false (ie. 0) this will "choose" the corresponding value in a. Everywhere else it will "choose" 100.0. static char doc_take[] = "m.take([i_0, ..., i_n], dimension=0). Selects the items from m along dimension"; Take is the equivalent of "full product form indexing" that for a number of reasons is not available in normal indexing. The most common use I have for take is to implement decoding from mu-law to 16-bit linear sound files. mu-law files map the bytes from 0-255 to a given 16-bit number using a non-linear scale as a form of compression. Assuming you have an array mu_law_array (of length 256) where mu_law_array[value_mu_law] = value_linear linear_encoded_array = mu_law_array.take(mu_law_encoded_array) 3. Shouldn't typecodes look more like real types than strings, sort of like exceptions? >>> b.typecode() 'l' vs. >>> b.typecode() LongInteger or whatever. The only hassle is that doing this would lead to yet another new object type added to python (after already adding array's, ufunc's, and slices (hopefully soon). I'm willing to be convinced that this is the right choice, but for now I opted for the minimal approach. That's all for now. Let me know what you think of the tutorial. Will look at it ASAP. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Feb 5 12:06:02 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA00557 for matrix-sig-people; Mon, 5 Feb 1996 12:06:02 -0500 Received: from typhoon.osg.saic.com (typhoon.osg.saic.com [149.8.94.25]) by python.org (8.6.12/8.6.12) with SMTP id MAA00552 for ; Mon, 5 Feb 1996 12:06:00 -0500 Received: by typhoon.osg.saic.com (4.1/SMI-4.1) id AA02953; Mon, 5 Feb 96 09:34:22 EST From: nick@osg.saic.com (Nick Seidenman) Message-Id: <9602051434.AA02953@typhoon.osg.saic.com> Subject: [PYTHON MATRIX-SIG] NumPython To: matrix-sig@python.org Date: Mon, 5 Feb 1996 09:34:21 -0500 (EST) X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 408 Sender: owner-matrix-sig@python.org Precedence: bulk Just downloaded and built 0.33. No problems so far! Compiled right out of the box on a linux 1.2.13 system rnning on my handy-dandy pentium laptop! Thanks, Jim! nick --------------------------- Nick Seidenman Science Applications International Corporation 1710 Goodridge Drive McLean, VA 22102 Phone: (703) 448-6497 (fax): (703) 821-3576 email: nick@osg.saic.com http://www.osg.saic.com/~nick/ ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Feb 5 15:11:38 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA02003 for matrix-sig-people; Mon, 5 Feb 1996 15:11:38 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id PAA01999 for ; Mon, 5 Feb 1996 15:11:35 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id LAA07625 (8.6.11/IDA-1.6); Mon, 5 Feb 1996 11:40:41 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id LAA09848; Mon, 5 Feb 1996 11:40:39 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id LAA05335; Mon, 5 Feb 1996 11:40:38 -0500 Date: Mon, 5 Feb 1996 11:40:38 -0500 Message-Id: <199602051640.LAA05335@cyclone.ERE.UMontreal.CA> To: da@maigret.cog.brown.edu CC: matrix-sig@python.org In-reply-to: <199602042300.SAA08594@maigret> (da@maigret.cog.brown.edu) Subject: Re: [PYTHON MATRIX-SIG] New Tutorial / Some Questions From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk 3. Shouldn't typecodes look more like real types than strings, sort of like exceptions? >>> b.typecode() 'l' vs. >>> b.typecode() LongInteger or whatever. Definitely. Especially now that we have the stuff in Precision.py for specifying types, it should be used consistently wherever possible. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Feb 5 16:25:47 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA02567 for matrix-sig-people; Mon, 5 Feb 1996 16:25:47 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id QAA02562 for ; Mon, 5 Feb 1996 16:25:40 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id QAA19541 (8.6.11/IDA-1.6); Mon, 5 Feb 1996 16:23:59 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA18864; Mon, 5 Feb 1996 16:23:58 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA24690; Mon, 5 Feb 1996 16:23:57 -0500 Date: Mon, 5 Feb 1996 16:23:57 -0500 Message-Id: <199602052123.QAA24690@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: da@maigret.cog.brown.edu, matrix-sig@python.org In-reply-to: <9602051556.AA26139@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] New Tutorial / Some Questions From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk 1. I can't seem to find the counterpart to the toFile() method. There used to be such a function in Numeric.py. The reason it's not there any more is to strongly encourage people to use pickling of matrices and better yet of objects containing matrices instead of the raw toFile and fromFile methods. This is the method I use to store That makes sense, but then toFile() should also disappear. Take is the equivalent of "full product form indexing" that for a number of reasons is not available in normal indexing. The most common use I have for take is to implement decoding from mu-law to 16-bit linear sound files. mu-law files map the bytes from 0-255 to a given 16-bit number using a non-linear scale as a form of compression. Assuming you have an array mu_law_array (of length 256) where mu_law_array[value_mu_law] = value_linear linear_encoded_array = mu_law_array.take(mu_law_encoded_array) Translation tables are one of the important applications of this operation. Another is sorting, ...ehhh... *would* be sorting if it were implemented in a more flexible way. Right now we have the built-in function sort() which works also on arrays. What is missing (and should be added) is a function that returns the indices of the elements in sorted order. As an example, suppose you have an array of measurements of some function f(x) as a rank-2 array a; a[0] being the evaluation points and a[1] the values. You want to sort the array by evaluation points. The answer is a.take(a[0].sortIndex(), 1) BTW, it would be extremely helpful to have a list of all existing functions/methods with a short definition. It is difficult to comment on whether the current set is sufficient and/or appropriately defined and names without knowing what is out there! Do you happen to have something suitable? 3. Shouldn't typecodes look more like real types than strings, sort of like exceptions? >>> b.typecode() 'l' vs. >>> b.typecode() LongInteger or whatever. The only hassle is that doing this would lead to yet another new object type added to python (after already adding array's, ufunc's, and slices (hopefully soon). I'm willing to be convinced that this is the right choice, but for now I opted for the minimal approach. There is no need for a new object type, since the typecode could be of type "type". All that has to be added is type objects for the new types. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Feb 5 16:45:03 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA02660 for matrix-sig-people; Mon, 5 Feb 1996 16:45:03 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id QAA02655 for ; Mon, 5 Feb 1996 16:45:00 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA27092; Mon, 5 Feb 96 16:38:14 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) id QAA11829; Mon, 5 Feb 1996 16:49:33 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199602052149.QAA11829@maigret> Subject: Re: [PYTHON MATRIX-SIG] New Tutorial / Some Questions To: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Mon, 5 Feb 1996 16:49:33 -0500 (EST) Cc: jjh@goldilocks.lcs.mit.edu, matrix-sig@python.org In-Reply-To: <199602052123.QAA24690@cyclone.ERE.UMontreal.CA> from "Konrad HINSEN" at Feb 5, 96 04:23:57 pm X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 573 Sender: owner-matrix-sig@python.org Precedence: bulk > BTW, it would be extremely helpful to have a list of all existing > functions/methods with a short definition. It is difficult to comment > on whether the current set is sufficient and/or appropriately defined > and names without knowing what is out there! Do you happen to have > something suitable? I've tried to list them all in the tutorial. Well, at least they should all be mentioned in the tutorial. A whole bunch of them have no definitions yet, but that will get fixed. I just got them from browsing the source, so I may very well have missed some. --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Feb 5 17:16:32 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA02924 for matrix-sig-people; Mon, 5 Feb 1996 17:16:32 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id RAA02920 for ; Mon, 5 Feb 1996 17:16:22 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA19150; Mon, 5 Feb 96 14:16:12 PST Received: from kristen (localhost) by kristen.llnl.gov (5.x/SMI-SVR4) id AA05541; Mon, 5 Feb 1996 14:16:04 -0800 Message-Id: <311681A4.1E79@llnl.gov> Date: Mon, 05 Feb 1996 14:16:04 -0800 From: "Paul. Dubois" Organization: Lawrence Livermore National Laboratory X-Mailer: Mozilla 2.0b5 (X11; I; SunOS 5.4 sun4m) Mime-Version: 1.0 To: hinsenk@ere.umontreal.ca Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] New pretty printer References: <199602022311.SAA23116@cyclone.ERE.UMontreal.CA> <199602030006.TAA10516@monty> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk I suggest a default size of 79 (or 77 as suggested by Guido). Exactly 80 causes wrap in some display vehicles. I didn't look yet at how you did it but perhaps a user-settable precision could be managed. I have done that in both Basis and EiffelMath. Guido is right about the timing too; if there are enough components that you have to worry about the timing to print it, nobody is going to print it anyway or look at it if they do. One thing I did in Basis was an optional "compression" that compressed identical rows (or strings of identical elements in 1-D). Basis> ones(10000) ones(10000) shape: (10000) 1: 9976 - 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 9977: - 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Basis> shape([1,2,3]//ones(10000)//[5,6],2001,5) shape([1,2,3]//ones(10000)//[5,6],2001,5) shape: (2001,5) row col = 1 2 3 4 5 1: - 1 1 1 1 1 2: - 2 1 1 1 1 3: - 3 1 1 1 1 4:1999 - 1 1 1 1 1 2000: - 1 1 1 1 5 2001: - 1 1 1 1 6 Basis> This has proven useful. -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Feb 5 19:13:07 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id TAA03910 for matrix-sig-people; Mon, 5 Feb 1996 19:13:07 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id TAA03905 for ; Mon, 5 Feb 1996 19:13:02 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id TAA25328 (8.6.11/IDA-1.6); Mon, 5 Feb 1996 19:11:52 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id TAA27363; Mon, 5 Feb 1996 19:11:51 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id TAA05144; Mon, 5 Feb 1996 19:11:50 -0500 Date: Mon, 5 Feb 1996 19:11:50 -0500 Message-Id: <199602060011.TAA05144@cyclone.ERE.UMontreal.CA> To: dubois1@llnl.gov CC: matrix-sig@python.org In-reply-to: <311681A4.1E79@llnl.gov> (dubois1@llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] New pretty printer From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk I didn't look yet at how you did it but perhaps a user-settable precision could be managed. I have done that in both Basis and Just implemented :-) The problem with both this and the line width is that there is no convenient way to set them when simply printing arrays. Right now, you have to call the function arrayToString() explicitly with additional parameters. I could of course define global variables in module Numeric to handle this, but I am not sure whether that is a good idea. Maybe a good compromise is optional parameters that are set from global variables if undefined. Comments? EiffelMath. Guido is right about the timing too; if there are enough components that you have to worry about the timing to print it, nobody is going to print it anyway or look at it if they do. I tend to agree, especially since you can always interrupt the output, as long as the function is written in Python. One thing I did in Basis was an optional "compression" that compressed identical rows (or strings of identical elements in 1-D). That could indeed be useful, and not too complicated to implement. Another option... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 6 09:22:07 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA07856 for matrix-sig-people; Tue, 6 Feb 1996 09:22:07 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id JAA07852 for ; Tue, 6 Feb 1996 09:21:34 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA20009; Tue, 6 Feb 96 09:21:02 EST Date: Tue, 6 Feb 96 09:21:02 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9602061421.AA20009@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: hinsenk@ere.umontreal.ca Cc: matrix-sig@python.org In-Reply-To: <199602060011.TAA05144@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Subject: Re: [PYTHON MATRIX-SIG] New pretty printer Sender: owner-matrix-sig@python.org Precedence: bulk From: hinsenk@ere.umontreal.ca (Konrad HINSEN) I didn't look yet at how you did it but perhaps a user-settable precision could be managed. I have done that in both Basis and Just implemented :-) The problem with both this and the line width is that there is no convenient way to set them when simply printing arrays. Right now, you have to call the function arrayToString() explicitly with additional parameters. I could of course define global variables in module Numeric to handle this, but I am not sure whether that is a good idea. Maybe a good compromise is optional parameters that are set from global variables if undefined. Comments? The compromise approach sounds perfect. I think that this is one of those rare cases where there is no potential for problems to be caused by global variables because you are not going to be altering the results of any computation, only the way those results are displayed. Without global variables there is just no reasonable way to alter the default behavior for "print a". -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 6 09:34:50 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA07920 for matrix-sig-people; Tue, 6 Feb 1996 09:34:50 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id JAA07916 for ; Tue, 6 Feb 1996 09:34:47 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id OAA10573; Tue, 6 Feb 1996 14:32:19 GMT From: "Jim Fulton, U.S. Geological Survey" Message-Id: <9602060932.ZM10571@dsdbqvarsa.er.usgs.gov> Date: Tue, 6 Feb 1996 09:32:18 -0500 In-Reply-To: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) "Re: [PYTHON MATRIX-SIG] New pretty printer" (Feb 6, 9:21am) References: <9602061421.AA20009@baribal.LCS.MIT.EDU.LCS.MIT.EDU> X-Phone: (703) 648-5622 X-Face: (;vf"lccr&V=y Subject: Re: [PYTHON MATRIX-SIG] New pretty printer > > From: hinsenk@ere.umontreal.ca (Konrad HINSEN) > > I didn't look yet at how you did it but perhaps a user-settable > precision could be managed. I have done that in both Basis and > > Just implemented :-) The problem with both this and the line width is > that there is no convenient way to set them when simply printing > arrays. Right now, you have to call the function arrayToString() > explicitly with additional parameters. I could of course define global > variables in module Numeric to handle this, but I am not sure whether > that is a good idea. Maybe a good compromise is optional parameters > that are set from global variables if undefined. Comments? > > The compromise approach sounds perfect. I think that this is one of > those rare cases where there is no potential for problems to be caused > by global variables because you are not going to be altering the > results of any computation, only the way those results are displayed. But other modules may have different display needs, so you can still effect a different piece of code. > Without global variables there is just no reasonable way to alter the > default behavior for "print a". You could make this parameter a part of the state of an array. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 10:12:12 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA16550 for matrix-sig-people; Wed, 7 Feb 1996 10:12:12 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id KAA16546 for ; Wed, 7 Feb 1996 10:12:07 -0500 Received: from chims1.CHIMCN.UMontreal.CA (chims1.CHIMCN.UMontreal.CA [132.204.11.106]) by harfang.CC.UMontreal.CA with ESMTP id KAA17236 (8.6.11/IDA-1.6); Wed, 7 Feb 1996 10:10:30 -0500 Received: by chims1.CHIMCN.UMontreal.CA (950215.SGI.8.6.10/5.17) id KAA03680; Wed, 7 Feb 1996 10:11:56 -0500 Date: Wed, 7 Feb 1996 10:11:56 -0500 From: hinsenk@CHIMCN.UMontreal.CA (Hinsen Konrad) Message-Id: <199602071511.KAA03680@chims1.CHIMCN.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] FTP address Sender: owner-matrix-sig@python.org Precedence: bulk Due to a major server breakdown I am cut off from all my files and my normal e-mail address. Nevertheless I'd like to go on playing with the array package. I have already reinstalled Python, but I have to get the numerics extension again. Could anyone please send me the address of Jim's FTP server? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@chims1.chimcn.umontreal.ca Departement de Chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 11:53:56 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA17415 for matrix-sig-people; Wed, 7 Feb 1996 11:53:56 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id LAA17410 for ; Wed, 7 Feb 1996 11:53:50 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id LAA21843 (8.6.11/IDA-1.6); Wed, 7 Feb 1996 11:52:17 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id LAA08117; Wed, 7 Feb 1996 11:52:34 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id LAA04701; Wed, 7 Feb 1996 11:52:33 -0500 Date: Wed, 7 Feb 1996 11:52:33 -0500 Message-Id: <199602071652.LAA04701@cyclone.ERE.UMontreal.CA> To: jfulton@usgs.gov CC: jjh@Goldilocks.LCS.MIT.EDU, matrix-sig@python.org In-reply-to: <9602060932.ZM10571@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] New pretty printer From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk But other modules may have different display needs, so you can still effect a different piece of code. > Without global variables there is just no reasonable way to alter the > default behavior for "print a". You could make this parameter a part of the state of an array. That implies that precision and line width are properties of an array. For precision this may be justified, but certainly not for line width! Such things really belong into a "system parameters" collection. The closest Python equivalent would be the module "sys". But that doesn't solve the problem of who is allowed to modify them how and when. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 12:07:42 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA17539 for matrix-sig-people; Wed, 7 Feb 1996 12:07:42 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id MAA17535 for ; Wed, 7 Feb 1996 12:07:38 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa09320; 7 Feb 96 12:02 EST Received: from localhost by monty.cnri.reston.va.us via SMTP (940816.SGI.8.6.9/940406.SGI) id MAA20712; Wed, 7 Feb 1996 12:01:39 -0500 Message-Id: <199602071701.MAA20712@monty.cnri.reston.va.us> To: Konrad HINSEN cc: jfulton@usgs.gov, jjh@goldilocks.lcs.mit.edu, matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] New pretty printer In-reply-to: Your message of "Wed, 07 Feb 1996 11:52:33 EST." <199602071652.LAA04701@cyclone.ERE.UMontreal.CA> References: <199602071652.LAA04701@cyclone.ERE.UMontreal.CA> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Wed, 07 Feb 1996 12:01:39 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk > Such things really belong into a "system parameters" collection. > The closest Python equivalent would be the module "sys". But > that doesn't solve the problem of who is allowed to modify them > how and when. It makes sense for it to be a property of the file object. But since adding properties to those is no fun, I'd suggest to go for a parameter is sys. Whoever feels like they can change sys.stdout can also change the line width if they feel like it. --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 12:08:49 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA17539 for matrix-sig-people; Wed, 7 Feb 1996 12:07:42 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id MAA17535 for ; Wed, 7 Feb 1996 12:07:38 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa09320; 7 Feb 96 12:02 EST Received: from localhost by monty.cnri.reston.va.us via SMTP (940816.SGI.8.6.9/940406.SGI) id MAA20712; Wed, 7 Feb 1996 12:01:39 -0500 Message-Id: <199602071701.MAA20712@monty.cnri.reston.va.us> To: Konrad HINSEN cc: jfulton@usgs.gov, jjh@goldilocks.lcs.mit.edu, matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] New pretty printer In-reply-to: Your message of "Wed, 07 Feb 1996 11:52:33 EST." <199602071652.LAA04701@cyclone.ERE.UMontreal.CA> References: <199602071652.LAA04701@cyclone.ERE.UMontreal.CA> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Wed, 07 Feb 1996 12:01:39 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk > Such things really belong into a "system parameters" collection. > The closest Python equivalent would be the module "sys". But > that doesn't solve the problem of who is allowed to modify them > how and when. It makes sense for it to be a property of the file object. But since adding properties to those is no fun, I'd suggest to go for a parameter is sys. Whoever feels like they can change sys.stdout can also change the line width if they feel like it. --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 14:28:13 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA18684 for matrix-sig-people; Wed, 7 Feb 1996 14:28:13 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id OAA18680 for ; Wed, 7 Feb 1996 14:28:06 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id MAA22656 (8.6.11/IDA-1.6); Wed, 7 Feb 1996 12:11:06 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA13731; Wed, 7 Feb 1996 12:11:23 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA07277; Wed, 7 Feb 1996 12:11:21 -0500 Date: Wed, 7 Feb 1996 12:11:21 -0500 Message-Id: <199602071711.MAA07277@cyclone.ERE.UMontreal.CA> To: guido@CNRI.Reston.VA.US CC: jfulton@usgs.gov, jjh@goldilocks.lcs.mit.edu, matrix-sig@python.org In-reply-to: <199602071701.MAA20712@monty.cnri.reston.va.us> (message from Guido van Rossum on Wed, 07 Feb 1996 12:01:39 -0500) Subject: Re: [PYTHON MATRIX-SIG] New pretty printer From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk It makes sense for it to be a property of the file object. But since adding properties to those is no fun, I'd suggest to go for a parameter is sys. Whoever feels like they can change sys.stdout can also change the line width if they feel like it. OK. Then it makes sense to add another parameter for printing precision and use it also for scalars. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 14:48:51 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA18784 for matrix-sig-people; Wed, 7 Feb 1996 14:48:51 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id OAA18780 for ; Wed, 7 Feb 1996 14:48:49 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA17725; Wed, 7 Feb 96 14:48:13 EST Date: Wed, 7 Feb 96 14:48:13 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9602071948.AA17725@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Steadily approaching BETA release Sender: owner-matrix-sig@python.org Precedence: bulk Chris Chase just sent me a bunch of patches to allow a[::-1, 1:-1] style indexing. So the grammar is solidifying. David Ascher's tutorial (and my own TUTORIAL.NumPy) are starting to produce something like an acceptable first pass at documentation. Konrad Hinsen's print function offers acceptable printing of arrays. I've received very few bug reports this week, so the base code is starting to look stable. So, what's left before a general BETA release? Note: I'm going to comment on each of these in a seperate follow-up piece of e-mail in order to try and seperate these two different issues. 1) Type coercion 2) Naming conventions What I've decided is not important to resolve before the BETA release: 1) static vs. dynamic linking (static linking is fine for now) 2) Contents of Numeric.py (I'm going to rule by fiat here) 3) The exact details of Konrad's print function (Unlikley that changes here will break anyone's code) ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 15:18:36 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA19113 for matrix-sig-people; Wed, 7 Feb 1996 15:18:36 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id PAA19109 for ; Wed, 7 Feb 1996 15:18:28 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA17817; Wed, 7 Feb 96 15:18:00 EST Date: Wed, 7 Feb 96 15:18:00 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9602072018.AA17817@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Type Coercion Sender: owner-matrix-sig@python.org Precedence: bulk I still don't see a great solution to this one. Here's my attempt to boil the issue down to its essence: (While I only talk about floats and doubles here, this same issue comes up with longs and shorts and bytes as well as float and double complex numbers). C (and FORTRAN) has two kinds of floating point numbers, float and double. A double has more precision and range than a float. In C if I multiply a double by a float I get a double. Python has only one floating point type, call this a PyFloat. Internally, these numbers store their data using C doubles. However, the standard C interface to PyFloats allows them to be extracted into either a C float or a C double (PyArg_ParseTuple), so its not clear that a PyFloat should really be considered either a float or a double. Nevertheless, its obvious that PyFloats are more closely related to doubles than floats. An array can be of type float or double. It is reasonable to have the result of multiplying an array of floats by an array of doubles be an array of doubles. What should happen if I multiply an array of floats by a PyFloat? Three things can happen: 1) Return an array of floats (possible undesired loss of precision) 2) Return an array of doubles (possible undesired gain of precision) 3) Raise an exception (very annoying to the user) Currently there is also the convention that when an array of floats or doubles returns a single element, that element will be a PyFloat. Perhaps this behavior should be altered as the following examples of confusing behavior using either approach 1 or approach 2 show. a_float and a_double are 1d arrays of the appropriate type. Using Method 1) a_float[0]*a_float -> a_float (good) a_double[0]*a_float -> a_float (not good) Using Method 2) a_float[0]*a_float -> a_double (not good) a_double[0]*a_float -> a_double (good) This particular problem could be eliminated by having a_float[0] return a 0d array of floats with a single element. Perhaps this is worth considering. The next confusing behavior involves python numeric constants. Again, considering the two non-exception producing methods discussed above: very_precise_constant = 1.2345678910 Using method 1) 0.5*a_float -> a_float (good) very_precise_constant*a_float -> a_float (probably not good) Using method 2) 0.5*a_float -> a_double (not good) very_precise_constant*a_float -> a_double (probably good) Using this method, the only way to write 0.5*a_float and get an array of floats back is to write: array(0.5, Float(32))*a_float This becomes very ugly for any reasonably sized expression. Currently, method 3 is used because it seems to make everybody unhappy, yet it doesn't do anything terribly "wrong". Personally I would prefer to use method 1, combined with the major change to the existing system that scalars are returned as 0d arrays, not as python scalars. This means that python scalars will be considered to have a malleable type (as they in fact seem to) and can be cast to either a float or a double as needed in the particular equation. If you have a high-precision number whose precision you need to hold on to, you will need to say: very_precise_constant = array(1.2345678910, Float(64)) As I said at the beginning, none of these three options really appeals to me, yet I feel that this is a very important issue that really needs to be resolved before the BETA release. Opinions? (like I have to ask on this issue) - Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 15:52:50 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA19318 for matrix-sig-people; Wed, 7 Feb 1996 15:52:50 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id PAA19314 for ; Wed, 7 Feb 1996 15:52:47 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id PAA02229 (8.6.11/IDA-1.6); Wed, 7 Feb 1996 15:51:12 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA25321; Wed, 7 Feb 1996 15:51:29 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA21905; Wed, 7 Feb 1996 15:51:28 -0500 Date: Wed, 7 Feb 1996 15:51:28 -0500 Message-Id: <199602072051.PAA21905@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: matrix-sig@python.org In-reply-to: <9602071948.AA17725@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] Steadily approaching BETA release From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk So, what's left before a general BETA release? Note: I'm going to comment on each of these in a seperate follow-up piece of e-mail in order to try and seperate these two different issues. 1) Type coercion 2) Naming conventions 3) Practical experience I don't know how much real-life code others have produced with the current release, but I find that even with the relatively modest amount of application code I have written, I keep finding weak spots in the current collection of functions, i.e. operations that are impossible or difficult to code efficiently. It may seem that this is not a major problem, since more functions can always be added later. But that will quickly lead to the same type of function chaos that other "historically evolved" systems suffer from. Sometimes new functionality is best implemented by extending the definition of an existing function, which should however be reflected by a new name - and at that point compatibility considerations make the best solution impossible once the core function set is frozen. To give an example: I need to repeat certain items in an array a specified number of times, e.g. given n = array([1,1,2,3]) x = array([2.,1.,6.,3.5]) I want to obtain something like y = repeat(n,x) yielding array([2.,1.,6.,6.,3.5,3.5,3.5]) I don't see any reasonably efficient way to implement the function repeat() with the current array operations. Supposing that this operation is generally useful (and my APL/J experience definitely indicates that it is), it should be implemented in C in the array module. But a little reflection shows that this is a simple generalization of the already existing function compress(); indeed compress() is identical to repeat() as long as the first argument contains only 0s and 1s. So the most rational design would be to have repeat() and eliminate compress(). This leaves the question of how to find out which operations are necessary and which aren't. Apart from practical experience (which takes a lot of time to accumulate, and much more users than we currently have), it makes sense to look what other array languages have. It has been my intention to check whether common APL/J function can be implemented easily with what we have, but without knowing the existing function set, this is very difficult! What I've decided is not important to resolve before the BETA release: 1) static vs. dynamic linking (static linking is fine for now) 2) Contents of Numeric.py (I'm going to rule by fiat here) 3) The exact details of Konrad's print function (Unlikley that changes here will break anyone's code) I agree. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 15:53:38 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA19334 for matrix-sig-people; Wed, 7 Feb 1996 15:53:38 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id PAA19326 for ; Wed, 7 Feb 1996 15:53:34 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id PAA02244 (8.6.11/IDA-1.6); Wed, 7 Feb 1996 15:51:53 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA25549; Wed, 7 Feb 1996 15:52:11 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA22142; Wed, 7 Feb 1996 15:52:09 -0500 Date: Wed, 7 Feb 1996 15:52:09 -0500 Message-Id: <199602072052.PAA22142@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: matrix-sig@python.org In-reply-to: <9602071948.AA17725@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] Steadily approaching BETA release From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk So, what's left before a general BETA release? Note: I'm going to comment on each of these in a seperate follow-up piece of e-mail in order to try and seperate these two different issues. 1) Type coercion 2) Naming conventions 3) Practical experience I don't know how much real-life code others have produced with the current release, but I find that even with the relatively modest amount of application code I have written, I keep finding weak spots in the current collection of functions, i.e. operations that are impossible or difficult to code efficiently. It may seem that this is not a major problem, since more functions can always be added later. But that will quickly lead to the same type of function chaos that other "historically evolved" systems suffer from. Sometimes new functionality is best implemented by extending the definition of an existing function, which should however be reflected by a new name - and at that point compatibility considerations make the best solution impossible once the core function set is frozen. To give an example: I need to repeat certain items in an array a specified number of times, e.g. given n = array([1,1,2,3]) x = array([2.,1.,6.,3.5]) I want to obtain something like y = repeat(n,x) yielding array([2.,1.,6.,6.,3.5,3.5,3.5]) I don't see any reasonably efficient way to implement the function repeat() with the current array operations. Supposing that this operation is generally useful (and my APL/J experience definitely indicates that it is), it should be implemented in C in the array module. But a little reflection shows that this is a simple generalization of the already existing function compress(); indeed compress() is identical to repeat() as long as the first argument contains only 0s and 1s. So the most rational design would be to have repeat() and eliminate compress(). This leaves the question of how to find out which operations are necessary and which aren't. Apart from practical experience (which takes a lot of time to accumulate, and much more users than we currently have), it makes sense to look what other array languages have. It has been my intention to check whether common APL/J function can be implemented easily with what we have, but without knowing the existing function set, this is very difficult! What I've decided is not important to resolve before the BETA release: 1) static vs. dynamic linking (static linking is fine for now) 2) Contents of Numeric.py (I'm going to rule by fiat here) 3) The exact details of Konrad's print function (Unlikley that changes here will break anyone's code) I agree. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 16:37:56 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA19893 for matrix-sig-people; Wed, 7 Feb 1996 16:37:56 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id QAA19889 for ; Wed, 7 Feb 1996 16:37:53 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA18068; Wed, 7 Feb 96 16:37:24 EST Date: Wed, 7 Feb 96 16:37:24 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9602072137.AA18068@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience Sender: owner-matrix-sig@python.org Precedence: bulk Personally, I'm not terribly unhappy with the current naming conventions and feature set. I agree that they could be better, but I lack the time to sit down and do a careful redesign. The current feature set is significantly more powerful than the basic array operations provided by matlab, the most popular array processing language that people buy. Rather than complaining about the lack of a list of the available functions, David Ascher recently sat down and pulled them out of the C and python source code (not too hard to do, since I did include doc strings for everything). Check out his beta tutorial (http://maigret.cog.brown.edu/python/arraytut.html) for the current set of names in use. It's still an early draft with the expected bugs, but its getting close to a fairly complete intro. I'm going to step out of my current role of moderator for this issue. If somebody else out there has the time and energy to design a complete set of functions and methods for the system, go for it. If you write this up nicely and can convince the people in this SIG that your feature set makes sense, I'll be happy to implement it (well, so long as you don't ask me to solve the halting problem). Otherwise, I'm tired of people asking for me to do things like make a better distinction between functions and methods. This is just not useful unless I had more free time than I do. Sorry for the negative tone, but I've been getting a few too many questions like "So how's your speech recognition system coming along?" from my advisor lately and I'm eager to get the array object "out the door". Bug fixes take up very little of my intellectual energy, but these issues of naming conventions and function sets use up way too much. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 16:43:29 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA19989 for matrix-sig-people; Wed, 7 Feb 1996 16:43:29 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id QAA19985 for ; Wed, 7 Feb 1996 16:43:26 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id VAA12757; Wed, 7 Feb 1996 21:39:09 GMT From: "Jim Fulton" Message-Id: <9602071639.ZM12755@dsdbqvarsa.er.usgs.gov> Date: Wed, 7 Feb 1996 16:39:08 -0500 In-Reply-To: hinsenk@ere.umontreal.ca (Konrad HINSEN) "Re: [PYTHON MATRIX-SIG] New pretty printer" (Feb 7, 12:11pm) References: <199602071711.MAA07277@cyclone.ERE.UMontreal.CA> X-Mailer: Z-Mail (3.2.0 06sep94) To: hinsenk@ere.umontreal.ca (Konrad HINSEN), guido@CNRI.Reston.VA.US Subject: Re: [PYTHON MATRIX-SIG] New pretty printer Cc: jjh@goldilocks.lcs.mit.edu, matrix-sig@python.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit Sender: owner-matrix-sig@python.org Precedence: bulk On Feb 7, 12:11pm, Konrad HINSEN wrote: > Subject: Re: [PYTHON MATRIX-SIG] New pretty printer > > It makes sense for it to be a property of the file object. But since > adding properties to those is no fun, I'd suggest to go for a > parameter is sys. Whoever feels like they can change sys.stdout can > also change the line width if they feel like it. > > OK. Then it makes sense to add another parameter for printing > precision and use it also for scalars. I'd rather see precision handled as an array attribute. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 16:44:45 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA20013 for matrix-sig-people; Wed, 7 Feb 1996 16:44:45 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id QAA20009 for ; Wed, 7 Feb 1996 16:44:42 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA01665; Wed, 7 Feb 96 13:44:11 PST Received: from kristen (localhost) by kristen.llnl.gov (5.x/SMI-SVR4) id AA17523; Wed, 7 Feb 1996 13:44:10 -0800 Message-Id: <31191D29.D42@llnl.gov> Date: Wed, 07 Feb 1996 13:44:09 -0800 From: "Paul. Dubois" Organization: Lawrence Livermore National Laboratory X-Mailer: Mozilla 2.0b5 (X11; I; SunOS 5.4 sun4m) Mime-Version: 1.0 To: James Hugunin Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Type Coercion References: <9602072018.AA17817@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk I hold these truths to be self-evident, your argument notwithstanding: a. A PyFloat is a C double The fact that some of the Python API lets you convert one to a C float easily doesn't change this. b. It is a GOOD THING that you can count on the precision of a PyFloat. Writing portable code between platforms is easy with Python, since even on 64 bit machines C doubles are implemented as 64 bit floats. (I know that isn't for certain but as a practical truth, it is true). c. It is a GOOD THING not to lose precision. d. It is a GOOD THING not to get an exception. The original design functioned perfectly, as far as I was concerned. The change has been made because of a desire not to GAIN precision. This is only a concern if you have a concern about space, really. (The time penalty on most workstations for double vs. float is nowhere near 2). So this means you must have a humongous amount of data. In which case, Python is a lousy choice anyway, since it will make lots of temporaries in the act of expression evaluation, while a nice Fortran routine won't. In short, I'd argue that it is not a good idea to pervert the matrix class in Python to satisfy a marginal need. But the alternative is unclean. The minute you find yourself proposing that elements of an array are zero-d arrays you should realize the slope has gotten too slippery. I return to my previous horror example: (1./3.) * a_float[i] vs. ((1./3.) * a_float)[i] Can you really stand for those to be different? Vectorize a loop and change the answer? -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 16:47:41 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA20051 for matrix-sig-people; Wed, 7 Feb 1996 16:47:41 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id QAA20045 for ; Wed, 7 Feb 1996 16:47:32 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id QAA04200 (8.6.11/IDA-1.6); Wed, 7 Feb 1996 16:45:52 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA10255; Wed, 7 Feb 1996 16:46:09 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA24976; Wed, 7 Feb 1996 16:46:08 -0500 Date: Wed, 7 Feb 1996 16:46:08 -0500 Message-Id: <199602072146.QAA24976@cyclone.ERE.UMontreal.CA> To: jfulton@usgs.gov CC: guido@CNRI.Reston.VA.US, jjh@goldilocks.lcs.mit.edu, matrix-sig@python.org In-reply-to: <9602071639.ZM12755@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] New pretty printer From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > OK. Then it makes sense to add another parameter for printing > precision and use it also for scalars. I'd rather see precision handled as an array attribute. But it isn't. If precision were an attribute of an object, you would expect arithmetic etc. to respect it. But what we are discussing is just a matter of output formatting. To be consistent, you would also have to require that the radix for integer output be a property of an object, creating separate types for decimal and hex numbers. If precision were an attribute, we would also have to decide what the precision of the result of an array operation is. This just doesn't make sense. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 16:56:04 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA20177 for matrix-sig-people; Wed, 7 Feb 1996 16:56:04 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id QAA20173 for ; Wed, 7 Feb 1996 16:56:01 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id VAA12806; Wed, 7 Feb 1996 21:55:36 GMT From: "Jim Fulton" Message-Id: <9602071655.ZM12804@dsdbqvarsa.er.usgs.gov> Date: Wed, 7 Feb 1996 16:55:36 -0500 In-Reply-To: hinsenk@ERE.UMontreal.CA (Konrad HINSEN) "Re: [PYTHON MATRIX-SIG] New pretty printer" (Feb 7, 4:46pm) References: <199602072146.QAA24976@cyclone.ERE.UMontreal.CA> X-Mailer: Z-Mail (3.2.0 06sep94) To: hinsenk@ere.umontreal.ca (Konrad HINSEN) Subject: Re: [PYTHON MATRIX-SIG] New pretty printer Cc: guido@CNRI.Reston.VA.US, jjh@goldilocks.lcs.mit.edu, matrix-sig@python.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit Sender: owner-matrix-sig@python.org Precedence: bulk On Feb 7, 4:46pm, Konrad HINSEN wrote: > Subject: Re: [PYTHON MATRIX-SIG] New pretty printer > > > OK. Then it makes sense to add another parameter for printing > > precision and use it also for scalars. > > I'd rather see precision handled as an array attribute. > > But it isn't. If precision were an attribute of an object, you would > expect arithmetic etc. to respect it. But what we are discussing is > just a matter of output formatting. To be consistent, you would > also have to require that the radix for integer output be a > property of an object, creating separate types for decimal and > hex numbers. > > If precision were an attribute, we would also have to decide what the > precision of the result of an array operation is. This just doesn't > make sense. Perhaps precision is a poor term. "output precision" is fine with me. My opinion is based on two ideas: - The output precision is needed specifically for printing arrays, mainly because you have lots of data to output. - Global variables are bad. A correlary is that changes to core things like sys is bad. I don't mind output width beig added to sys, because it clearly is not array specific. (Well, maybe I do. :) Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 16:56:43 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA20190 for matrix-sig-people; Wed, 7 Feb 1996 16:56:43 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id QAA20186 for ; Wed, 7 Feb 1996 16:56:40 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id VAA12812; Wed, 7 Feb 1996 21:56:31 GMT From: "Jim Fulton" Message-Id: <9602071656.ZM12810@dsdbqvarsa.er.usgs.gov> Date: Wed, 7 Feb 1996 16:56:30 -0500 In-Reply-To: "Paul. Dubois" "Re: [PYTHON MATRIX-SIG] Type Coercion" (Feb 7, 1:44pm) References: <9602072018.AA17817@baribal.LCS.MIT.EDU.LCS.MIT.EDU> <31191D29.D42@llnl.gov> X-Mailer: Z-Mail (3.2.0 06sep94) To: "Paul. Dubois" , James Hugunin Subject: Re: [PYTHON MATRIX-SIG] Type Coercion Cc: matrix-sig@python.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit Sender: owner-matrix-sig@python.org Precedence: bulk I agree. On Feb 7, 1:44pm, Paul. Dubois wrote: > Subject: Re: [PYTHON MATRIX-SIG] Type Coercion > I hold these truths to be self-evident, your argument notwithstanding: > a. A PyFloat is a C double > The fact that some of the Python API lets you convert one to a C > float easily doesn't change this. > > b. It is a GOOD THING that you can count on the precision of a PyFloat. > Writing portable code between platforms is easy with Python, since > even on 64 bit machines C doubles are implemented as 64 bit floats. (I > know that isn't for certain but as a practical truth, it is true). > > c. It is a GOOD THING not to lose precision. > d. It is a GOOD THING not to get an exception. > > The original design functioned perfectly, as far as I was concerned. > > The change has been made because of a desire not to GAIN precision. This > is only a concern if you have a concern about space, really. (The time > penalty on most workstations for double vs. float is nowhere near 2). > So this means you must have a humongous amount of data. In which case, > Python is a lousy choice anyway, since it will make lots of temporaries > in the act of expression evaluation, while a nice Fortran routine won't. > > In short, I'd argue that it is not a good idea to pervert the matrix > class in Python to satisfy a marginal need. > > But the alternative is unclean. The minute you find yourself proposing > that elements of an array are zero-d arrays you should realize the slope > has gotten too slippery. > > I return to my previous horror example: > > (1./3.) * a_float[i] > vs. > ((1./3.) * a_float)[i] > > Can you really stand for those to be different? Vectorize a loop and > change the answer? > -- > Paul F. Dubois, L-472 (510)-422-5426 > Lawrence Livermore National Laboratory FAX (510)-423-9969 > Livermore, CA 94550 dubois1@llnl.gov > Consulting: PaulDubois@aol.com > > ================= > MATRIX-SIG - SIG on Matrix Math for Python > > send messages to: matrix-sig@python.org > administrivia to: matrix-sig-request@python.org > ================= >-- End of excerpt from Paul. Dubois ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 17:01:19 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA20255 for matrix-sig-people; Wed, 7 Feb 1996 17:01:19 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id RAA20251 for ; Wed, 7 Feb 1996 17:01:16 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id QAA05041 (8.6.11/IDA-1.6); Wed, 7 Feb 1996 16:59:28 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA13640; Wed, 7 Feb 1996 16:59:45 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA25799; Wed, 7 Feb 1996 16:59:44 -0500 Date: Wed, 7 Feb 1996 16:59:44 -0500 Message-Id: <199602072159.QAA25799@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: matrix-sig@python.org In-reply-to: <9602072137.AA18068@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk lack the time to sit down and do a careful redesign. The current feature set is significantly more powerful than the basic array operations provided by matlab, the most popular array processing language that people buy. Also the worst one, mostly because it is "historically grown" from a very restrictive basis (LINPACK). Rather than complaining about the lack of a list of the available functions, David Ascher recently sat down and pulled them out of the C and python source code (not too hard to do, since I did include doc strings for everything). Check out his beta tutorial I know, and in fact I have postponed my own attempts waiting for him to be continue the investigation. I wasn't really complaining, but providing an excuse for not having done something until now. If somebody else out there has the time and energy to design a complete set of functions and methods for the system, go for it. If I volunteer to do so, but can't promise any deadline. I am also doing all my work on Python as a side project, so I fully sympathize with your problems of allocating time to it! Sorry for the negative tone, but I've been getting a few too many questions like "So how's your speech recognition system coming along?" from my advisor lately and I'm eager to get the array object "out the door". Bug fixes take up very little of my intellectual energy, but I fully understand that, but I don't see the need to rush. After all, we are doing something close to language design, which has never profited from tight deadlines. Rather than rush out a beta version to the public, I advocate a public alpha release that comes with no promise about stability of function names etc. We need all the test users we can get! Why make any commitments without anybody forcing us? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 17:13:20 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA20402 for matrix-sig-people; Wed, 7 Feb 1996 17:13:20 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id RAA20398 for ; Wed, 7 Feb 1996 17:13:16 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id QAA03906 (8.6.11/IDA-1.6); Wed, 7 Feb 1996 16:36:42 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA07838; Wed, 7 Feb 1996 16:36:59 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA24352; Wed, 7 Feb 1996 16:36:58 -0500 Date: Wed, 7 Feb 1996 16:36:58 -0500 Message-Id: <199602072136.QAA24352@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: matrix-sig@python.org In-reply-to: <9602072018.AA17817@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] Type Coercion From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk Python has only one floating point type, call this a PyFloat. Internally, these numbers store their data using C doubles. However, the standard C interface to PyFloats allows them to be extracted into either a C float or a C double (PyArg_ParseTuple), so its not clear that a PyFloat should really be considered either a float or a double. The association of PyFloats with C floats exists only in the C interface, presumably to facilitate interfacing to existing C code. No part of the standard language makes use of it, and to an average Python user PyFloat are definitely C doubles. Three things can happen: 1) Return an array of floats (possible undesired loss of precision) 2) Return an array of doubles (possible undesired gain of precision) 3) Raise an exception (very annoying to the user) The potential problems in 1) and 2) are also very annoying to the user! This particular problem could be eliminated by having a_float[0] return a 0d array of floats with a single element. Perhaps this is worth considering. I don't see any serious problem with that. Most Python code will work just as well with an array of rank 0 as with a scalar. Only code that does explicit type checks will have to take care of the difference. And explicit conversion would always be possible using the standard conversion functions. Personally I would prefer to use method 1, combined with the major change to the existing system that scalars are returned as 0d arrays, not as python scalars. Personally I see this as the least desirable option, my preference being in the order 3) 2) 1). My argument is again the principle of least surprise: if something unexpected is going to happen, I prefer it to be 1) an exception 2) a loss in efficiency 3) a loss in accuracy. in that order. Let me propose another solution, fully knowing that Guido might throw a 16-ton weight at me: Suppose we modify the Python float object by adding a flag "single precision". This flag would be set if either 1) A constant is created that can be represented in a C float (such as 0.5). 2) A constant is created with a special suffix like 'S' (to be used for low-precision constants with no exact binary representation, such as 0.1, if precision doesn't matter). 3) When a PyFloat is generated as a rank-0 float array. 4) Possibly as the result of an explicit conversion to a "low precision scalar", if that turns out to be useful. The default behaviour of PyFloats would not change at all, and so there are no compatibility problems. Any result of a scalar operation would not have the flag set, of course. There is also no significant speed penalty, since only the creation of a constant becomes more involved, and that happens at compile time. The only price to pay would be an increase in the size of a float object. The only place where the new flag would matter is in coercion to an array object. It should take care of most the problems of method 2), which after all is the most common coercion convention in programming languages. Unwanted promotion to double can still occur in situations such as 2.*pi*array([1.,2.,3.], Float(32)), because 2.*pi would be "non-degradable" even if pi was, because it is the result of a scalar operation. But such standard cases can be handled by something like twopi = short_float(2.*pi), where short_float() is the conversion function mentioned above. To help pinning down unwanted coercion to double, one could further offer some option that raises an exception for such operations. This would have the status of a debugging aid rather than a language feature and could, for example, be activated by a command line switch. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 17:20:32 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA20479 for matrix-sig-people; Wed, 7 Feb 1996 17:20:32 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id RAA20475 for ; Wed, 7 Feb 1996 17:20:19 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id RAA05530 (8.6.11/IDA-1.6); Wed, 7 Feb 1996 17:13:39 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA18180; Wed, 7 Feb 1996 17:13:57 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA26801; Wed, 7 Feb 1996 17:13:56 -0500 Date: Wed, 7 Feb 1996 17:13:56 -0500 Message-Id: <199602072213.RAA26801@cyclone.ERE.UMontreal.CA> To: jfulton@usgs.gov CC: guido@CNRI.Reston.VA.US, jjh@goldilocks.lcs.mit.edu, matrix-sig@python.org In-reply-to: <9602071655.ZM12804@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] New pretty printer From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk Perhaps precision is a poor term. "output precision" is fine with me. I guess the name is the least problem... The main question remains: If precision becomes a property, what is the precision of a+b, if a has precision 2 and b has precision 7? And why? If this question doesn't make sense, then output precision should not be an attribute. My opinion is based on two ideas: - The output precision is needed specifically for printing arrays, mainly because you have lots of data to output. Needed, yes. But it makes sense and could be easily implemented for scalars as well. - Global variables are bad. A correlary is that changes to core things like sys is bad. I don't mind output width beig added to sys, because it clearly is not array specific. (Well, maybe I do. :) Line width is just as much array specific, as long as no other module makes use of it. I think we are overly dramatizing the "global variable" problem here. There is very little reason to change this global variable except for interactive use. Any module that want output with a specific format would call the output function Numeric.arrayToString() directly and supply line width and precision as optional arguments. The default values from the global variables would be used *only* when someone writes "print a". They should be set only by the end user, never by any library code. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 17:40:58 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA20626 for matrix-sig-people; Wed, 7 Feb 1996 17:40:58 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id RAA20622 for ; Wed, 7 Feb 1996 17:40:54 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA03223; Wed, 7 Feb 96 14:40:23 PST Received: from kristen (localhost) by kristen.llnl.gov (5.x/SMI-SVR4) id AA17910; Wed, 7 Feb 1996 14:40:23 -0800 Message-Id: <31192A57.6665@llnl.gov> Date: Wed, 07 Feb 1996 14:40:23 -0800 From: "Paul. Dubois" Organization: Lawrence Livermore National Laboratory X-Mailer: Mozilla 2.0b5 (X11; I; SunOS 5.4 sun4m) Mime-Version: 1.0 To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] printing arrays References: <199602071711.MAA07277@cyclone.ERE.UMontreal.CA> <9602071639.ZM12755@dsdbqvarsa.er.usgs.gov> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk Here is my take on this: >From a purely object-oriented point of view, the width/precision are attributes of the *array formatter*, ie an object that knows how to print arrays. At the moment, this gets confounded with the array itself. The default formatter object could be a class variable (i.e., when an array is created, it has as its default formatter a single shared formatter). This means that changing the precision in the shared formatter would change it across the board. The discussion has really arisen because array isn't really a class and the "object" that does the printing in it is a function that doesn't have any state, so we've started to talk about where to put that state. In general, while I appreciate Jim Fulton's concern about compartmentalizing things, I think in practice the idea of a user-changeable default printing precision is well established in similar products. "How do I increase the number of decimals" is certainly a FAQ for Basis users. They don't ask, "How do I increase the number of decimals for printing THIS array only." -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 17:52:52 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA20695 for matrix-sig-people; Wed, 7 Feb 1996 17:52:52 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id RAA20691 for ; Wed, 7 Feb 1996 17:52:48 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id WAA12951; Wed, 7 Feb 1996 22:52:47 GMT From: "Jim Fulton" Message-Id: <9602071752.ZM12949@dsdbqvarsa.er.usgs.gov> Date: Wed, 7 Feb 1996 17:52:47 -0500 In-Reply-To: "Paul. Dubois" "[PYTHON MATRIX-SIG] printing arrays" (Feb 7, 2:40pm) References: <199602071711.MAA07277@cyclone.ERE.UMontreal.CA> <9602071639.ZM12755@dsdbqvarsa.er.usgs.gov> <31192A57.6665@llnl.gov> X-Mailer: Z-Mail (3.2.0 06sep94) To: "Paul. Dubois" , matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] printing arrays Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit Sender: owner-matrix-sig@python.org Precedence: bulk On Feb 7, 2:40pm, Paul. Dubois wrote: > Subject: [PYTHON MATRIX-SIG] printing arrays > Here is my take on this: > > >From a purely object-oriented point of view, the width/precision are > attributes of the *array formatter*, ie an object that knows how to > print arrays. At the moment, this gets confounded with the array itself. > The default formatter object could be a class variable (i.e., when an > array is created, it has as its default formatter a single shared > formatter). This means that changing the precision in the shared > formatter would change it across the board. > > The discussion has really arisen because array isn't really a class and > the "object" that does the printing in it is a function that doesn't > have any state, so we've started to talk about where to put that state. > > In general, while I appreciate Jim Fulton's concern about > compartmentalizing things, I think in practice the idea of a > user-changeable default printing precision is well established in > similar products. "How do I increase the number of decimals" is > certainly a FAQ for Basis users. They don't ask, "How do I increase the > number of decimals for printing THIS array only." OK, I have no problem with a user settable default precision. Why not have a default set in Numeric and let arrays also have attributes that are initialized with the default, and can be set by the user? Actually, my main gripe at this point is that I don't like diddling sys for. I can live with having global variables set in some array-specific module. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 17:54:52 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA20710 for matrix-sig-people; Wed, 7 Feb 1996 17:54:52 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id RAA20706 for ; Wed, 7 Feb 1996 17:54:50 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id RAA06743 (8.6.11/IDA-1.6); Wed, 7 Feb 1996 17:53:09 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA04624; Wed, 7 Feb 1996 17:53:28 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA27524; Wed, 7 Feb 1996 17:26:00 -0500 Date: Wed, 7 Feb 1996 17:26:00 -0500 Message-Id: <199602072226.RAA27524@cyclone.ERE.UMontreal.CA> To: dubois1@llnl.gov CC: jjh@goldilocks.lcs.mit.edu, matrix-sig@python.org In-reply-to: <31191D29.D42@llnl.gov> (dubois1@llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] Type Coercion From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk The change has been made because of a desire not to GAIN precision. This is only a concern if you have a concern about space, really. (The time penalty on most workstations for double vs. float is nowhere near 2). So this means you must have a humongous amount of data. In which case, Python is a lousy choice anyway, since it will make lots of temporaries in the act of expression evaluation, while a nice Fortran routine won't. You might need floats just for interfacing with existing code. Or for a big application in which most of the processing happens in C/Fortran modules. Anyway, if there is a need to have float arrays, then there is also a need to keep them to float size in a reasonable way. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 18:04:30 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA20831 for matrix-sig-people; Wed, 7 Feb 1996 18:04:30 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id SAA20826 for ; Wed, 7 Feb 1996 18:04:26 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA00651; Wed, 7 Feb 96 17:58:00 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) id SAA18847; Wed, 7 Feb 1996 18:09:29 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199602072309.SAA18847@maigret> Subject: Re: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience To: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Wed, 7 Feb 1996 18:09:29 -0500 (EST) Cc: jjh@goldilocks.lcs.mit.edu, matrix-sig@python.org In-Reply-To: <199602072159.QAA25799@cyclone.ERE.UMontreal.CA> from "Konrad HINSEN" at Feb 7, 96 04:59:44 pm X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 572 Sender: owner-matrix-sig@python.org Precedence: bulk > If somebody else out there has the time and energy to design a > complete set of functions and methods for the system, go for it. If > > I volunteer to do so, but can't promise any deadline. I am also doing > all my work on Python as a side project, so I fully sympathize with > your problems of allocating time to it! You know, I think we should setup a support group for people who'se lives are disrupted by this damn computer language. Hell, my SO would qualify under that definition. With a special low membership fee for PhD students, of course. --da ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 18:11:17 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA20905 for matrix-sig-people; Wed, 7 Feb 1996 18:11:17 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id SAA20901 for ; Wed, 7 Feb 1996 18:11:13 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA03576; Wed, 7 Feb 96 15:10:42 PST Received: from kristen (localhost) by kristen.llnl.gov (5.x/SMI-SVR4) id AA17993; Wed, 7 Feb 1996 15:10:40 -0800 Message-Id: <31193170.31B3@llnl.gov> Date: Wed, 07 Feb 1996 15:10:40 -0800 From: "Paul. Dubois" Organization: Lawrence Livermore National Laboratory X-Mailer: Mozilla 2.0b5 (X11; I; SunOS 5.4 sun4m) Mime-Version: 1.0 To: Konrad HINSEN Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Type Coercion References: <199602072226.RAA27524@cyclone.ERE.UMontreal.CA> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk Konrad remarks that you have to "keep" yourself at the lower precision so you can interact to C/Fortran. This is not strictly correct. You just have to get yourself *back* first. But: We have been doing just that lately and in fact you have to do a conversion of some sort nearly every time if you want to be safe, anyway. This has to do with the contiguous issue. For example, you have a C plotting array that wants an array of C floats. You might even have an array of C floats but in general you don't know it is contiguous. It might be a slice of something else, or the result of any number of calculations that destroy contiguity (if that is a word!). So even if you have made yourself double, you aren't any worse off as you repack yourself contiguous/float to get to the C routine. -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 18:12:12 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA20920 for matrix-sig-people; Wed, 7 Feb 1996 18:12:12 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id SAA20916 for ; Wed, 7 Feb 1996 18:12:09 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id SAA07169 (8.6.11/IDA-1.6); Wed, 7 Feb 1996 18:10:32 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id SAA10297; Wed, 7 Feb 1996 18:10:50 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id SAA01109; Wed, 7 Feb 1996 18:10:49 -0500 Date: Wed, 7 Feb 1996 18:10:49 -0500 Message-Id: <199602072310.SAA01109@cyclone.ERE.UMontreal.CA> To: jfulton@usgs.gov CC: dubois1@llnl.gov, matrix-sig@python.org In-reply-to: <9602071752.ZM12949@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov) Subject: Re: [PYTHON MATRIX-SIG] printing arrays From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk Actually, my main gripe at this point is that I don't like diddling sys for. I can live with having global variables set in some array-specific module. I don't care too much whether this goes to Numeric or sys. I still consider sys more appropriate just because these definitions are not specific to arrays, but I won't insist. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 18:12:58 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA20933 for matrix-sig-people; Wed, 7 Feb 1996 18:12:58 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id SAA20928 for ; Wed, 7 Feb 1996 18:12:55 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA03639; Wed, 7 Feb 96 15:12:24 PST Received: from kristen (localhost) by kristen.llnl.gov (5.x/SMI-SVR4) id AA18006; Wed, 7 Feb 1996 15:12:22 -0800 Message-Id: <311931D6.6B1E@llnl.gov> Date: Wed, 07 Feb 1996 15:12:22 -0800 From: "Paul. Dubois" Organization: Lawrence Livermore National Laboratory X-Mailer: Mozilla 2.0b5 (X11; I; SunOS 5.4 sun4m) Mime-Version: 1.0 To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] [Fwd: Foreign Function Interface Generator available.] Content-Type: multipart/mixed; boundary="------------521F5C6038EE" Sender: owner-matrix-sig@python.org Precedence: bulk This is a multi-part message in MIME format. --------------521F5C6038EE Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit I thought this audience might find this interesting, considering some of the discussions at Spam III. -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com --------------521F5C6038EE Content-Type: message/rfc822 Content-Transfer-Encoding: 7bit Content-Disposition: inline Return-Path: Received: from icf.llnl.gov by kristen.llnl.gov (5.x/SMI-SVR4) id AA17149; Wed, 7 Feb 1996 12:28:33 -0800 Received: from pierce.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA00439; Wed, 7 Feb 96 12:28:32 PST Received: by pierce.llnl.gov (8.6.10/LLNL-1.18/llnl.gov-03.95) id MAA20155; Wed, 7 Feb 1996 12:28:31 -0800 Received: from alpha.xerox.com by pierce.llnl.gov (8.6.10/LLNL-1.18/llnl.gov-03.95) id MAA20146; Wed, 7 Feb 1996 12:28:29 -0800 Received: from Homer.Parc.Xerox.xns by alpha.xerox.com via XNS id <15450(15)>; Wed, 7 Feb 1996 12:19:11 PST Received: from skinner.cs.uoregon.edu ([128.223.4.13]) by alpha.xerox.com with SMTP id <15439(2)>; Wed, 7 Feb 1996 12:06:23 PST Received: from blackrabbit.cs.uoregon.edu by skinner.cs.uoregon.edu with SMTP id AA13990 (5.65/IDA-1.4.2 for ilu@parc.xerox.com); Wed, 7 Feb 96 12:05:22 -0800 Received: from blackrabbit.cs.uoregon.edu (localhost [127.0.0.1]) by blackrabbit.cs.uoregon.edu (8.6.12/8.6.12) with ESMTP id MAA20417; Wed, 7 Feb 1996 12:02:27 -0800 X-Ns-Transport-Id: 0800207424C500CC559B Date: Wed, 7 Feb 1996 12:02:26 PST From: Lars Thomas Hansen Subject: Foreign Function Interface Generator available. To: fjh%cs.mu.OZ.AU@uunet.uu.net, will@ccs.neu.edu, jcg@cs.cmu.edu, gurr@snap.med.ge.com, nr@cs.purdue.edu, Geoff.Wyant@east.sun.com, kalsow@sctc.com, nayeri@gte.com, ilu@parc.xerox.com, scheme@mc.lcs.mit.edu Cc: lth@cs.uoregon.edu, malony@cs.uoregon.edu, hersey@cs.uoregon.edu Message-Id: <199602072002.MAA20417@blackrabbit.cs.uoregon.edu> Content-Type: text Content-Length: 901 X-Mozilla-Status: 0001 FFIGEN Release 1 is now available from http://www.cs.uoregon.edu/~lth/ffigen. FFIGEN (Foreign Function Interface GENerator) is a program suite which facilitates the writing of translators from C header files to foreign function interfaces for particular language implementations. It is based on the lcc C compiler and handles nearly all of ANSI C, including the preprocessor. (Missing features are bitfields and general type qualifiers; support for those will be included in a future version). This release includes the entire front-end, a back-end for Chez Scheme version 5, and documents which explain the ideas behind FFIGEN, its use, and how to write of custom back-ends for you language. This is a preliminary release; it works but is neither full-function nor polished. In particular, installation is crude. See the Web page for further details, download information, and so on. --lars --------------521F5C6038EE-- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 18:22:44 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA21024 for matrix-sig-people; Wed, 7 Feb 1996 18:22:44 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id SAA21019 for ; Wed, 7 Feb 1996 18:22:39 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id SAA07445 (8.6.11/IDA-1.6); Wed, 7 Feb 1996 18:21:01 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id SAA13954; Wed, 7 Feb 1996 18:21:20 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id SAA01631; Wed, 7 Feb 1996 18:21:19 -0500 Date: Wed, 7 Feb 1996 18:21:19 -0500 Message-Id: <199602072321.SAA01631@cyclone.ERE.UMontreal.CA> To: dubois1@llnl.gov CC: matrix-sig@python.org In-reply-to: <31193170.31B3@llnl.gov> (dubois1@llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] Type Coercion From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk a C plotting array that wants an array of C floats. You might even have an array of C floats but in general you don't know it is contiguous. It might be a slice of something else, or the result of any number of calculations that destroy contiguity (if that is a word!). Believe it or not, it is accepted by "spell" on my system ;-) Contiguity might not be an issue when interfacing to Fortran code, which can often handle non-contiguous arrays. Anyway, the C-API should have the utility functions "copy if not contiguous" and "copy if not sufficiently contiguous to be passed to a Fortran function". There could be versions that include a cast. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 18:31:39 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA21073 for matrix-sig-people; Wed, 7 Feb 1996 18:31:39 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id SAA21064 for ; Wed, 7 Feb 1996 18:31:21 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA18989; Wed, 7 Feb 96 18:30:50 EST Date: Wed, 7 Feb 96 18:30:50 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9602072330.AA18989@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: hinsenk@ere.umontreal.ca Cc: dubois1@llnl.gov, matrix-sig@python.org In-Reply-To: <199602072321.SAA01631@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Subject: Re: [PYTHON MATRIX-SIG] Type Coercion Sender: owner-matrix-sig@python.org Precedence: bulk From: hinsenk@ere.umontreal.ca (Konrad HINSEN) a C plotting array that wants an array of C floats. You might even have an array of C floats but in general you don't know it is contiguous. It might be a slice of something else, or the result of any number of calculations that destroy contiguity (if that is a word!). Believe it or not, it is accepted by "spell" on my system ;-) Contiguity might not be an issue when interfacing to Fortran code, which can often handle non-contiguous arrays. Anyway, the C-API should have the utility functions "copy if not contiguous" and "copy if not sufficiently contiguous to be passed to a Fortran function". There could be versions that include a cast. Just to clarify the facts on this issue. The C-API has the standard function PyArray_ContiguousFromObject which is the standard method of obtaining an array. If the object is a contiguous PyArray of the same type you desire, no memory copying occurs. Otherwise there is obviously a copying step and possibly a cast required. The two other "standard" ways of obtaining an array from a python object are PyArray_CopyFromObject which guarantees you get a copy and can do with the memory area what you please. PyArray_FromObject which can give you discontiguous arrays if you want to deal with them (this is what the internal ofuncs use). So the facts are that there is in fact a cost proportional to the size of the array when you pass an array of doubles to one expecting floats. However, the cast operation is fairly efficient, so I doubt that this particular case really matters in practice at all. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 19:02:19 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id TAA21323 for matrix-sig-people; Wed, 7 Feb 1996 19:02:19 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id TAA21318 for ; Wed, 7 Feb 1996 19:02:12 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id TAA08334 (8.6.11/IDA-1.6 for ); Wed, 7 Feb 1996 19:00:36 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id TAA23655; Wed, 7 Feb 1996 19:00:55 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id TAA04035; Wed, 7 Feb 1996 19:00:54 -0500 Date: Wed, 7 Feb 1996 19:00:54 -0500 Message-Id: <199602080000.TAA04035@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Pretty Printer From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk And here's another pretty printer. Float and complex formatting is much improved, but there is still no line wrapping. Precision control works, however. Note that due to the problem in compress() complex arrays will still often be printed with an inappropriate format. It is also still impossible to print a float array whose elements are all zero. This will be possible with one of Jim's upcoming bug fixes. Line width and precision are currently taken from global variables in sys if these exist and no explicit parameters are given. If the general opinion favors Numeric, I'll be happy to change this. For now you should not forget to put an "import sys" in your Numeric.py. Before I forget: Precision is indicated by an integer specifying the number of decimal digits after the decimal point. If there are good arguments for another specification, I'll be happy to change this. I am now quite happy with formatting of floats and complex numbers, although I admit that I haven't done extensive tests. I hope there won't be complaints about speed or lack thereof. I'd appreciate any feedback on how you like the float formatting. I now remove all trailing zeros, although Python scalars get printed with at least one. Would you prefer full compatibility in this case? Or would you prefer to have no decimal point if there are no fractional digits? And now the code: --------------------------------------------------------------------------- import sys def arrayToString(a, max_line_width = None, precision = None): if not max_line_width: try: max_line_width = sys.output_line_width except AttributeError: max_line_width = 77 if not precision: try: precision = sys.float_output_precision except AttributeError: precision = 8 if a.contiguous(): data = a.reshape(None) else: data = a.copy().reshape(None) type = a.typecode() items_per_line = a.shape[-1] if type == 'b' or type == '1' or type == 's' or type == 'i' \ or type == 'l': max_str_len = max(len(str(maximum.reduce(data))), len(str(minimum.reduce(data)))) format = '%' + str(max_str_len) + 'd' item_length = max_str_len+1 format_function = lambda x, f = format: _formatInteger(x, f) elif type == 'f' or type == 'd': format, item_length = _floatFormat(data, precision) format_function = lambda x, f = format: _formatFloat(x, f) elif type == 'F' or type == 'D': real_format, real_item_length = _floatFormat(data.real, precision, sign=0) imag_format, imag_item_length = _floatFormat(data.imaginary, precision, sign=1) item_length = real_item_length + imag_item_length + 2 format_function = lambda x, f1 = real_format, f2 = imag_format: \ _formatComplex(x, f1, f2) elif type == 'c': format = '%s' item_length = 1 format_function = lambda x, f = format: _formatCharacter(x, f) else: # nothing for 'O' return str(a) line_width = item_length*items_per_line - (type != 'c') if line_width > max_line_width: pass return _arrayToString(a, format_function, len(a.shape), line_width)[:-1] def _floatFormat(data, precision, sign = 0): exp_format = 0 non_zero = abs(compress(data.notEqual(0), data)) if len(non_zero) == 0: max_val = 0. min_val = 0. else: max_val = maximum.reduce(non_zero) min_val = minimum.reduce(non_zero) if max_val >= 1.e12 or min_val < 0.0001 or max_val/min_val > 1000.: exp_format = 1 if exp_format: large_exponent = 0 < min_val < 1e-99 or max_val >= 1e100 max_str_len = 8 + precision + large_exponent if sign: format = '%+' else: format = '%' format = format + str(max_str_len) + '.' + str(precision) + 'e' if large_exponent: format = format + '3' item_length = max_str_len + 1 else: format = '%.' + str(precision) + 'f' precision = min(precision, apply(max, tuple(map(lambda x, p=precision, f=format: _digits(x,p,f), data)))) max_str_len = len(str(int(max_val))) + precision + 2 if sign: format = '%#+' else: format = '%#' format = format + str(max_str_len) + '.' + str(precision) + 'f' item_length = max_str_len + 1 return (format, item_length) def _digits(x, precision, format): s = format % x zeros = len(s) while s[zeros-1] == '0': zeros = zeros-1 return precision-len(s)+zeros def _arrayToString(a, format_function, rank, line_width): if rank == 0: return str(a[0]) elif rank == 1: s = '' for i in range(a.shape[0]): s = s + format_function(a[i]) if s[-1] == ' ': s = s[:-1] s = s + '\n' else: s = '' for i in range(a.shape[0]-1): s = s + _arrayToString(a[i], format_function, rank-1, line_width) if rank == 3: s = s + '\n' elif rank > 3: s = s + (rank-3)*(line_width*'-'+'\n') s = s + _arrayToString(a[a.shape[0]-1], format_function, rank-1, line_width) return s def _formatInteger(x, format): return format % x + ' ' def _formatFloat(x, format, strip_zeros = 1): if format[-1] == '3': format = format[:-1] s = format % x third = s[-3] if third == '+' or third == '-': s = s[1:-2] + '0' + s[-2:] elif format[-1] == 'f': s = format % x if strip_zeros: zeros = len(s) while s[zeros-1] == '0': zeros = zeros-1 s = s[:zeros] + (len(s)-zeros)*' ' else: s = format % x return s + ' ' def _formatComplex(x, real_format, imag_format): r = _formatFloat(x.real, real_format)[:-1] i = _formatFloat(x.imag, imag_format, 0)[:-1] spaces = 0 while r[spaces] == ' ': spaces = spaces + 1 r = spaces*' ' + '(' + r[spaces:] if imag_format[-1] == 'f': zeros = len(i) while i[zeros-1] == '0': zeros = zeros-1 i = i[:zeros] + 'j)' + (len(i)-zeros)*' ' else: i = i + 'j)' return r + i + ' ' def _formatCharacter(x, format): return format % x ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 7 22:18:10 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id WAA22525 for matrix-sig-people; Wed, 7 Feb 1996 22:18:10 -0500 Received: from atr-sw.atr-sw.atr.co.jp (atr-sw.atr-sw.atr.co.jp [133.186.20.117]) by python.org (8.6.12/8.6.12) with ESMTP id WAA22487 for ; Wed, 7 Feb 1996 22:12:24 -0500 Received: from ciris21.atr-sw.atr.co.jp (ciris21.atr-sw.atr.co.jp [133.186.20.41]) by atr-sw.atr-sw.atr.co.jp (8.6.11+2.5Wb2/3.4Wbeta3/95031616ATR-SW) with ESMTP id MAA23968; Thu, 8 Feb 1996 12:11:05 +0900 Received: from localhost by ciris21.atr-sw.atr.co.jp (940816.SGI.8.6.9/6.4J.6) id MAA03526; Thu, 8 Feb 1996 12:11:02 +0900 Message-Id: <199602080311.MAA03526@ciris21.atr-sw.atr.co.jp> To: hinsenk@ere.umontreal.ca (Konrad HINSEN) CC: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Pretty Printer In-reply-to: Your message of "Wed, 07 Feb 1996 19:00:54 JST" References: <199602080000.TAA04035@cyclone.ERE.UMontreal.CA> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Date: Thu, 08 Feb 1996 12:11:01 +0900 From: "Perry A. Stoll" Sender: owner-matrix-sig@python.org Precedence: bulk As Konrad's pretty printer continues to evolve (please!) it seems to be taking over most of the Numeric module. How about we make it a separate file? I've done the following in my Python: 1) Save his pretty printer as a separate python file ArrayFormatter.py 2) Add "import Numeric" to the top of ArrayFormatter.py file and prepend "Numeric." to the compress, maximum, and minimum calls. 3) Put the following near the end of Numeric.py: from ArrayFormatter import arrayToString # Set the official array printing function set_print_function(arrayToString) Seems like a nice division. -Perry ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 8 03:06:03 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id DAA23743 for matrix-sig-people; Thu, 8 Feb 1996 03:06:03 -0500 Received: from catch.ivab.se (root@catch.ivab.se [193.180.173.22]) by python.org (8.6.12/8.6.12) with ESMTP id DAA23739 for ; Thu, 8 Feb 1996 03:05:57 -0500 Received: from arnold.image.ivab.se (arnold.image.ivab.se [193.180.173.119]) by catch.ivab.se (8.6.12/8.6.12) with SMTP id JAA30437; Thu, 8 Feb 1996 09:04:39 +0100 Received: by arnold.image.ivab.se; (5.65v3.2/1.1.8.2/16Nov95-0104AM) id AA26470; Thu, 8 Feb 1996 09:04:22 +0100 Date: Thu, 8 Feb 1996 09:04:22 +0100 Message-Id: <9602080804.AA26470@arnold.image.ivab.se> From: Fredrik Lundh To: hinsenk@ere.umontreal.ca Cc: matrix-sig@python.org In-Reply-To: <199602072159.QAA25799@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Subject: Re: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience Reply-To: fredrik_lundh@ivab.se Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk > Rather than rush out a beta version to the public, I advocate a > public alpha release that comes with no promise about stability of > function names etc. We need all the test users we can get! Why make > any commitments without anybody forcing us? I support this view, if that's any help :-) /F BTW: what about work on standard packages?; I'd very much like some linear algebra stuff, for example to invert matrices (but don't know enough about it to contribute). In addition, I'd would like to use the matrix extension to do some of the serious processing stuff in my forthcoming Python Imaging library, like FFTs and 2D convolutions. Any ideas on how to achieve this? ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 8 03:08:50 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id DAA23743 for matrix-sig-people; Thu, 8 Feb 1996 03:06:03 -0500 Received: from catch.ivab.se (root@catch.ivab.se [193.180.173.22]) by python.org (8.6.12/8.6.12) with ESMTP id DAA23739 for ; Thu, 8 Feb 1996 03:05:57 -0500 Received: from arnold.image.ivab.se (arnold.image.ivab.se [193.180.173.119]) by catch.ivab.se (8.6.12/8.6.12) with SMTP id JAA30437; Thu, 8 Feb 1996 09:04:39 +0100 Received: by arnold.image.ivab.se; (5.65v3.2/1.1.8.2/16Nov95-0104AM) id AA26470; Thu, 8 Feb 1996 09:04:22 +0100 Date: Thu, 8 Feb 1996 09:04:22 +0100 Message-Id: <9602080804.AA26470@arnold.image.ivab.se> From: Fredrik Lundh To: hinsenk@ere.umontreal.ca Cc: matrix-sig@python.org In-Reply-To: <199602072159.QAA25799@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Subject: Re: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience Reply-To: fredrik_lundh@ivab.se Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk > Rather than rush out a beta version to the public, I advocate a > public alpha release that comes with no promise about stability of > function names etc. We need all the test users we can get! Why make > any commitments without anybody forcing us? I support this view, if that's any help :-) /F BTW: what about work on standard packages?; I'd very much like some linear algebra stuff, for example to invert matrices (but don't know enough about it to contribute). In addition, I'd would like to use the matrix extension to do some of the serious processing stuff in my forthcoming Python Imaging library, like FFTs and 2D convolutions. Any ideas on how to achieve this? ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 8 04:33:52 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id EAA24178 for matrix-sig-people; Thu, 8 Feb 1996 04:33:52 -0500 Received: from atr-sw.atr-sw.atr.co.jp (atr-sw.atr-sw.atr.co.jp [133.186.20.117]) by python.org (8.6.12/8.6.12) with ESMTP id EAA24173 for ; Thu, 8 Feb 1996 04:31:36 -0500 Received: from ciris21.atr-sw.atr.co.jp (ciris21.atr-sw.atr.co.jp [133.186.20.41]) by atr-sw.atr-sw.atr.co.jp (8.6.11+2.5Wb2/3.4Wbeta3/95031616ATR-SW) with ESMTP id SAA07151; Thu, 8 Feb 1996 18:22:15 +0900 Received: from localhost by ciris21.atr-sw.atr.co.jp (940816.SGI.8.6.9/6.4J.6) id SAA04171; Thu, 8 Feb 1996 18:22:13 +0900 Message-Id: <199602080922.SAA04171@ciris21.atr-sw.atr.co.jp> To: fredrik_lundh@ivab.se Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience In-reply-to: Your message of "Thu, 08 Feb 1996 09:04:22 JST" References: <9602080804.AA26470@arnold.image.ivab.se> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Date: Thu, 08 Feb 1996 18:22:12 +0900 From: "Perry A. Stoll" Sender: owner-matrix-sig@python.org Precedence: bulk >>>>> "FL" == Fredrik Lundh writes: FL> would like to use the matrix extension to do some of the FL> serious processing stuff in my forthcoming Python Imaging FL> library, like FFTs and 2D convolutions. Any ideas on how to FL> achieve this? I'm working on something similar. I've got various FFT packages off the net - I'll try to interface them, but I don't have time right now. If you are interested in doing it, let me know. In the mean time, for those wanting to play take a look at the file attached at the end. It's not the stuff legends are made of, but it should work. -Perry --------------------- cut here ------------------------------------ """ conv2 - support convolution of a 2 dimensional matrix by a 2 dimensional kernel. This routine was inspired by the matlab(c) version of conv2. Sample usage: >>> import conv2 # a sobel edge finding kernel >>> sobel = Numeric.array([[1.0 ,2, 1],[0, 0, 0],[-1, -2, -1]]) # a test array >>> A = Numeric.zeros(10,10) >>> S = Numeric.Slice(2,8) >>> A[(S,S)] = 1 # do the convolution >>> conv2.conv2(A,sobel) """ # Two dimension convolution for the Python Numeric package. # Should be redone in C. # # questions to: # Perry A. Stoll # or import Numeric,umath class AddSlice(Numeric.Slice): def __init__(self, start=0, stop=None, step=1): Numeric.Slice.__init__(self,start,stop, step) def __add__(self,x): newslice = AddSlice(self.start,self.stop,self.step) newslice.start = newslice.start + x if newslice.stop != None: newslice.stop = newslice.stop + x return newslice __radd__ = __add__ class IncSlice(Numeric.Slice): def __init__(self, start=0, stop=None, step=1): Numeric.Slice.__init__(self,start,stop,step) def inc(self): self.start = self.start + 1 if self.stop != None: self.stop = self.stop + 1 def conv2(arg1,arg2,shape='full'): """ C = conv2(A, B) performs the 2-D convolution of matrices A and B. If [ya,xa] == A.shape and [yb,xb] == B.shape, then C.shape == [ya+yb-1,xa+xb-1]. """ if (len(arg1.shape) != 2) or (len(arg2.shape) !=2): raise TypeError, "both arguments must be 2 dimensional" # we always want to use the small matrix as arg2, # so swap args if necessary swapped = 0 Nmr = Numeric.multiply.reduce size1,size2 = Nmr(arg1.shape),Nmr(arg2.shape) if (size2 > size1): swapped = 1 arg1,arg2 = arg2,arg1 # unpack shape tuples (y1,x1),(y2,x2) = arg1.shape,arg2.shape # create output matrix out = Numeric.zeros(y2+y1-1,x2+x1-1) cols,rows = IncSlice(0,x2),IncSlice(0,y2) for j in range(0,y1): rows.start,rows.stop = 0,y2 for i in range(0,x1): w = arg1[j,i] if w != 0: out[cols,rows] = out[cols,rows] + w*arg2 rows.inc() cols.inc() # not sure we need to do this... #if swapped: #arg1,arg2 = arg2,arg1 if (shape == 'full'): pass elif (shape == 'same'): rows = int(umath.floor(x2/2.0)) + AddSlice(0,x1) cols = int(umath.floor(y2/2.0)) + AddSlice(0,y1) out = out[cols,rows] elif (shape == 'valid'): rows = x2 -1 + AddSlice(0,x1-x2 + 1) cols = y2 -1 + AddSlice(0,y1-y2 + 1) out = out[cols,rows] return out ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 8 09:34:45 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA25243 for matrix-sig-people; Thu, 8 Feb 1996 09:34:45 -0500 Received: from eeel.nist.gov (sparky.eeel.nist.gov [129.6.64.163]) by python.org (8.6.12/8.6.12) with SMTP id JAA25239 for ; Thu, 8 Feb 1996 09:34:42 -0500 Received: from acdc.eeel.nist.gov by eeel.nist.gov (4.1/SMI-3.2-del.6) id AA16578; Thu, 8 Feb 96 09:34:14 EST Received: by acdc.eeel.nist.gov (4.1/SMI-3.2-del.5) id AA17418; Thu, 8 Feb 96 09:30:23 EST Date: Thu, 8 Feb 96 09:30:23 EST Message-Id: <9602081430.AA17418@acdc.eeel.nist.gov> From: Michael McLay To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Type Names In-Reply-To: <31191D29.D42@llnl.gov> Sender: owner-matrix-sig@python.org Precedence: bulk Paul. Dubois writes: > I hold these truths to be self-evident, your argument notwithstanding: > a. A PyFloat is a C double > The fact that some of the Python API lets you convert one to a C > float easily doesn't change this. > > b. It is a GOOD THING that you can count on the precision of a PyFloat. > Writing portable code between platforms is easy with Python, since > even on 64 bit machines C doubles are implemented as 64 bit floats. (I > know that isn't for certain but as a practical truth, it is true). The idea of naming the sizes of numbers "double" and "float" in a programming language is analogous to continuing the use the U.S. units of measure. If you're familiar with the units it's not a problem, but consider the task of introducing the convention to someone new to programming. Why does the naming convention have to be obscure? To get a sense of the nonsense of this naming convention, answer the following question. How many gils are in a gallon? (The first questions you need to ask of course is whether I mean an Imperial gallon or a U.S. gallon.) What do you think of the idea of explicitly naming the numeric types to reflect the actual machine size of the number? This would make the size of an object perfectly clear. It would also eliminate all future problems with portability of numbers between machine architectures. It won't break existing code since the old names can still be used as aliases for the new names. Here's all the names that would be needed. float32 float64 (alias is float) float128 int8 int16 int32 (alias for int) int64 int128 The advantages are: 1) It would make the size of a number obvious thereby eliminating one more source of stupid programming errors. 2) It would eliminate one more thing that must be learned and remembered by rote. 3) It will be easy for new users to grasp and remember the convention. Some of the disadvantages are: 1) It's something new. 2) It may require introducing some new keywords. (This probably isn't necessary.) 3) It isn't an "elegant" solution. 4) For individuals who already know what a float is, and who have a talent for remembering minute details, the change may seem gratuitous. While an "elegant", theoretical solution, such as Ada's ability to declare types for numbers of any size, may be possible, an implementation that allowed this would be slow for any size number other than those supported directly in hardware. (With the exception of float128, the type names listed above are close to being universally supported in modern hardware.) Consequently, these are probably the only numbers that will be of interest for number crunching applications. For problems where speed isn't as important and more control over the calculation is needed then the application could use an arbitrary length numbers class and add size constraints as needed. Isn't it time to drop a convention that was introduced when machine words varied considerably from system to system? Michael ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 8 09:41:02 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA25286 for matrix-sig-people; Thu, 8 Feb 1996 09:41:02 -0500 Received: from dsdbqvarsa.er.usgs.GOV (dsdbqvarsa.er.usgs.gov [130.11.51.78]) by python.org (8.6.12/8.6.12) with ESMTP id JAA25282 for ; Thu, 8 Feb 1996 09:41:00 -0500 Received: (jfulton@localhost) by dsdbqvarsa.er.usgs.GOV (Geomail 1.2.2) id OAA13458; Thu, 8 Feb 1996 14:40:39 GMT From: "Jim Fulton" Message-Id: <9602080940.ZM13456@dsdbqvarsa.er.usgs.gov> Date: Thu, 8 Feb 1996 09:40:39 -0500 In-Reply-To: Fredrik Lundh "Re: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience" (Feb 8, 9:04am) References: <9602080804.AA26470@arnold.image.ivab.se> X-Mailer: Z-Mail (3.2.0 06sep94) To: fredrik_lundh@ivab.se, hinsenk@ere.umontreal.ca Subject: Re: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience Cc: matrix-sig@python.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit Sender: owner-matrix-sig@python.org Precedence: bulk On Feb 8, 9:04am, Fredrik Lundh wrote: > Subject: Re: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experien > > > Rather than rush out a beta version to the public, I advocate a > > public alpha release that comes with no promise about stability of > > function names etc. We need all the test users we can get! Why make > > any commitments without anybody forcing us? > > I support this view, if that's any help :-) Me too. Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 8 10:01:59 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA25741 for matrix-sig-people; Thu, 8 Feb 1996 10:01:59 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id KAA25729 for ; Thu, 8 Feb 1996 10:01:37 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id JAA22999 (8.6.11/IDA-1.6); Thu, 8 Feb 1996 09:53:05 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA16361; Thu, 8 Feb 1996 09:53:25 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA19532; Thu, 8 Feb 1996 09:53:24 -0500 Date: Thu, 8 Feb 1996 09:53:24 -0500 Message-Id: <199602081453.JAA19532@cyclone.ERE.UMontreal.CA> To: fredrik_lundh@ivab.se CC: matrix-sig@python.org In-reply-to: <9602080804.AA26470@arnold.image.ivab.se> (message from Fredrik Lundh on Thu, 8 Feb 1996 09:04:22 +0100) Subject: Re: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk BTW: what about work on standard packages?; I'd very much like some linear algebra stuff, for example to invert matrices (but don't know enough about it to contribute). In addition, I'd would like to use I agree that this would be useful, both directly and indirectly by providing a real-life example of a C or Fortran extension. Such extensions should also be considered in the big naming debate. A "everything in methods" strategy won't work, because a module can't define methods for objects implemented in another module. So all linear-algebra (or other application-specific) operations have to be implemented as functions. This suggests that built-in functions of the array module should also be functions unless there is some clear semantic difference. A difficult decision for linear algebra is which library to use. My experience is limited to Fortran libraries, of which I had the best results with LAPACK. There is a C version of LAPACK, which however I have never used. There are also a couple of LA libraries directly designed for C, rather than being translations from Fortran. It would certainly be easiest to use a C library, which eliminates all the Fortran interfacing machine dependencies and doesn't force users to have a Fortran compiler. On the other hand, Fortran libraries are better tested and on some systems (such as SGI) the performance difference is enormous. Any comments? Does anyone have experience with a C library for linear algebra? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 8 10:27:58 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA26291 for matrix-sig-people; Thu, 8 Feb 1996 10:27:58 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id KAA26287 for ; Thu, 8 Feb 1996 10:27:54 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA01429; Thu, 8 Feb 96 10:20:12 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) id KAA20717; Thu, 8 Feb 1996 10:31:39 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199602081531.KAA20717@maigret> Subject: Re: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience To: stoll@atr-sw.atr.co.jp (Perry A. Stoll) Date: Thu, 8 Feb 1996 10:31:38 -0500 (EST) Cc: fredrik_lundh@ivab.se, matrix-sig@python.org In-Reply-To: <199602080922.SAA04171@ciris21.atr-sw.atr.co.jp> from "Perry A. Stoll" at Feb 8, 96 06:22:12 pm X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 660 Sender: owner-matrix-sig@python.org Precedence: bulk > I'm working on something similar. I've got various FFT packages off > the net - I'll try to interface them, but I don't have time right > now. If you are interested in doing it, let me know. Maybe we should start a subsig for linalg, one for fft/image processing? > In the mean time, for those wanting to play take a look at the file > attached at the end. It's not the stuff legends are made of, but it > should work. I did something similar in Python, and then redid it in C. The C version was orders of magnitude faster, not surprisingly. I don't know enough about such things to write really efficient code for this sort of stuff, though. --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 8 12:03:33 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA26873 for matrix-sig-people; Thu, 8 Feb 1996 12:03:33 -0500 Received: from catch.ivab.se (root@catch.ivab.se [193.180.173.22]) by python.org (8.6.12/8.6.12) with ESMTP id MAA26869 for ; Thu, 8 Feb 1996 12:03:27 -0500 Received: from arnold.image.ivab.se (arnold.image.ivab.se [193.180.173.119]) by catch.ivab.se (8.6.12/8.6.12) with SMTP id SAA31831; Thu, 8 Feb 1996 18:02:49 +0100 Received: by arnold.image.ivab.se; (5.65v3.2/1.1.8.2/16Nov95-0104AM) id AA32049; Thu, 8 Feb 1996 18:02:45 +0100 Date: Thu, 8 Feb 1996 18:02:45 +0100 Message-Id: <9602081702.AA32049@arnold.image.ivab.se> From: Fredrik Lundh To: da@maigret.cog.brown.edu Cc: matrix-sig@python.org In-Reply-To: <199602081526.KAA20669@maigret> (da@maigret.cog.brown.edu) Subject: [PYTHON MATRIX-SIG] Re: [PYTHON IMAGE-SIG???] Reply-To: fredrik_lundh@ivab.se Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk > Maybe we should start a subsig for linalg, one for fft/image > processing? Why not an Image SIG, targeted for graphics, publishing, GIS and good ole "scientific image processing" users... I've almost completed the first release of an image handling and processing library for Python. This release focuses on file formats and basic operations (see the attached "preannouncement"). The design is to some extent ready for more serious image processing (including support for 32-bit integer and floating point images), and I would very much like to interface (not merge) this with the matrix package to avoid duplicating stuff like FFTs etc. I've also got ok from my bosses to distribute the library for free use, and to spend some time on promoting it :-) What do you think? Regards/F ==================================================================== Fredrik Lundh | mail to fredrik_lundh@ivab.se IV Image Systems AB | or call +46 13 200100 Teknikringen 9 | or fax to +46 13 214897 583 30 LINKOPING SWEDEN | ==================================================================== -------------------------------------------------------- Preannouncement: The Python Imaging Library, Release 0.1 -------------------------------------------------------- The Python Imaging Library adds an image object to your Python interpreter. You can load image objects from a variety of file formats, and apply a rich set of image operations to them. Features: + Identify and extract image characteristics from a large set of image file formats. + Read and write PPM, BMP, GIF, and JPEG files. Supports progressive reading of all formats through ImageIO objects. + BW, Palette, RGB, RGBA and CMYK pixel formats. + Crop, cut/paste, convert between pixel formats. + Image geometry: resize, rotate, transpose, user defined affine transform. Nearest neighbour only. + Image enhancement: blur, contour, detail, edge enhance, emboss, find edges, smooth, sharpen, and user defined 3x3 and 5x5 integer kernels. User defined point transforms. + Image analysis: histogram, global statistics. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 8 12:10:37 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA26873 for matrix-sig-people; Thu, 8 Feb 1996 12:03:33 -0500 Received: from catch.ivab.se (root@catch.ivab.se [193.180.173.22]) by python.org (8.6.12/8.6.12) with ESMTP id MAA26869 for ; Thu, 8 Feb 1996 12:03:27 -0500 Received: from arnold.image.ivab.se (arnold.image.ivab.se [193.180.173.119]) by catch.ivab.se (8.6.12/8.6.12) with SMTP id SAA31831; Thu, 8 Feb 1996 18:02:49 +0100 Received: by arnold.image.ivab.se; (5.65v3.2/1.1.8.2/16Nov95-0104AM) id AA32049; Thu, 8 Feb 1996 18:02:45 +0100 Date: Thu, 8 Feb 1996 18:02:45 +0100 Message-Id: <9602081702.AA32049@arnold.image.ivab.se> From: Fredrik Lundh To: da@maigret.cog.brown.edu Cc: matrix-sig@python.org In-Reply-To: <199602081526.KAA20669@maigret> (da@maigret.cog.brown.edu) Subject: [PYTHON MATRIX-SIG] Re: [PYTHON IMAGE-SIG???] Reply-To: fredrik_lundh@ivab.se Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk > Maybe we should start a subsig for linalg, one for fft/image > processing? Why not an Image SIG, targeted for graphics, publishing, GIS and good ole "scientific image processing" users... I've almost completed the first release of an image handling and processing library for Python. This release focuses on file formats and basic operations (see the attached "preannouncement"). The design is to some extent ready for more serious image processing (including support for 32-bit integer and floating point images), and I would very much like to interface (not merge) this with the matrix package to avoid duplicating stuff like FFTs etc. I've also got ok from my bosses to distribute the library for free use, and to spend some time on promoting it :-) What do you think? Regards/F ==================================================================== Fredrik Lundh | mail to fredrik_lundh@ivab.se IV Image Systems AB | or call +46 13 200100 Teknikringen 9 | or fax to +46 13 214897 583 30 LINKOPING SWEDEN | ==================================================================== -------------------------------------------------------- Preannouncement: The Python Imaging Library, Release 0.1 -------------------------------------------------------- The Python Imaging Library adds an image object to your Python interpreter. You can load image objects from a variety of file formats, and apply a rich set of image operations to them. Features: + Identify and extract image characteristics from a large set of image file formats. + Read and write PPM, BMP, GIF, and JPEG files. Supports progressive reading of all formats through ImageIO objects. + BW, Palette, RGB, RGBA and CMYK pixel formats. + Crop, cut/paste, convert between pixel formats. + Image geometry: resize, rotate, transpose, user defined affine transform. Nearest neighbour only. + Image enhancement: blur, contour, detail, edge enhance, emboss, find edges, smooth, sharpen, and user defined 3x3 and 5x5 integer kernels. User defined point transforms. + Image analysis: histogram, global statistics. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 8 12:35:29 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA27110 for matrix-sig-people; Thu, 8 Feb 1996 12:35:29 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id MAA27106 for ; Thu, 8 Feb 1996 12:35:24 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id MAA00623 (8.6.11/IDA-1.6); Thu, 8 Feb 1996 12:33:27 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA27535; Thu, 8 Feb 1996 12:33:46 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id MAA02062; Thu, 8 Feb 1996 12:33:45 -0500 Date: Thu, 8 Feb 1996 12:33:45 -0500 Message-Id: <199602081733.MAA02062@cyclone.ERE.UMontreal.CA> To: mclay@eeel.nist.gov CC: matrix-sig@python.org In-reply-to: <9602081430.AA17418@acdc.eeel.nist.gov> (message from Michael McLay on Thu, 8 Feb 96 09:30:23 EST) Subject: Re: [PYTHON MATRIX-SIG] Type Names From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk of measure. If you're familiar with the units it's not a problem, but consider the task of introducing the convention to someone new to programming. Why does the naming convention have to be obscure? To Because we want newcomers to appreciate that we are doing something very complicated. Isn't that the origin of most technical terms? following question. How many gils are in a gallon? (The first questions you need to ask of course is whether I mean an Imperial gallon or a U.S. gallon.) Fortunately Canada has changed to the metric system long before I arrived here! What do you think of the idea of explicitly naming the numeric types to reflect the actual machine size of the number? This would make the size of an object perfectly clear. It would also eliminate all future I have nothing against this, but I wonder where these names would actually be used in Python. For the array constructors, I am very happy with the current system of "type constructors" as implemented in Precision.py. It would be nice to have the same for output instead of typecodes, of course, but we are not really adding all these things as new types to Python. While an "elegant", theoretical solution, such as Ada's ability to declare types for numbers of any size, may be possible, an implementation that allowed this would be slow for any size number other than those supported directly in hardware. (With the exception The current scheme in Precision.py of picking always the next larger size available in hardware seems like a good compromise to me. crunching applications. For problems where speed isn't as important and more control over the calculation is needed then the application could use an arbitrary length numbers class and add size constraints as needed. Arbitrary size floats would indeed be a nice addition to Python... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 8 12:57:40 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA27263 for matrix-sig-people; Thu, 8 Feb 1996 12:57:40 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id MAA27259 for ; Thu, 8 Feb 1996 12:57:36 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA24484; Thu, 8 Feb 96 09:56:55 PST Received: from kristen (localhost) by kristen.llnl.gov (5.x/SMI-SVR4) id AA22849; Thu, 8 Feb 1996 09:56:50 -0800 Message-Id: <311A3962.5818@llnl.gov> Date: Thu, 08 Feb 1996 09:56:50 -0800 From: "Paul. Dubois" Organization: Lawrence Livermore National Laboratory X-Mailer: Mozilla 2.0b5 (X11; I; SunOS 5.4 sun4m) Mime-Version: 1.0 To: fredrik_lundh@ivab.se Cc: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Re: ffts References: <9602081702.AA32049@arnold.image.ivab.se> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk We have an FFT package in Basis which in the course of time will be converted to a Python package. If anybody wants to jump the gun, get the Basis 11.2 distribution from ftp-icf.llnl.gov/pub/basis, and look in /dist/library/fft. Maybe a little FIDL will do it? -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 8 17:16:43 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA29053 for matrix-sig-people; Thu, 8 Feb 1996 17:16:43 -0500 Received: from catch.ivab.se (root@catch.ivab.se [193.180.173.22]) by python.org (8.6.12/8.6.12) with ESMTP id RAA29049 for ; Thu, 8 Feb 1996 17:16:39 -0500 Received: from arnold.image.ivab.se (arnold.image.ivab.se [193.180.173.119]) by catch.ivab.se (8.6.12/8.6.12) with SMTP id XAA32283; Thu, 8 Feb 1996 23:15:55 +0100 Received: by arnold.image.ivab.se; (5.65v3.2/1.1.8.2/16Nov95-0104AM) id AA01433; Thu, 8 Feb 1996 23:15:43 +0100 Date: Thu, 8 Feb 1996 23:15:43 +0100 Message-Id: <9602082215.AA01433@arnold.image.ivab.se> From: Fredrik Lundh To: hinsenk@ere.umontreal.ca Cc: matrix-sig@python.org In-Reply-To: <199602081453.JAA19532@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Subject: Re: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience Reply-To: fredrik_lundh@ivab.se Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk Konrad wrote: > A "everything in methods" strategy won't work, because a module > can't define methods for objects implemented in another module. Try this: class foo: def spam(self): print "spam" bar = foo() def egg(self): print "egg" foo.egg = egg bar.spam() bar.egg() Yes, it's weird :-) /F ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 8 17:32:40 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA29147 for matrix-sig-people; Thu, 8 Feb 1996 17:32:40 -0500 Received: from harfang.CC.UMontreal.CA ([132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id RAA29143 for ; Thu, 8 Feb 1996 17:32:36 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id RAA12645 (8.6.11/IDA-1.6); Thu, 8 Feb 1996 17:30:31 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA03615; Thu, 8 Feb 1996 17:30:51 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA22293; Thu, 8 Feb 1996 17:30:50 -0500 Date: Thu, 8 Feb 1996 17:30:50 -0500 Message-Id: <199602082230.RAA22293@cyclone.ERE.UMontreal.CA> To: fredrik_lundh@ivab.se CC: matrix-sig@python.org In-reply-to: <9602082215.AA01433@arnold.image.ivab.se> (message from Fredrik Lundh on Thu, 8 Feb 1996 23:15:43 +0100) Subject: Re: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk Konrad wrote: > A "everything in methods" strategy won't work, because a module > can't define methods for objects implemented in another module. Try this: class foo: def spam(self): print "spam" ... You are right, I should have specified "C module". For Python classes this is possible, although not a particularly good idea from the point of view of modularity. Anyway, arrays are implemented in C... A possible solution would be to require extensions to use a Python wrapper like UserArray and subclass it before adding new methods. In addition to the loss of efficiency that this causes, it would also make our system more complicated for the casual user. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 8 17:41:56 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA29242 for matrix-sig-people; Thu, 8 Feb 1996 17:41:56 -0500 Received: from catch.ivab.se (root@catch.ivab.se [193.180.173.22]) by python.org (8.6.12/8.6.12) with ESMTP id RAA29238 for ; Thu, 8 Feb 1996 17:41:53 -0500 Received: from arnold.image.ivab.se (arnold.image.ivab.se [193.180.173.119]) by catch.ivab.se (8.6.12/8.6.12) with SMTP id XAA32311; Thu, 8 Feb 1996 23:41:15 +0100 Received: by arnold.image.ivab.se; (5.65v3.2/1.1.8.2/16Nov95-0104AM) id AA01587; Thu, 8 Feb 1996 23:41:09 +0100 Date: Thu, 8 Feb 1996 23:41:09 +0100 Message-Id: <9602082241.AA01587@arnold.image.ivab.se> From: Fredrik Lundh To: hinsenk@ere.umontreal.ca Cc: matrix-sig@python.org In-Reply-To: <199602082230.RAA22293@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA) Subject: Re: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience Reply-To: fredrik_lundh@ivab.se Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk > You are right, I should have specified "C module". For Python > classes this is possible, although not a particularly good idea from > the point of view of modularity. And I completely agree. Finding the perfect balance between methods and functions can sometimes be tricky, though. /F ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 8 18:51:10 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA29694 for matrix-sig-people; Thu, 8 Feb 1996 18:51:10 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id SAA29689 for ; Thu, 8 Feb 1996 18:51:01 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id SAA14492 (8.6.11/IDA-1.6 for ); Thu, 8 Feb 1996 18:49:01 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id SAA20214; Thu, 8 Feb 1996 18:49:22 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id SAA27150; Thu, 8 Feb 1996 18:49:21 -0500 Date: Thu, 8 Feb 1996 18:49:21 -0500 Message-Id: <199602082349.SAA27150@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Final (!?) pretty printer From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk Here it is: the fully functional pretty printer for Python arrays. Unless there are bugs to be fixed or complaints or requests for reasonable new features, I'd like to leave it at this stage. Line wrapping is now implemented and respects the maximum line width whenever possible, but it still puts at least one element on each line. Note that continuation lines are indented by at least 6 characters in such a way as to make sure that their items do *not* align with the main lines. This makes visual parsing easier. And now, as you expected, the code: def arrayToString(a, max_line_width = None, precision = None): if not max_line_width: try: max_line_width = sys.output_line_width except AttributeError: max_line_width = 77 if not precision: try: precision = sys.float_output_precision except AttributeError: precision = 8 if a.contiguous(): data = a.reshape(None) else: data = a.copy().reshape(None) type = a.typecode() items_per_line = a.shape[-1] if type == 'b' or type == '1' or type == 's' or type == 'i' \ or type == 'l': max_str_len = max(len(str(maximum.reduce(data))), len(str(minimum.reduce(data)))) format = '%' + str(max_str_len) + 'd' item_length = max_str_len+1 format_function = lambda x, f = format: _formatInteger(x, f) elif type == 'f' or type == 'd': format, item_length = _floatFormat(data, precision) format_function = lambda x, f = format: _formatFloat(x, f) elif type == 'F' or type == 'D': real_format, real_item_length = _floatFormat(data.real, precision, sign=0) imag_format, imag_item_length = _floatFormat(data.imaginary, precision, sign=1) item_length = real_item_length + imag_item_length + 2 format_function = lambda x, f1 = real_format, f2 = imag_format: \ _formatComplex(x, f1, f2) elif type == 'c': format = '%s' item_length = 1 format_function = lambda x, f = format: _formatCharacter(x, f) else: # give up on 'O' return str(a) final_spaces = (type != 'c') line_width = item_length*items_per_line - final_spaces if line_width > max_line_width: indent = 6 if indent == item_length: indent = 8 items_first = (max_line_width+final_spaces)/item_length if items_first < 1: items_first = 1 items_continuation = (max_line_width+final_spaces-indent)/item_length if items_continuation < 1: items_continuation = 1 line_width = max(item_length*items_first, item_length*items_continuation+indent) - final_spaces number_of_lines = 1 + (items_per_line-items_first + items_continuation-1)/items_continuation line_format = (number_of_lines, items_first, items_continuation, indent, line_width) else: line_format = (1, items_per_line, 0, 0, line_width) return _arrayToString(a, format_function, len(a.shape), line_format)[:-1] def _floatFormat(data, precision, sign = 0): exp_format = 0 non_zero = abs(compress(data.notEqual(0), data)) if len(non_zero) == 0: max_val = 0. min_val = 0. else: max_val = maximum.reduce(non_zero) min_val = minimum.reduce(non_zero) if max_val >= 1.e12 or min_val < 0.0001 or max_val/min_val > 1000.: exp_format = 1 if exp_format: large_exponent = 0 < min_val < 1e-99 or max_val >= 1e100 max_str_len = 8 + precision + large_exponent if sign: format = '%+' else: format = '%' format = format + str(max_str_len) + '.' + str(precision) + 'e' if large_exponent: format = format + '3' item_length = max_str_len + 1 else: format = '%.' + str(precision) + 'f' precision = min(precision, apply(max, tuple(map(lambda x, p=precision, f=format: _digits(x,p,f), data)))) max_str_len = len(str(int(max_val))) + precision + 2 if sign: format = '%#+' else: format = '%#' format = format + str(max_str_len) + '.' + str(precision) + 'f' item_length = max_str_len + 1 return (format, item_length) def _digits(x, precision, format): s = format % x zeros = len(s) while s[zeros-1] == '0': zeros = zeros-1 return precision-len(s)+zeros def _arrayToString(a, format_function, rank, line_format): if rank == 0: return str(a[0]) elif rank == 1: s = '' items = line_format[1] indent = 0 index = 0 for j in range(line_format[0]): s = s + indent * ' ' for i in range(items): s = s + format_function(a[index]) index = index + 1 if index == a.shape[0]: break if s[-1] == ' ': s = s[:-1] s = s + '\n' items = line_format[2] indent = line_format[3] else: s = '' for i in range(a.shape[0]-1): s = s + _arrayToString(a[i], format_function, rank-1, line_format) if rank == 3: s = s + '\n' elif rank > 3: s = s + (rank-3)*(line_format[4]*'-'+'\n') s = s + _arrayToString(a[a.shape[0]-1], format_function, rank-1, line_format) return s def _formatInteger(x, format): return format % x + ' ' def _formatFloat(x, format, strip_zeros = 1): if format[-1] == '3': format = format[:-1] s = format % x third = s[-3] if third == '+' or third == '-': s = s[1:-2] + '0' + s[-2:] elif format[-1] == 'f': s = format % x if strip_zeros: zeros = len(s) while s[zeros-1] == '0': zeros = zeros-1 s = s[:zeros] + (len(s)-zeros)*' ' else: s = format % x return s + ' ' def _formatComplex(x, real_format, imag_format): r = _formatFloat(x.real, real_format)[:-1] i = _formatFloat(x.imag, imag_format, 0)[:-1] spaces = 0 while r[spaces] == ' ': spaces = spaces + 1 r = spaces*' ' + '(' + r[spaces:] if imag_format[-1] == 'f': zeros = len(i) while i[zeros-1] == '0': zeros = zeros-1 i = i[:zeros] + 'j)' + (len(i)-zeros)*' ' else: i = i + 'j)' return r + i + ' ' def _formatCharacter(x, format): return format % x ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Feb 10 16:51:04 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA09663 for matrix-sig-people; Sat, 10 Feb 1996 16:51:04 -0500 Received: from muecke.appl-math.tu-muenchen.de (muecke.appl-math.tu-muenchen.de [129.187.242.1]) by python.org (8.6.12/8.6.12) with ESMTP id QAA09659 for ; Sat, 10 Feb 1996 16:50:59 -0500 Received: from hamster.appl-math.tu-muenchen.de by muecke.appl-math.tu-muenchen.de via ESMTP (940816.SGI.8.6.9/911001.SGI) for id WAA25352; Sat, 10 Feb 1996 22:49:42 +0100 Received: by hamster.appl-math.tu-muenchen.de (940816.SGI.8.6.9/911001.SGI) for matrix-sig@python.org id WAA12001; Sat, 10 Feb 1996 22:49:41 +0100 From: "Thomas Schwaller" Message-Id: <9602102249.ZM11999@hamster.appl-math.tu-muenchen.de> Date: Sat, 10 Feb 1996 22:49:25 +0100 X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] plmodule, openglmodule and delaunaymodule (Version 0.33) Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-matrix-sig@python.org Precedence: bulk Hi all, just uploaded plmodule-0.33.c.gz openglmodule-0.33.c.gz and the experimental: delaunaymodule-0.33.c.gz for computing 3D Delaunay Triangulations. Publish or Perish! :-) Tom ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Feb 11 10:36:59 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA12817 for matrix-sig-people; Sun, 11 Feb 1996 10:36:59 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id KAA12813 for ; Sun, 11 Feb 1996 10:36:36 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id KAA15351 (8.6.11/IDA-1.6); Sun, 11 Feb 1996 10:33:21 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id KAA25655; Sun, 11 Feb 1996 10:33:44 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id KAA15537; Sun, 11 Feb 1996 10:33:43 -0500 Date: Sun, 11 Feb 1996 10:33:43 -0500 Message-Id: <199602111533.KAA15537@cyclone.ERE.UMontreal.CA> To: et@appl-math.tu-muenchen.de CC: matrix-sig@python.org In-reply-to: <9602102037.ZM10034@hamster.appl-math.tu-muenchen.de> (et@appl-math.tu-muenchen.de) Subject: Re: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk The Problem is mostly that all of these packages use their own matrix format Of course; in C it is not at all obvious how to represent a matrix... which is not that efficient. FORTRAN routines are somehow uniform in style for matrix stuff, but I would prefer a new clean module taking the code from there, not just interfacing it. But that means not profiting from the enormous work others have put into existing libraries. I'd much prefer using an existing library which has been tested on many machines and is constantly kept up to date with new machines, new compilers etc. We are much too small a group to do that ourselves, even we were determined to try. How to do a clean exception handling whitout knowing what a procedure does. Sometimes you just get crashes, which are unacceptable, I want an exception (not FORTRAN or other style coding.) I don't see that as a problem. LAPACK, for example, returns an error code indicating what is going wrong. This can be converted into an exception by the interface. Of course libraries that print error messages or even crash on an error should be avoided. The more I think about it, the more I tend to favour LAPACK. It has the advantage of being available in a C version as well as in Fortran, so those who could gain from it (like me ;-) could use the Fortran version, and yet everyone could use the module without needing a Fortran compiler. And, of course, it is a very reliable and complete library. >Why not an Image SIG, targeted for graphics, publishing, GIS and >good ole "scientific image processing" users... I suggest to do that later, at the moment I would like to see us writing code for the matrixmodule in one group. When we have some I agree! ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Feb 12 09:38:17 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA18587 for matrix-sig-people; Mon, 12 Feb 1996 09:38:17 -0500 Received: from alfven.jhuapl.edu (alfven.jhuapl.edu [128.244.146.147]) by python.org (8.6.12/8.6.12) with SMTP id JAA18583 for ; Mon, 12 Feb 1996 09:38:13 -0500 Message-Id: <199602121438.JAA18583@python.org> Received: by alfven.jhuapl.edu (1.38.193.4/16.2) id AA08282; Mon, 12 Feb 1996 09:48:00 -0500 Date: Mon, 12 Feb 1996 09:48:00 -0500 From: Chris Chase S1A To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Type Coercion In-Reply-To: <9602071656.ZM12810@dsdbqvarsa.er.usgs.gov> References: <9602072018.AA17817@baribal.LCS.MIT.EDU.LCS.MIT.EDU> <31191D29.D42@llnl.gov> <9602071656.ZM12810@dsdbqvarsa.er.usgs.gov> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk Regarding the comments on type coercion. I agree with Paul's comments: it is good not to lose precision and to not get an exception. However, I would want the capability to turn on exceptions for those who want to identify unwanted coercions. How could this be made an option similar to capabilities of numerical packages that can turn on/off traps on floating point exceptions? I also agree with those who have voiced opinions to not to impose deadlines on the module release. I would prefer an alpha release with flexibility rather than a beta with fixed names/feature set. I do not think there is a rush on this. Besides, deadlines for this project can not supercede the main responsibilities that James and most of us have in our day jobs. Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Feb 12 14:14:39 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA20489 for matrix-sig-people; Mon, 12 Feb 1996 14:14:39 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id OAA20485 for ; Mon, 12 Feb 1996 14:14:34 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id OAA14906 (8.6.11/IDA-1.6); Mon, 12 Feb 1996 14:11:07 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA22560; Mon, 12 Feb 1996 14:11:34 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA10227; Mon, 12 Feb 1996 14:11:33 -0500 Date: Mon, 12 Feb 1996 14:11:33 -0500 Message-Id: <199602121911.OAA10227@cyclone.ERE.UMontreal.CA> To: chris.chase@jhuapl.edu CC: matrix-sig@python.org In-reply-to: <199602121438.JAA18583@python.org> (message from Chris Chase S1A on Mon, 12 Feb 1996 09:48:00 -0500) Subject: Re: [PYTHON MATRIX-SIG] Type Coercion From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk exception. However, I would want the capability to turn on exceptions for those who want to identify unwanted coercions. How could this be made an option similar to capabilities of numerical packages that can turn on/off traps on floating point exceptions? The problem gets a bit more complicated by the fact that you might want this check to be applied only to part of the code. For example, you might want to do "precision debugging" on your code while using a library that coerces happily between float and double. Such things are best left to a debugger, which already has provisions for doing things on specific code regions and under specific conditions. And Python fortunately has a debugger. For such an approach, the umath module should provide some hook to register a function that is called in case of a float->double coercion. This hook would then be used by an extended debugger. The main question is: who is willing to look at and modify the debugger? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Feb 12 15:09:51 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA20969 for matrix-sig-people; Mon, 12 Feb 1996 15:09:51 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id PAA20965 for ; Mon, 12 Feb 1996 15:09:49 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA27148; Mon, 12 Feb 96 15:08:07 EST Date: Mon, 12 Feb 96 15:08:07 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9602122008.AA27148@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Release 0.34 now available Sender: owner-matrix-sig@python.org Precedence: bulk The major news for this release is that Chris Chase's grammar patches to implement multidimensional slices are included. This means that you can write: a[1:, :-1] and other exciting variations now work. The old keywords All, Reverse, and Slice no longer work in this new version. I should warn people that I rushed out this version because a few people were particularly eager to get there hands on it. I haven't tested this as well as I'd like to, so as usual use at your own risk. I hope to release something more stable by this Friday, so if you're not into the cutting edge... It's reasonably likely that the implementation details of multi-dimensional slices will change before they become an official patch to the python core. Nevertheless, it's very likely that the syntax/semantics of these slices will be exactly as they are in this distribution. Personally I find that my own code looks a lot cleaner without all the Slice(None, -1) stuff in it, thanks Chris! As usual, this release also includes many minor bug fixes. (It also includes a small performance optimization, but its unlikely you'll notice that). Also, as of this release I officially give in to the will of the sig on the type coercion issue. Paul DuBois's comment on the relative performance differences between floats and doubles was probably the straw that broke my camels back. floats are now automatically coerced to doubles, and all the other coercions that you'd expect. I really like Konrad's idea of adding some hooks into the debugger to detect automatic type coercions and I'm looking into that. Also, this release should compile on the Mac (thanks to Steve Spicklemire for making this happen). He's also got a version working under windows, but I didn't have the time to make the necessary changes before this release. Now if I can just get somebody to test this on a BeBox! Patches from 0.33 are in ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumPy.patch-0.33-0.34.gz The whole thing is in ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericPython-0.34.tar.gz Enjoy - Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Feb 12 22:35:10 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id WAA25043 for matrix-sig-people; Mon, 12 Feb 1996 22:35:10 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id WAA25037 for ; Mon, 12 Feb 1996 22:35:04 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA07567; Mon, 12 Feb 96 22:27:17 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) for matrix-sig@python.org id WAA02625; Mon, 12 Feb 1996 22:39:01 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199602130339.WAA02625@maigret> Subject: [PYTHON MATRIX-SIG] RubberIndex To: matrix-sig@python.org Date: Mon, 12 Feb 1996 22:39:01 -0500 (EST) X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 842 Sender: owner-matrix-sig@python.org Precedence: bulk I'm still trying to figure out the ... index. If I understand correctly, it stands for "however many ':' I need depending on the rank of the object I'm indexing, so that the indices I *do* specify are at the end of the index list as opposed to the usual beginning. So, if I have a rank-3 array A, then A[...,0] is the same thing as A[:,:,0] but if B is rank-4, then B[...,0] is the same thing as: B[:,:,:,0] This "stretching" only works from the left: In other words, if '...' is used *after* a non-pseudo index, then it is equivalent to ':', and does not stretch. [should ... be allowed after a non-pseudo index?] If C is a rank-5 array, then C[...,0,...] is the same thing as C[...,0,:] and C[:,:,:,0,:] but NOT: C[:,:,0,:,:] This seems to work with my test cases. Am I missing some key functionality? --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 13 08:49:53 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id IAA28939 for matrix-sig-people; Tue, 13 Feb 1996 08:49:53 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id IAA28935 for ; Tue, 13 Feb 1996 08:49:49 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id IAA08913 (8.6.11/IDA-1.6); Tue, 13 Feb 1996 08:46:42 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id IAA06798; Tue, 13 Feb 1996 08:47:09 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id IAA19640; Tue, 13 Feb 1996 08:47:08 -0500 Date: Tue, 13 Feb 1996 08:47:08 -0500 Message-Id: <199602131347.IAA19640@cyclone.ERE.UMontreal.CA> To: da@maigret.cog.brown.edu CC: matrix-sig@python.org In-reply-to: <199602130339.WAA02625@maigret> (da@maigret.cog.brown.edu) Subject: Re: [PYTHON MATRIX-SIG] RubberIndex From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk This "stretching" only works from the left: In other words, if '...' is used *after* a non-pseudo index, then it is equivalent to ':', and does not stretch. [should ... be allowed after a non-pseudo index?] '...' at the end of an index expression makes no sense, as you could just as well leave it out (a[0, ...] is the same as a[0]). Neither does it make sense to have more than one '...' in an index expression, since it stands for "all remaining axes", which you can use up only once. It would be good if this would lead to an exception. But it does make sense to have '...' between two groups of other indices, i.e. a[1, ..., 0] or a[2, ..., NewAxis]. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 13 11:27:18 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA29930 for matrix-sig-people; Tue, 13 Feb 1996 11:27:18 -0500 Received: from alfven.jhuapl.edu (alfven.jhuapl.edu [128.244.146.147]) by python.org (8.6.12/8.6.12) with SMTP id LAA29926 for ; Tue, 13 Feb 1996 11:27:15 -0500 Message-Id: <199602131627.LAA29926@python.org> Received: by alfven.jhuapl.edu (1.38.193.4/16.2) id AA11347; Tue, 13 Feb 1996 11:36:50 -0500 Date: Tue, 13 Feb 1996 11:36:50 -0500 From: Chris Chase S1A To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] RubberIndex In-Reply-To: <199602131347.IAA19640@cyclone.ERE.UMontreal.CA> References: <199602130339.WAA02625@maigret> <199602131347.IAA19640@cyclone.ERE.UMontreal.CA> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk The implementation should only allow a single '...'. If multiple rubber indexes are used then an exception should be raised. I could change Grammar so that multiple '...' causes a syntax error at compile time. Think of the rubber index as justifying/aligning the index list within the shape of the array. Indexes preceding '...' are left-justified and succeeding indexes are right-justified. Let P1,P2,..,PM be indexes preceding and S1,S2,..,SL be indexes succeeding '...' in an index list. If the shape is D1,D2,D3,..,DN-1,DN for a rank N array, P1,P2,..,PL are left-justified so they are associated with dimensions D1,D2,..,DL. S1,S2,..,SM are right-justified so SM -> DN, SM-1 -> DN-1, etc. Any dimension Di not associated with one of the actual indexes is assumed to be selected with a ":". The rubber index is analogous to negative limits for slices. Just as a negative slice limit (a[0:-2]) is relative to last element in the sequence without knowing the length apriori, the rubber index '...' allows one to specify indexes relative to the end of the shape without knowing the rank apriori. Any psuedo indexes (specified by NewAxis) are placed relative to the Pi, Si indexes next to it. Even when the psuedo index is next to a rubber index there is no ambiguity as it will be placed relative to the actual indexes to its left or right. Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 13 11:51:36 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA30152 for matrix-sig-people; Tue, 13 Feb 1996 11:51:36 -0500 Received: from oasis.unl.edu (oasis.unl.edu [129.93.34.17]) by python.org (8.6.12/8.6.12) with ESMTP id LAA30148 for ; Tue, 13 Feb 1996 11:51:32 -0500 Received: by oasis.unl.edu (950911.SGI.8.6.12.PATCH825/940406.SGI) for matrix-sig@python.org id KAA29944; Tue, 13 Feb 1996 10:49:42 -0600 From: drh@oasis.unl.edu (Doug Heisterkamp) Message-Id: <199602131649.KAA29944@oasis.unl.edu> Subject: [PYTHON MATRIX-SIG] LAPACK module questions To: matrix-sig@python.org Date: Tue, 13 Feb 1996 10:49:41 -0600 (CST) X-Mailer: ELM [version 2.4 PL23] Content-Type: text Content-Length: 2064 Sender: owner-matrix-sig@python.org Precedence: bulk Hi, I have completed a first pass at a C module to access the LAPACK library. I have a few questions as I move on an try to refine it. But first, what I've done is to write a parser that reads the source code from CLAPACK and writes an access call for each function. The name and parameters are the same as in the library with the exception of using the array object for matrices and vectors. And now the questions. What is the proper way to access the data of an array object? I did not pay attention to the abstract object interface discussion on the main group and am not sure how to do it through that interface. Currently I'm just directly accessing it such as (for a double a from array b) a=(double *)((PyArrayObject *)PyArray_ContiguousFromObject(b,PyArray_DOUBLE, 1,0))->data; How should I package this C module with a python module? Should it become a subclass of Matrix or UserArray? Should it be a module with just a bunch of functions in it? Are there other people working on accessing math libraries? It would be nice if we could set up some conventions on function names, calling parameters, and exceptions for some of the common items. That way code that needs, say eigenvalues, could import whichever math library that is on the machine and still use a standard call to get the eigenvectors and values of a matrix. I'm planning on writing a python wrapper for eigenvectors, determinants, and svd. What other functions are commonly used and would be nice to have a clean interface to (as compared to the fortran interface of the lapack library)? If people are interested in the module, I could place it on an ftp server somewhere. It currently uses libraries built from CLapack which are just the f2c conversion of the original fortran source. Elf binaries of the lapack and blas libraries are available at the linux sites (if you need the directory, email me. I have the location written down elsewhere). The source for lapack, clapack, and f2c are available from netlib.att.com. Doug Heisterkamp drh@oasis.unl.edu ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 13 11:52:23 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA30171 for matrix-sig-people; Tue, 13 Feb 1996 11:52:23 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id LAA30167 for ; Tue, 13 Feb 1996 11:52:18 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id LAA18145 (8.6.11/IDA-1.6); Tue, 13 Feb 1996 11:47:38 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id LAA20499; Tue, 13 Feb 1996 11:48:05 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id LAA28672; Tue, 13 Feb 1996 11:48:04 -0500 Date: Tue, 13 Feb 1996 11:48:04 -0500 Message-Id: <199602131648.LAA28672@cyclone.ERE.UMontreal.CA> To: chris.chase@jhuapl.edu CC: matrix-sig@python.org In-reply-to: <199602131627.LAA29926@python.org> (message from Chris Chase S1A on Tue, 13 Feb 1996 11:36:50 -0500) Subject: Re: [PYTHON MATRIX-SIG] RubberIndex From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk The implementation should only allow a single '...'. If multiple rubber indexes are used then an exception should be raised. I could change Grammar so that multiple '...' causes a syntax error at compile time. That is not a good idea, as someone might want to use this syntax with a somewhat different meaning for other sequence objects. It would be better to have an exception raised at runtime. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 13 12:04:59 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA30245 for matrix-sig-people; Tue, 13 Feb 1996 12:04:59 -0500 Received: from alfven.jhuapl.edu (alfven.jhuapl.edu [128.244.146.147]) by python.org (8.6.12/8.6.12) with SMTP id MAA30240 for ; Tue, 13 Feb 1996 12:04:50 -0500 Message-Id: <199602131704.MAA30240@python.org> Received: by alfven.jhuapl.edu (1.38.193.4/16.2) id AA11821; Tue, 13 Feb 1996 12:14:21 -0500 Date: Tue, 13 Feb 1996 12:14:21 -0500 From: Chris Chase S1A To: matrix-sig@python.org Cc: hinsenk@ere.umontreal.ca (Konrad HINSEN) Subject: Re: [PYTHON MATRIX-SIG] RubberIndex In-Reply-To: <199602131347.IAA19640@cyclone.ERE.UMontreal.CA> References: <199602130339.WAA02625@maigret> <199602131347.IAA19640@cyclone.ERE.UMontreal.CA> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk >>>>> "Konrad" == Konrad HINSEN writes: Konrad> '...' at the end of an index expression makes no sense, as you could Konrad> just as well leave it out (a[0, ...] is the same as a[0]). It makes perfect sense. Just because there is another way to accomplish the same thing does not make it bad. In fact, I would prefer to eliminate a[0] in preference of a[0,...]. I do not like the fact that an error is not generated if I use to few indexes. For example, suppose 'a' has shape (2,4,5,6). I want the first element but either I forget that it is rank 4 or incorrectly enter the index: scalar = a[0,0,0] Instead of a scalar, I get an array with shape (6,). This "feature" requires that I check the rank of every parameter passed into a function for the desired rank. If I really want the last dimension as a vector I could specify vector = a[0,0,0,...] vector = a[0,0,0,:] I would prefer an error be produced by a[0,0,0]. An error already occurs when specifying too many indexes, i.e. a[0,0,0,0,0]. The same should occur for too few indexes unless a rubber index "..." is present (although even with a rubber index too indexes is an error). When using IDL, I have found that an error for too few indexes to be very useful in quickly finding mistakes in code. Without this error, I might not find out about the bug until much later in my code and then I might have a difficult time tracing it back to its origin. The only purpose I see for a[0] to be valid is to make it look like a list for indexing so that a[0][1] is valid. I suppose that if we want an array to be a specialized or subclassed list (so that it works wherever a list occurs) then this has to be supported. But if we think of an array as more of a dictionary than a list it should not support this kind of indexing. I could see that it could lead to confusion when we have multi-dimensional arrays of other types of python objects. Suppse 'a' is a rank 2 array containing a subscriptable python object. a[0,0][1] and a[0][0][1] would be valid. However, if 'a' had a rank different than 2 then we could have it produce an error - alerting the user that the object is not what he thought. Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 13 13:34:34 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA30876 for matrix-sig-people; Tue, 13 Feb 1996 13:34:34 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id NAA30872 for ; Tue, 13 Feb 1996 13:34:29 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA12640; Tue, 13 Feb 96 13:32:24 EST Date: Tue, 13 Feb 96 13:32:24 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9602131832.AA12640@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: drh@oasis.unl.edu Cc: matrix-sig@python.org In-Reply-To: <199602131649.KAA29944@oasis.unl.edu> (drh@oasis.unl.edu) Subject: Re: [PYTHON MATRIX-SIG] LAPACK module questions Sender: owner-matrix-sig@python.org Precedence: bulk I'm probably the only one here who can answer these questions (yes I know that documentation for the C API would be nice). First a question. Are you building this on top of Guido's bgen tool? I really think that this is the right way to go. Hopefully bgen can be eventually turned into a nice generic tool for specifying interfaces to any C/FORTRAN library. It already handles a large collection of interface specifications. It has the very nice modular design of one automatic part that tries to extract the interface definition which produces interface definition files that can be easily edited by the user for the cases where it guesses wrong. Adding array objects to bgen would not be a whole lot of work (so many things I'd like to do if I just had a free 10 hours or so). I think that creating a raw C level binding to these libraries is the right thing to do, and then to write a python module that provides a nice friendly interface on top of the raw functions created in the module. The basic point here is that this should be doable without writing any C code. I'd be interested in seeing a copy of what you've done so far. Arrays are not part of the abstract object interface. The right way to write a function that takes in a single argument and passes it on to a C function expecting a 1d array of doubles and returning a 1d array of doubles is as follows: static PyObject *function(PyObject *ignored, PyObject *args) { PyArrayObject *ap, *apr; PyObject *op; int n; TRY(PyArg_ParseTuple(args, "O", &op)); TRY(ap = (PyArrayObject *)PyArray_ContiguousFromObject(op, PyArray_DOUBLE, 1, 1)); n = ap->dimensions[0]; TRY(apr = (PyArrayObject *)PyArray_FromDims(1, &n, PyArray_DOUBLE)); c_function((double *)ap->data, (double *)apr->data, n) Py_DECREF(ap); /* Needed because PyArray_ContiguousFromObject INCREFs it. */ return (PyObject *)apr; } In fact, if there is an error in allocating the return array this will leak memory, but I figure anytime you're getting exceptions during array allocation, enough things are wrong with your system that a little memory leak won't be a problem. Hopefully from the general complexity of this you can see the value in a bgen style approach. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 13 15:17:00 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA31658 for matrix-sig-people; Tue, 13 Feb 1996 15:17:00 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id PAA31653 for ; Tue, 13 Feb 1996 15:16:54 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA09181; Tue, 13 Feb 96 12:15:00 PST Received: by kristen.llnl.gov (5.x/SMI-SVR4) id AA25919; Tue, 13 Feb 1996 12:15:00 -0800 Date: Tue, 13 Feb 1996 12:15:00 -0800 Message-Id: <9602132015.AA25919@kristen.llnl.gov> From: "P. Dubois" To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Naming in arrayobject.c Sender: owner-matrix-sig@python.org Precedence: bulk SIG'ers, Here are some thoughts about the names in arrayobject.c; I am now leaving town until the 20th so that I don't have to listen to the resulting howls of outrage (:->. My philosophy is close to that used in the ISE Eiffel libraries and as described in the book Reusable Software by Meyer. 1. These are ok as is: transpose (ok, imperative verb for something that modifies self) cast (ok, ditto) typecode (ok, noun = attribute) itemsize (ok, noun = attribute) toString toFile choose (this is so weird a function anyway ...) 2. These I think need changing: Current -> Proposed Reason ------------------------------------ byteswap -> byteswapped (returns result, shouldn't be an imperative verb) reshape -> shaped (returns result, shouldn't be an imperative verb) If worried about typos because shape and shaped too similar, consider shaped_as. But not reshape, which definitely sounds like self is being modified. isnonzero -> nonzero_indices (sounds too much like test of not all zeros) This returns a list of indices where x is not zero. (Note: maybe a new function x.is_zero() would be useful to mean true => all elements are 0) copy -> clone copy is an ambiguous word as to whether one is the copier or the copyee. But clone definitely carries more of a feeling that this is going to produce a copy. In Eiffel, x.copy(y) means set x's attributes from y, for example. concat -> append (philosophically equivalent to list.append) If you don't like append, then change it to concatenate; abbreviations are not worth the trouble they cause. contiguous -> is_contiguous This is a query, not a command to make self contiguous 3. These are not clear to me take -> gather m.take(i_list) returns [m[i_list[0]], m[i_list[1]], ...] Gather is a more traditional name but I won't argue this too hard. conjugate -> conjugated Ambiguous as to whether it modifies self. But, you could make the case "conjugate" is a noun, not a verb in which case it is ok. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 13 15:19:12 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA31670 for matrix-sig-people; Tue, 13 Feb 1996 15:19:12 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id PAA31666 for ; Tue, 13 Feb 1996 15:19:08 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id PAA27067 (8.6.11/IDA-1.6); Tue, 13 Feb 1996 15:15:39 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA10934; Tue, 13 Feb 1996 15:16:06 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA11799; Tue, 13 Feb 1996 15:16:05 -0500 Date: Tue, 13 Feb 1996 15:16:05 -0500 Message-Id: <199602132016.PAA11799@cyclone.ERE.UMontreal.CA> To: drh@oasis.unl.edu CC: matrix-sig@python.org In-reply-to: <199602131649.KAA29944@oasis.unl.edu> (drh@oasis.unl.edu) Subject: Re: [PYTHON MATRIX-SIG] LAPACK module questions From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk I have completed a first pass at a C module to access the LAPACK library. I Great! have a few questions as I move on an try to refine it. But first, what I've done is to write a parser that reads the source code from CLAPACK and writes an access call for each function. The name and parameters are the same as in the library with the exception of using the array object for matrices and vectors. Not bad for a low-level access module; some people are familiar with these names and will probably want to use them. I just wonder what you do with the routines for special storage methods (banded, packed symmetric, etc.); they don't make any sense in Python. How should I package this C module with a python module? Should it become a subclass of Matrix or UserArray? Should it be a module with just a bunch of functions in it? I'd say the latter. I have thought a bit about how the current array operations and extensions such as linear algebra can best be combined and used with a consistent access scheme. I have come to the conclusion that the best solution is to have only functions in various modules for all mathematical and structural operations, and use methods only for special stuff like typecode inquiries. This proposal is certainly somewhat surprising for true believers in object-oriented programming, but I'll argue that it is still the best compromise. The alternative in a true object-oriented spirit would be to implement most (if not all) operations as methods on array objects. For extensions like linear algebra, one would define a subclass of arrays with added methods. Someone who wants to use several such extensions, e.g. linear algebra and fft, would then create another class inheriting both from linear-algebra-arrays and fft-arrays. There are three problems with this approach, one specific to Python, and two generic. The Python-specific problem is that one cannot inherit from C types, and that C types cannot inherit from anything. That would make a Python wrapper for all array classes obligatory, with a resulting performance penalty (mostly noticeable for small arrays). One of the generic problems (at least in languages without type checking) is that mixing different array classes causes a lot of headaches. If some library returns an fft-array and someone want to use it as a linear-algebra-array, then it has to be converted first. This would be worst for the standard constructors, like zeros(), whose results could not be used in any derived module. The other generic problem is caused by the relatively many operations that depend symmetrically on more than one object. It doesn't really make sense to consider an addition as an operation on the first number with the second as parameter, and this is basically what causes all the coercion problems for binary operators that Python suffers from. I am not saying that a more sensible approach is impossible (one could for example define addition as an operation on a tuple of numbers), but I am not aware of any language that supports this in a clean way. The best way out in Python is, in my opinion, not to implement any operation as a method. That leads to a clean system without confusion, and Python's module system takes care of name space pollution. It may be old-fashioned, but it works. Are there other people working on accessing math libraries? It would be nice if we could set up some conventions on function names, calling parameters, and exceptions for some of the common items. That way code that needs, say Definitely. Still all I personally need is LAPACK... I'm planning on writing a python wrapper for eigenvectors, determinants, and svd. What other functions are commonly used and would be nice to have a clean interface to (as compared to the fortran interface of the lapack library)? Inverse, generalized inverse, and solution of linear equations. As a second step, generalized eigenvalue problems and linear equations with iterative refinement. If people are interested in the module, I could place it on an ftp server somewhere. It currently uses libraries built from CLapack which are just Yes, please! the f2c conversion of the original fortran source. Elf binaries of the Great, so one should be able to substitute the Fortran library easily. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 13 15:25:38 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA31732 for matrix-sig-people; Tue, 13 Feb 1996 15:25:38 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id PAA31721 for ; Tue, 13 Feb 1996 15:25:33 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id PAA27512 (8.6.11/IDA-1.6); Tue, 13 Feb 1996 15:22:16 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA12289; Tue, 13 Feb 1996 15:22:44 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA12218; Tue, 13 Feb 1996 15:22:43 -0500 Date: Tue, 13 Feb 1996 15:22:43 -0500 Message-Id: <199602132022.PAA12218@cyclone.ERE.UMontreal.CA> To: chris.chase@jhuapl.edu CC: matrix-sig@python.org In-reply-to: <199602131702.MAA00480@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Tue, 13 Feb 1996 12:14:21 -0500) Subject: Re: [PYTHON MATRIX-SIG] RubberIndex From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk In fact, I would prefer to eliminate a[0] in preference of a[0,...]. I do not like the fact that an error is not generated if I use to few indexes. For example, suppose 'a' has shape (2,4,5,6). I want the You certainly have a point, but it also makes sense to keep the usability of arrays as sequence objects. For example, you can now iterate over the first axis of an array with for x in array([[1,2],[3,4]]): print x*x Also, most array functions accept nested lists as well as arrays. I'd like to keep this analogy as far as possible. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 13 15:50:21 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA31954 for matrix-sig-people; Tue, 13 Feb 1996 15:50:21 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id PAA31950 for ; Tue, 13 Feb 1996 15:50:17 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA08428; Tue, 13 Feb 96 15:42:14 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) for matrix-sig@python.org id PAA05052; Tue, 13 Feb 1996 15:54:00 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199602132054.PAA05052@maigret> Subject: Re: [PYTHON MATRIX-SIG] Naming in arrayobject.c To: matrix-sig@python.org Date: Tue, 13 Feb 1996 15:54:00 -0500 (EST) In-Reply-To: <9602132015.AA25919@kristen.llnl.gov> from "P. Dubois" at Feb 13, 96 12:15:00 pm X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 1116 Sender: owner-matrix-sig@python.org Precedence: bulk Mostly, I think Paul's suggestions are clear and good. > copy -> clone > copy is an ambiguous word as to whether one is the copier or the > copyee. But clone definitely carries more of a feeling that this > is going to produce a copy. > In Eiffel, x.copy(y) means set x's attributes from y, for example. This is a little odd since python uses copy generally, I think. Clone is certainly less ambiguous. > 3. These are not clear to me > > take -> gather > m.take(i_list) returns [m[i_list[0]], m[i_list[1]], ...] > Gather is a more traditional name but I won't argue this too hard. Neither is very intuitive, but since the function isn't either... > conjugate -> conjugated > Ambiguous as to whether it modifies self. > But, you could make the case "conjugate" is a noun, not a verb > in which case it is ok. Not really, since my first read was that it was a verb. It's not what was intended which should matter, but what is understood. I'll ask my students tonight what they think of these, and see if I come up with any interesting feedback. --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 13 16:42:23 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA32322 for matrix-sig-people; Tue, 13 Feb 1996 16:42:23 -0500 Received: from oasis.unl.edu (oasis.unl.edu [129.93.34.17]) by python.org (8.6.12/8.6.12) with ESMTP id QAA32318 for ; Tue, 13 Feb 1996 16:42:16 -0500 Received: by oasis.unl.edu (950911.SGI.8.6.12.PATCH825/940406.SGI) for matrix-sig@python.org id PAA00422; Tue, 13 Feb 1996 15:40:08 -0600 From: drh@oasis.unl.edu (Doug Heisterkamp) Message-Id: <199602132140.PAA00422@oasis.unl.edu> Subject: Re: [PYTHON MATRIX-SIG] LAPACK module questions To: matrix-sig@python.org Date: Tue, 13 Feb 1996 15:40:07 -0600 (CST) In-Reply-To: <9602131832.AA12640@baribal.LCS.MIT.EDU.LCS.MIT.EDU> from "James Hugunin" at Feb 13, 96 01:32:24 pm X-Mailer: ELM [version 2.4 PL23] Content-Type: text Content-Length: 2644 Sender: owner-matrix-sig@python.org Precedence: bulk '' > > > I'm probably the only one here who can answer these questions (yes I > know that documentation for the C API would be nice). > > First a question. Are you building this on top of Guido's bgen tool? > I really think that this is the right way to go. Hopefully bgen can > be eventually turned into a nice generic tool for specifying > interfaces to any C/FORTRAN library. It already handles a large > collection of interface specifications. It has the very nice modular > design of one automatic part that tries to extract the interface > definition which produces interface definition files that can be > easily edited by the user for the cases where it guesses wrong. > Adding array objects to bgen would not be a whole lot of work (so many > things I'd like to do if I just had a free 10 hours or so). Unfortunately, I did not use bgen. I'll take a look at to see what it can do. > > I think that creating a raw C level binding to these libraries is the > right thing to do, and then to write a python module that provides a > nice friendly interface on top of the raw functions created in the > module. The basic point here is that this should be doable without > writing any C code. > > I'd be interested in seeing a copy of what you've done so far. I'll place a copy on the ftp server here at unl. I'll make some of the changes from below before placing it there. I'll let the list know when it's placed on the server and it actual location. Doug > > Arrays are not part of the abstract object interface. The right way > to write a function that takes in a single argument and passes it on > to a C function expecting a 1d array of doubles and returning a 1d > array of doubles is as follows: > > static PyObject *function(PyObject *ignored, PyObject *args) { > PyArrayObject *ap, *apr; > PyObject *op; > int n; > > TRY(PyArg_ParseTuple(args, "O", &op)); > TRY(ap = (PyArrayObject *)PyArray_ContiguousFromObject(op, PyArray_DOUBLE, 1, 1)); > > n = ap->dimensions[0]; > > TRY(apr = (PyArrayObject *)PyArray_FromDims(1, &n, PyArray_DOUBLE)); > > c_function((double *)ap->data, (double *)apr->data, n) > > Py_DECREF(ap); /* Needed because PyArray_ContiguousFromObject INCREFs it. */ > > return (PyObject *)apr; > } > > In fact, if there is an error in allocating the return array this will > leak memory, but I figure anytime you're getting exceptions during > array allocation, enough things are wrong with your system that a > little memory leak won't be a problem. > > Hopefully from the general complexity of this you can see the value in > a bgen style approach. > > -Jim > ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 13 17:15:10 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA32522 for matrix-sig-people; Tue, 13 Feb 1996 17:15:10 -0500 Received: from oasis.unl.edu (oasis.unl.edu [129.93.34.17]) by python.org (8.6.12/8.6.12) with ESMTP id RAA32517 for ; Tue, 13 Feb 1996 17:14:58 -0500 Received: by oasis.unl.edu (950911.SGI.8.6.12.PATCH825/940406.SGI) for matrix-sig@python.org id QAA00459; Tue, 13 Feb 1996 16:13:07 -0600 From: drh@oasis.unl.edu (Doug Heisterkamp) Message-Id: <199602132213.QAA00459@oasis.unl.edu> Subject: Re: [PYTHON MATRIX-SIG] LAPACK module questions To: matrix-sig@python.org Date: Tue, 13 Feb 1996 16:13:07 -0600 (CST) In-Reply-To: <3120DBFE.4C9E@llnl.gov> from "Paul. Dubois" at Feb 13, 96 10:44:14 am X-Mailer: ELM [version 2.4 PL23] Content-Type: text Content-Length: 1600 Sender: owner-matrix-sig@python.org Precedence: bulk '' > > We'd be interested in hearing about your parser. We face many tasks like > this. Actually, parser may be a little to generous to describe it. I wrote it as a throwaway program target at f2c version of the Lapack source code. It scans for the first function definition in a file and grabs it. The parameter list is parsed and depending on the parameter type, different information is added to the end of a number of different lists. These lists are then used to generate the C function to access the original function. The first comment after the function definition is placed as the __doc__ attribute of the access function. Of course this is written in python. I made a number of assumption about the parameter list. The main one is that I assume that all parameters which are double or complex are actually pointers to arrays. This was needed since in the C version, both a double and an array of doubles is referred to as double *a, in the function definition. I will have to change the cases where this is the wrong assumption by hand. Until I make the change, the function could still be used, but it would be necessary to place the double in a python array object of a single element and then use this array object in the function call. Similarly, I made the assumption that all integer parameters are not arrays. My goal was to get close and then make changes as needed. Doug > -- > Paul F. Dubois, L-472 (510)-422-5426 > Lawrence Livermore National Laboratory FAX (510)-423-9969 > Livermore, CA 94550 dubois1@llnl.gov > Consulting: PaulDubois@aol.com > ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 13 17:59:24 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA00097 for matrix-sig-people; Tue, 13 Feb 1996 17:59:24 -0500 Received: from oasis.unl.edu (oasis.unl.edu [129.93.34.17]) by python.org (8.6.12/8.6.12) with ESMTP id RAA00093 for ; Tue, 13 Feb 1996 17:59:21 -0500 Received: by oasis.unl.edu (950911.SGI.8.6.12.PATCH825/940406.SGI) for matrix-sig@python.org id QAA00502; Tue, 13 Feb 1996 16:57:21 -0600 From: drh@oasis.unl.edu (Doug Heisterkamp) Message-Id: <199602132257.QAA00502@oasis.unl.edu> Subject: Re: [PYTHON MATRIX-SIG] LAPACK module questions To: matrix-sig@python.org Date: Tue, 13 Feb 1996 16:57:21 -0600 (CST) In-Reply-To: <199602132016.PAA11799@cyclone.ERE.UMontreal.CA> from "Konrad HINSEN" at Feb 13, 96 03:16:05 pm X-Mailer: ELM [version 2.4 PL23] Content-Type: text Content-Length: 2943 Sender: owner-matrix-sig@python.org Precedence: bulk > > Not bad for a low-level access module; some people are familiar with > these names and will probably want to use them. I just wonder what you > do with the routines for special storage methods (banded, packed > symmetric, etc.); they don't make any sense in Python. I assume that the caller knows the structure of the storage methods and packs it into an python array object. I have not used these functions yet (I really just started to use the Lapack library), but they seem to want one or two dimensional arrays with specific information at certain locations. The low-level access module assumes this has already been done for the objects passed to it. A higher level access module would be useful in setting up these structures. > > How should I package this C module with a python module? Should it become > a subclass of Matrix or UserArray? Should it be a module with just a bunch > of functions in it? > > I'd say the latter. I have thought a bit about how the current array > operations and extensions such as linear algebra can best be combined > and used with a consistent access scheme. I have come to the conclusion > that the best solution is to have only functions in various modules > for all mathematical and structural operations, and use methods only > for special stuff like typecode inquiries. > > > The best way out in Python is, in my opinion, not to implement any > operation as a method. That leads to a clean system without confusion, > and Python's module system takes care of name space pollution. It may > be old-fashioned, but it works. I agree. > > Are there other people working on accessing math libraries? It would be nice > if we could set up some conventions on function names, calling parameters, > and exceptions for some of the common items. That way code that needs, say > > Definitely. Still all I personally need is LAPACK... > > I'm planning on writing a python wrapper for eigenvectors, determinants, > and svd. What other functions are commonly used and would be nice to > have a clean interface to (as compared to the fortran interface of the > lapack library)? > > Inverse, generalized inverse, and solution of linear equations. As a > second step, generalized eigenvalue problems and linear equations with > iterative refinement. What is a "generalized inverse"? If I remember correctly, the generalized eigenvalue problem is (A - lambda C)x = 0 where C is a matrix. But I don't remember running across a generalized inverse. > > If people are interested in the module, I could place it on an ftp server > somewhere. It currently uses libraries built from CLapack which are just > > Yes, please! > > the f2c conversion of the original fortran source. Elf binaries of the > > Great, so one should be able to substitute the Fortran library > easily. I hope so as the Fortran libraries should be faster. Doug ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 14 08:48:14 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id IAA04155 for matrix-sig-people; Wed, 14 Feb 1996 08:48:14 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id IAA04151 for ; Wed, 14 Feb 1996 08:48:09 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id IAA13218 (8.6.11/IDA-1.6); Wed, 14 Feb 1996 08:44:44 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id IAA13288; Wed, 14 Feb 1996 08:45:12 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id IAA19287; Wed, 14 Feb 1996 08:45:10 -0500 Date: Wed, 14 Feb 1996 08:45:10 -0500 Message-Id: <199602141345.IAA19287@cyclone.ERE.UMontreal.CA> To: drh@oasis.unl.edu CC: matrix-sig@python.org In-reply-to: <199602132257.QAA00502@oasis.unl.edu> (drh@oasis.unl.edu) Subject: Re: [PYTHON MATRIX-SIG] LAPACK module questions From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk I assume that the caller knows the structure of the storage methods and packs it into an python array object. I have not used these functions Yes, that's a reasonable assumption. After all, it's low level. A good interface can always be added later by defining a Python class representing e.g. a packed symmetric matrix. What is a "generalized inverse"? If I remember correctly, the generalized eigenvalue problem is (A - lambda C)x = 0 where C is a matrix. But I don't remember running across a generalized inverse. It's also known as a pseudoinverse or Moore-Penrose-inverse. There is a uniquely defined generalized inverse for any matrix, and in the case of a square non-singular matrix it is equal to the ordinary inverse. Generalized inverses can be used to express the solution of over- or underdetermined (or both) systems of linear equations conveniently, and they are very useful in least-squares problems. The preferred way to calculate the generalized inverse uses SVD: take the original matrix, do SVD, invert all the non-zero singular values, multiply the three matrices again, and return the transpose. Trivial to code in Python based on low-level SVD calls. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 14 11:46:42 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA05436 for matrix-sig-people; Wed, 14 Feb 1996 11:46:42 -0500 Received: from alfven.jhuapl.edu (alfven.jhuapl.edu [128.244.146.147]) by python.org (8.6.12/8.6.12) with SMTP id LAA05432 for ; Wed, 14 Feb 1996 11:46:38 -0500 Message-Id: <199602141646.LAA05432@python.org> Received: by alfven.jhuapl.edu (1.38.193.4/16.2) id AA14974; Wed, 14 Feb 1996 11:56:02 -0500 Date: Wed, 14 Feb 1996 11:56:02 -0500 From: Chris Chase S1A Cc: hinsenk@ere.umontreal.ca (Konrad HINSEN) To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] RubberIndex In-Reply-To: <199602132022.PAA12218@cyclone.ERE.UMontreal.CA> References: <199602131702.MAA00480@cyclone.ERE.UMontreal.CA> <199602132022.PAA12218@cyclone.ERE.UMontreal.CA> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk >>>>> "Konrad" == Konrad HINSEN writes: Konrad> In fact, I would prefer to eliminate a[0] in preference of Konrad> a[0,...]. I do not like the fact that an error is not Konrad> generated if I use to few indexes. For example, suppose Konrad> 'a' has shape (2,4,5,6). I want the Konrad> You certainly have a point, but it also makes sense to keep Konrad> the usability of arrays as sequence objects. For example, you Konrad> can now iterate over the first axis of an array with Konrad> for x in array([[1,2],[3,4]]): print x*x It would be better to have an iterator method that would allow one to access regular sub-arrays. At the minimum it would allow you to specify iteration along any chosen dimension (i.e. a.iterator(dim)). I am not sure how to extend it to allow iterating over other sub-arrays (e.g. 2x2 sub-matrices). Konrad> Also, most array functions accept nested lists as well as arrays. Konrad> I'd like to keep this analogy as far as possible. Generating an error when using too few indexes when subscripting an array does not prohibit using nested lists as arguments to array functions. Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 14 13:34:40 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA06088 for matrix-sig-people; Wed, 14 Feb 1996 13:34:40 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id NAA06083 for ; Wed, 14 Feb 1996 13:34:34 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id NAA00868 (8.6.11/IDA-1.6); Wed, 14 Feb 1996 13:31:04 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA23186; Wed, 14 Feb 1996 13:31:32 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA04933; Wed, 14 Feb 1996 13:31:31 -0500 Date: Wed, 14 Feb 1996 13:31:31 -0500 Message-Id: <199602141831.NAA04933@cyclone.ERE.UMontreal.CA> To: chris.chase@jhuapl.edu CC: matrix-sig@python.org In-reply-to: <199602141643.LAA28371@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Wed, 14 Feb 1996 11:56:02 -0500) Subject: Re: [PYTHON MATRIX-SIG] RubberIndex From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk It would be better to have an iterator method that would allow one to access regular sub-arrays. At the minimum it would allow you to specify iteration along any chosen dimension (i.e. a.iterator(dim)). I am not sure how to extend it to allow iterating over other sub-arrays (e.g. 2x2 sub-matrices). That should certainly be available, but that was not my point... Konrad> Also, most array functions accept nested lists as well as arrays. Konrad> I'd like to keep this analogy as far as possible. Generating an error when using too few indexes when subscripting an array does not prohibit using nested lists as arguments to array functions. That wasn't my point either. My point is that since arrays and nested lists are treated as interchangeable in many situations, it makes sense to use this analogy as much as possible and allow an indexing scheme for arrays that matches that for nested arrays. WHich means that it should always be allowed to have just one index, independent of the rank of the array. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 14 17:28:26 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA07959 for matrix-sig-people; Wed, 14 Feb 1996 17:28:26 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id RAA07955 for ; Wed, 14 Feb 1996 17:28:21 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA09876; Wed, 14 Feb 96 17:19:57 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) id RAA08591; Wed, 14 Feb 1996 17:30:18 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199602142230.RAA08591@maigret> Subject: Re: [PYTHON MATRIX-SIG] RubberIndex To: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Wed, 14 Feb 1996 17:30:16 -0500 (EST) Cc: chris.chase@jhuapl.edu, matrix-sig@python.org In-Reply-To: <199602141831.NAA04933@cyclone.ERE.UMontreal.CA> from "Konrad HINSEN" at Feb 14, 96 01:31:31 pm X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 614 Sender: owner-matrix-sig@python.org Precedence: bulk > That wasn't my point either. My point is that since arrays and nested > lists are treated as interchangeable in many situations, it makes > sense to use this analogy as much as possible and allow an indexing > scheme for arrays that matches that for nested arrays. WHich means > that it should always be allowed to have just one index, independent > of the rank of the array. This is sort of irrelevant, but it has occured to me, now that I understand and really like [::-1], I was sort of hoping that I could do the same with lists. How easy would it be to extend that third argument to list slices? --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 15 16:28:16 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA15218 for matrix-sig-people; Thu, 15 Feb 1996 16:28:16 -0500 Received: from muecke.appl-math.tu-muenchen.de (muecke.appl-math.tu-muenchen.de [129.187.242.1]) by python.org (8.6.12/8.6.12) with ESMTP id QAA15214 for ; Thu, 15 Feb 1996 16:28:05 -0500 Received: from hamster.appl-math.tu-muenchen.de by muecke.appl-math.tu-muenchen.de via ESMTP (940816.SGI.8.6.9/911001.SGI) for id WAA22195; Thu, 15 Feb 1996 22:25:36 +0100 Received: by hamster.appl-math.tu-muenchen.de (940816.SGI.8.6.9/911001.SGI) for matrix-sig@python.org id WAA29683; Thu, 15 Feb 1996 22:25:36 +0100 From: "Thomas Schwaller" Message-Id: <9602152225.ZM29681@hamster.appl-math.tu-muenchen.de> Date: Thu, 15 Feb 1996 22:25:25 +0100 X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] LAPACK Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-matrix-sig@python.org Precedence: bulk I hope so as the Fortran libraries should be faster. I have the possiblity to test the FORTRAN libraries with a new commercial Fortran Compiler on Linux. Will see! BTW g77 is told to be faster than f2c on Linux e.g. Tom Eagerly waiting for the Lapack/Python module.... Sounds great. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Feb 16 02:59:18 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id CAA18858 for matrix-sig-people; Fri, 16 Feb 1996 02:59:18 -0500 Received: from atr-sw.atr-sw.atr.co.jp (atr-sw.atr-sw.atr.co.jp [133.186.20.117]) by python.org (8.6.12/8.6.12) with ESMTP id CAA18853 for ; Fri, 16 Feb 1996 02:58:49 -0500 Received: from ciris21.atr-sw.atr.co.jp (ciris21.atr-sw.atr.co.jp [133.186.20.41]) by atr-sw.atr-sw.atr.co.jp (8.6.11+2.5Wb2/3.4Wbeta3/95031616ATR-SW) with ESMTP id QAA29191; Fri, 16 Feb 1996 16:55:07 +0900 Received: from localhost by ciris21.atr-sw.atr.co.jp (940816.SGI.8.6.9/6.4J.6) id QAA18871; Fri, 16 Feb 1996 16:55:06 +0900 Message-Id: <199602160755.QAA18871@ciris21.atr-sw.atr.co.jp> To: matrix-sig@python.org cc: jjh@goldilocks.lcs.mit.edu Subject: [PYTHON MATRIX-SIG] changes to compress Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Date: Fri, 16 Feb 1996 16:55:06 +0900 From: "Perry A. Stoll" Sender: owner-matrix-sig@python.org Precedence: bulk Howdy All, Question: How is the function Numeric.compress supposed to behave if none of the elements of m meet the condition? def compress(condition, m, dimension=-1): """compress(condition, x, dimension=-1) = those elements of x corresponding to those elements of condition that are "true". condition must be the same size as the given dimension of x.""" Right now it raises an exception. I think it should return some NULL-type object, either None, [] or maybe an empty array object if such a thing exists. The problem stems from array methods not accepting empty lists as arguments. Couldn't it just return an empty Array object, which would act just like any other empty mapping or sequence object? For now, I've changed compress to: def compress(condition, m, dimension=-1): nonzero = condition.nonzero() if (nonzero): return m.take(nonzero, dimension) else: return [] I noticed this while using Konrad's pretty printer - did anyone else notice this? Konrad? -Perry ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Feb 16 09:27:39 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA20434 for matrix-sig-people; Fri, 16 Feb 1996 09:27:39 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id JAA20426 for ; Fri, 16 Feb 1996 09:27:32 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id JAA05029 (8.6.11/IDA-1.6); Fri, 16 Feb 1996 09:22:32 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA08471; Fri, 16 Feb 1996 09:23:02 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA17508; Fri, 16 Feb 1996 09:23:01 -0500 Date: Fri, 16 Feb 1996 09:23:01 -0500 Message-Id: <199602161423.JAA17508@cyclone.ERE.UMontreal.CA> To: stoll@atr-sw.atr.co.jp CC: matrix-sig@python.org, jjh@goldilocks.lcs.mit.edu In-reply-to: <199602160755.QAA18871@ciris21.atr-sw.atr.co.jp> (stoll@atr-sw.atr.co.jp) Subject: Re: [PYTHON MATRIX-SIG] changes to compress From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk Right now it raises an exception. I think it should return some NULL-type object, either None, [] or maybe an empty array object if such a thing exists. It should definitely return an empty array object. And all the array functions should be able to deal with empty arrays without raising exceptions. As any APL programmer will confirm, empty arrays occur very often in array manipulations. It is even necessary to keep the correct shapes for empty arrays, i.e. an array of shape (0,2) must be distinct from an array of shape (3,0,4), even though both contain no elements. Therefore a generic "empty" object like None can't be used as a substitute. The problem stems from array methods not accepting empty lists as arguments. Couldn't it just return an empty Array object, which would act just like any other empty mapping or sequence object? That's what it should do. I noticed this while using Konrad's pretty printer - did anyone else notice this? Konrad? Yes, I have even pointed it out in one of my messages about the pretty printer. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Feb 16 18:29:41 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA24317 for matrix-sig-people; Fri, 16 Feb 1996 18:29:41 -0500 Received: from oasis.unl.edu (oasis.unl.edu [129.93.34.17]) by python.org (8.6.12/8.6.12) with ESMTP id SAA24313 for ; Fri, 16 Feb 1996 18:29:38 -0500 Received: by oasis.unl.edu (950911.SGI.8.6.12.PATCH825/940406.SGI) for matrix-sig@python.org id RAA20173; Fri, 16 Feb 1996 17:27:03 -0600 From: drh@oasis.unl.edu (Doug Heisterkamp) Message-Id: <199602162327.RAA20173@oasis.unl.edu> Subject: [PYTHON MATRIX-SIG] Python-LAPACK module is available To: matrix-sig@python.org Date: Fri, 16 Feb 1996 17:27:02 -0600 (CST) X-Mailer: ELM [version 2.4 PL23] Content-Type: text Content-Length: 509 Sender: owner-matrix-sig@python.org Precedence: bulk For those who may wish to try my python-LAPACK module, it is now available (finally). The ftp location is: ftp.cs.unl.edu/pub/drh/python Two versions are available. One with and one without the __doc__ string attributes. The one without the __doc__ is about 1/4 the size of the other one. Let me know if it works for your machine. I have built it on an SGI and an Linux box. This the first version so please send me the bug reports so we can make a useful module out of it. Doug drh@oasis.unl.edu ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Feb 18 18:41:26 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA01921 for matrix-sig-people; Sun, 18 Feb 1996 18:41:26 -0500 Received: from muecke.appl-math.tu-muenchen.de (muecke.appl-math.tu-muenchen.de [129.187.242.1]) by python.org (8.6.12/8.6.12) with ESMTP id SAA01917 for ; Sun, 18 Feb 1996 18:40:56 -0500 Received: from hamster.appl-math.tu-muenchen.de by muecke.appl-math.tu-muenchen.de via ESMTP (940816.SGI.8.6.9/911001.SGI) for id AAA18062; Mon, 19 Feb 1996 00:37:51 +0100 Received: by hamster.appl-math.tu-muenchen.de (940816.SGI.8.6.9/911001.SGI) for matrix-sig@python.org id AAA05681; Mon, 19 Feb 1996 00:37:50 +0100 From: "Thomas Schwaller" Message-Id: <9602190037.ZM5679@hamster.appl-math.tu-muenchen.de> Date: Mon, 19 Feb 1996 00:37:40 +0100 X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] trimodule.c Anybody interested? Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-matrix-sig@python.org Precedence: bulk I've written some triangulation modules for Python and promised some people to release them. Most of them use C++-class libraries and are difficult to handle (some compilable only with special compilers), so to make them public is probably not so wise. During the weekend I found some time to write a new one with some code I found usefull for it. I consists of only one module, adding tri trimodule.c to Setup is enough. It is based on a french FEM package and I started to write some documentation for it. It works in 2 dimensions only, but there it seems to be a fine and fast tool. (Handling of local refinements, holes, cracks, subdomains is possible, to name just a few possibilities) As I got no feedback for my delaunaymodule.c, I'll first ask this time. Is anybody interested in this module? If yes, I'll release it next week after writing some more stuff for the documentation which I wrangled out of the Source-Code (It is mostly in French, so I have to do that job for this module to be useful for others). And I'll have to test it a little bit more (perhaps adding some pl-module stuff displaying it or a Postscript output methods, which is not that difficult) Comments? Tom ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 20 12:46:53 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA00437 for matrix-sig-people; Tue, 20 Feb 1996 12:46:53 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id MAA00433 for ; Tue, 20 Feb 1996 12:46:50 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA27251; Tue, 20 Feb 96 11:22:27 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) for matrix-sig@python.org id LAA21501; Tue, 20 Feb 1996 11:34:36 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199602201634.LAA21501@maigret> Subject: [PYTHON MATRIX-SIG] returning scalars or rank-0 arrays.. To: matrix-sig@python.org Date: Tue, 20 Feb 1996 11:34:36 -0500 (EST) X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 909 Sender: owner-matrix-sig@python.org Precedence: bulk I'm working on a 'literate' array class, which labels axes and indices. The point is that one can do slicing based on keyword references to the axes, and labels for index values. I'll of course make it available when it's ready for testing. One issue I've come accross is the "what do I return, a rank-0 entity or a scalar?". My understanding of the current scheme is that indexing returns scalars, while slicing returns ranked entities, even if that rank is 0. Is that correct? >>> a = arange(9).reshape(3,3) >>> b = UserArray(a) >>> b UserArray([[0,1,2],[3,4,5],[6,7,8]], 'l') >>> b[0] UserArray([0,1,2], 'l') >>> b[0,2] 2 >>> b[0,2:3] UserArray([2], 'l') If that's correct, then I guess that's what subclasses of UserArray should do, although it's not exactly intuitive. Also, should there be an _as_scalar method in UserArray which returns a scalar in the cases where self.array is rank-0? -david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 20 15:13:20 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA01352 for matrix-sig-people; Tue, 20 Feb 1996 15:13:20 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id PAA01348 for ; Tue, 20 Feb 1996 15:13:13 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id PAA05847 (8.6.11/IDA-1.6 for ); Tue, 20 Feb 1996 15:12:10 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA10271; Tue, 20 Feb 1996 15:12:09 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA17939; Tue, 20 Feb 1996 15:12:08 -0500 Date: Tue, 20 Feb 1996 15:12:08 -0500 Message-Id: <199602202012.PAA17939@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Functions and names From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk I have made an attempt to put some order into the array functions. I have made a list of functions that should be available and proposed a name for each function. Until now I have covered constructors and structural functions; the rest will follow later, assuming I am not hit by too many 16-ton weights before ;-) Lines that describe new or modified operations are marked with a '+'. Comments are at the end of the file. As I have suggested before, it is useful to have an additional module defining abbreviations and/or more meaningful names for often-used combinations. These should not be part of the standard module to reduce name space pollution and learning overhead, but they should nevertheless be part of the standard distribution to ensure uniformity. I'll include recommended abbreviations in the respective sections. Comments of any kind are of course welcome. Constructors ============ 1) Construct an array from an arbitrary sequence object or a function: + array(sequenceOrFunction, shape=None, type=None) Returns an array of the given shape and type using the elements from the flattened sequence. If not all elements of the sequence are used, the rest is discarded. If more elements are needed, the sequence is reused from the beginning. If a callable object is passed instead of a sequence, it is called for each element with its indices as parameters. Calling the current constructor oldArray(), the new one behaves like def array(sequenceOrFunction, shape=None, type=None): if isSequence(sequenceOrFunction): a = oldArray(sequence, type) if shape is not None: a = copyAndReshape(a, shape) return a else: return fromFunction(map(shape), sequenceOrFunction) This constructor replaces the current constructors array(), copyAndReshape(), and fromFunction(), and makes special cases like zeros() so easy that they don't have to be standard functions any more. 2) Construct a rank-1 array of equally spaced points: + arrayRange(x1, x2=None, x3=None) Currently: function arange() does exactly the same. Abbreviations: zeros(n1,n2,...) equals array(0,(n1,n2,...)) ones(n1,n2,...) equals array(1,(n1,n2,...)) unitMatrix(n) equals array([1]+n*[0], (n,n)) arange equals arrayRange Structural array operations =========================== 1) Selecting subarrays 1.1) Selecting on a "raster" along each coordinate: Done by indexing with integers and slices 1.2) Selecting arbitrary elements specified by an array of indices i: + take(a, i, axis=0) Conditions: i must be of an integer array type, minimum.reduce(ravel(i)) >= 0, maximum.reduce(ravel(i)) < a.shape[axis] Currently: method a.take(i, axis=0), i can be of any type, but must have rank 1. 1.3) Selecting the diagonal, i.e. those values where the indices along two axes are equal (see comment 1): + diagonal(a, axis1=0, axis2=1) Conditions: axis1 < len(a.shape), axis2 < len(a.shape), a.shape[axis1] == a.shape[axis2] 2) Rearranging array elements 2.1) Changing the shape 2.1.1) General reshaping: + reshape(a,shape) shape can be of any non-nested sequence type Condition: multiply.reduce(shape) == multiply.reduce(a.shape) Currently: method a.reshape(shape), shape must be a tuple 2.1.2) Reshaping to a rank-1 array: ravel(a) equivalent to reshape(a,(multiply.reduce(a.shape),)) 2.1.3) Combining a group of axes into one axis (see comment 2): + a[i1, i2, ......, i3, i4] (see comment 3) The double ellipsis works similar to the single one, but contracts all the axes covered into a single axis. 2.1.4) Adding an axis of length 1: a[..., NewAxis, ...] 2.2) Transposition: + transpose(a, axes) axes is a non-nested sequence of non-negative integers with maximum.reduce(axes) < len(a.shape) Currently: method a.transpose(axes) does the same, but there is a bug that sometimes gives wrong results if an axis is used more than once in the list. It also insists that len(axes) == len(a.shape), although there is no need for this restriction. 3) Replicating and combining array elements 3.1) Replicating elements along an axis: + repeat(a, n, axis=0) n is a non-nested integer sequence with len(n) == a.shape[axis] Each element in a is repeated as often as indicated by the corresponding element in n. The length of the result along the specified axis is add.reduce(n). Currently: function compress(n, a, axis=0) is a special case limited to boolean (0/1) elements in n. 3.2) Concatenation of arrays along an axis: + concatenate((a1,a2,a3,...), axis=0) Condition: all arrays must have the same shape for the remaining axes. Currently: method a.concat(a1,a2,a3,...) works only along first axis. 3.3) Concatenation of arrays along a new axis: Can be done by combining concatenate() and indexing with NewAxis. (see comment 4) Abbreviation: reverse(a) = a[::-1] Comments ======== 1) APL uses a special case of transposition for selecting a diagonal, but this is very confusing. 2) In J this is done by using ravel() with reduced rank, but as long as we don't have functions with bounded ranks, we need a special function. 3) I tried to find a construction that does not require yet another syntax and this is the best I could think of. But I am not particularly attached to it, so feel free to think of something better. 4) APL does this with fractional indices, which is probably the weirdest feature in APL. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 20 15:16:49 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA01378 for matrix-sig-people; Tue, 20 Feb 1996 15:16:49 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id PAA01374 for ; Tue, 20 Feb 1996 15:16:45 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id PAA05962 (8.6.11/IDA-1.6); Tue, 20 Feb 1996 15:15:41 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA11111; Tue, 20 Feb 1996 15:15:40 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id PAA18143; Tue, 20 Feb 1996 15:15:39 -0500 Date: Tue, 20 Feb 1996 15:15:39 -0500 Message-Id: <199602202015.PAA18143@cyclone.ERE.UMontreal.CA> To: da@maigret.cog.brown.edu CC: matrix-sig@python.org In-reply-to: <199602201634.LAA21501@maigret> (da@maigret.cog.brown.edu) Subject: Re: [PYTHON MATRIX-SIG] returning scalars or rank-0 arrays.. From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk One issue I've come accross is the "what do I return, a rank-0 entity or a scalar?". My understanding of the current scheme is that indexing returns scalars, while slicing returns ranked entities, even if that rank is 0. Is that correct? I am not sure what the current implementation does, but the idea was that rank-0 arrays should always automatically be converted to scalars. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 20 15:22:22 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA01428 for matrix-sig-people; Tue, 20 Feb 1996 15:22:22 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id PAA01424 for ; Tue, 20 Feb 1996 15:22:20 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA17263; Tue, 20 Feb 96 15:22:09 EST Date: Tue, 20 Feb 96 15:22:09 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9602202022.AA17263@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: da@maigret.cog.brown.edu Cc: matrix-sig@python.org In-Reply-To: <199602201634.LAA21501@maigret> (da@maigret.cog.brown.edu) Subject: Re: [PYTHON MATRIX-SIG] returning scalars or rank-0 arrays.. Sender: owner-matrix-sig@python.org Precedence: bulk From: da@maigret.cog.brown.edu (David Ascher) One issue I've come accross is the "what do I return, a rank-0 entity or a scalar?". My understanding of the current scheme is that indexing returns scalars, while slicing returns ranked entities, even if that rank is 0. Is that correct? >>> a = arange(9).reshape(3,3) >>> b = UserArray(a) >>> b UserArray([[0,1,2],[3,4,5],[6,7,8]], 'l') >>> b[0] UserArray([0,1,2], 'l') >>> b[0,2] 2 >>> b[0,2:3] UserArray([2], 'l') If that's correct, then I guess that's what subclasses of UserArray should do, although it's not exactly intuitive. Also, should there be an _as_scalar method in UserArray which returns a scalar in the cases where self.array is rank-0? Note: UserArray([2], 'l') is a rank 1 array of shape (1,). It is not a rank 0 array. According to the standard python notation, a slice does not reduce the dimensionality of an array, even if it selects only one element. Scalars are ALWAYS returned as python scalars (unless you explictitly ask for array(3.14)). In general, a user should never manage to produce a UserArray or a C array of rank-0, unless they somehow specifically ask for it. Consider the following using standard python lists: >>> b = [1,2,3] >>> b[1:2] [2] >>> b[1] 2 -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 20 15:29:20 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA01477 for matrix-sig-people; Tue, 20 Feb 1996 15:29:20 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id PAA01472 for ; Tue, 20 Feb 1996 15:29:16 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA17288; Tue, 20 Feb 96 15:29:05 EST Date: Tue, 20 Feb 96 15:29:05 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9602202029.AA17288@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: hinsenk@ere.umontreal.ca Cc: matrix-sig@python.org In-Reply-To: <199602202012.PAA17939@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Subject: Re: [PYTHON MATRIX-SIG] Functions and names Sender: owner-matrix-sig@python.org Precedence: bulk I've read both Paul's and Konrad's proposal's for new naming/function schemes. I just want to make it clear that I would have no objections to implementing any part of either of these schemes, but I only intend to go to the effort if I can see that some consensus is developing on this list for one particular proposal and I'm personally going to stay out of most of that discussion. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 20 17:21:41 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA01160 for matrix-sig-people; Tue, 20 Feb 1996 17:21:41 -0500 Received: from alfven.jhuapl.edu (alfven.jhuapl.edu [128.244.146.147]) by python.org (8.6.12/8.6.12) with SMTP id RAA01153 for ; Tue, 20 Feb 1996 17:21:30 -0500 Message-Id: <199602202221.RAA01153@python.org> Received: by alfven.jhuapl.edu (1.38.193.4/16.2) id AA12751; Tue, 20 Feb 1996 17:33:19 -0500 Date: Tue, 20 Feb 1996 17:33:19 -0500 From: Chris Chase S1A To: matrix-sig@python.org Cc: hinsenk@ere.umontreal.ca (Konrad HINSEN) Subject: [PYTHON MATRIX-SIG] Functions and names In-Reply-To: <199602202012.PAA17939@cyclone.ERE.UMontreal.CA> References: <199602202012.PAA17939@cyclone.ERE.UMontreal.CA> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk >>>>> "Konrad" == Konrad HINSEN writes: Konrad> Comments of any kind are of course welcome. I like Konrad's other suggestions along with those already suggested by Paul. Paul seems to have a well thought-out naming convention. I would like to make some comments and additions below: Konrad> Constructors Konrad> ============ Konrad> 1) Construct an array from an arbitrary sequence object or a function: Konrad> + array(sequenceOrFunction, shape=None, type=None) Konrad> Returns an array of the given shape and type using the Konrad> elements from the flattened sequence. I must admit that for constructing matrices the nested list argument seemed natural. This version does not allow it. Otherwise, I like its additional functionality. Konrad> Structural array operations Konrad> =========================== Konrad> 1) Selecting subarrays Support for mapped indexing (like the Tela language) would be useful. Perhaps it should be an attribute so as to allow assignment: print a.mapped[c1,c2,c3] a.mapped[c1,c2,c3] = b c1,c2,c3 index vectors that must have the same shape when converted to an array. Elements of 'a' are chosen by the indexes c1[i],c2[i],c3[i] and the result has the same shape as the shape of the index vectors. Think of the index vectors as being coordinates for some space (I use this style of indexing for paths in two-dimensional image maps or three-dimensional volume data). For assignment 'b' must have the same shape. Konrad> 1.2) Selecting arbitrary elements specified by an array of indices i: Konrad> + take(a, i, axis=0) Konrad> Conditions: i must be of an integer array type, Konrad> minimum.reduce(ravel(i)) >= 0, Konrad> maximum.reduce(ravel(i)) < a.shape[axis] As Paul suggested, perhaps gather() is a better name than take() (gather is a name used in vector libraries). But it really does not matter very much to me. There needs to be a dual capability for assigning the items specified by an array of indices (a completely general form of product indexing). Perhaps something like: a.index[1,i,j] = b with i and j index sequences applied to the second and third dimensions. Here, the 'index' attribute permits index vectors that are scalars, sequences or slices. There is no equivalent to take() since all dimensions must treated simultaneously. Perhaps a better name would be 'subarray'. In fact, such an 'subarray' attribute could also be used for selection, only it would not return a reference but a copy and always return the same rank as 'a' (i.e. does reduce the rank for scalar indexes). If it was only being used for assignment it could be called "scatter" (the dual of gather). An insertion method for inserting a subarray into a larger array would also be useful. Something like: a.insert(b, c1, c2, c3) or a.insert[c1,c2,c3] = b Perhaps someone can come up with a better idea. The concept is: 'b' is inserted into 'a' starting at index c1,c2,c3. The operation overwrites elements of 'a'. It is more convenient then using slice notation which would require one to compute the upper limit on the slices. It could have different behaviors by using a keyword argument for when 'b' does not fit within the dimensions of 'a' (e.g. truncate, wrap around or signal error.) Konrad> 2) Rearranging array elements Konrad> 2.1.2) Reshaping to a rank-1 array: Konrad> ravel(a) Konrad> equivalent to reshape(a,(multiply.reduce(a.shape),)) flatten() could also be a possible name. Why should this be a function rather than a method or attribute? For example, a.flat == ravel(a) but a.flat[s] would allow subscription/assignment of 'a' in flattened form using a scalar, sequence object, or a slice. Of course a combination of take() and ravel() and subscripting can accomplish this. However, flat() and subarray() as methods could completely replace ravel() and take() functions with the ability to simply support flattened or product indexing subscription or assignment. Konrad> 2.1.3) Combining a group of axes into one axis (see comment 2): Konrad> + a[i1, i2, ......, i3, i4] (see comment 3) Konrad> The double ellipsis works similar to the single one, but Konrad> contracts all the axes covered into a single axis. I do not like this. Perhaps an attribute 'collapse' would work? E.g. a.collapse[i1, i2, ..., i3, i4] collapse would interpret '...' to mean contract the axes. Otherwise it behaves exactly like normal subscripting. This is not necessarily a good idea either. Konrad> 2.1.4) Adding an axis of length 1: Konrad> a[..., NewAxis, ...] Of course the "..." are notational and have nothing to do with ellipses indexes. Otherwise, this does not make sense. Konrad> 2.2) Transposition: Konrad> + transpose(a, axes) Konrad> axes is a non-nested sequence of non-negative integers Konrad> with maximum.reduce(axes) < len(a.shape) Yorick has a nice general form of the transpose() function. It allows a variable length argument list where the arguments are cyclic permutations of the indexes. It is actually a little more general. Rather than explain here, a description is available in the Yorick online docs at: ftp://icf.llnl.gov/pub/Yorick/html/yorick_49.html#SEC49 Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Feb 20 17:40:29 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA01322 for matrix-sig-people; Tue, 20 Feb 1996 17:40:29 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id RAA01318 for ; Tue, 20 Feb 1996 17:40:16 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id RAA12973 (8.6.11/IDA-1.6); Tue, 20 Feb 1996 17:38:42 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA04233; Tue, 20 Feb 1996 17:38:42 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA24811; Tue, 20 Feb 1996 17:38:41 -0500 Date: Tue, 20 Feb 1996 17:38:41 -0500 Message-Id: <199602202238.RAA24811@cyclone.ERE.UMontreal.CA> To: chris.chase@jhuapl.edu CC: matrix-sig@python.org In-reply-to: <199602202220.RAA23965@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Tue, 20 Feb 1996 17:33:19 -0500) Subject: Re: [PYTHON MATRIX-SIG] Functions and names From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk I must admit that for constructing matrices the nested list argument seemed natural. This version does not allow it. Otherwise, I like its additional functionality. Of course this is still allowed! That was what I meant with the somewhat cryptic comment that the shape is inferred from the sequence if not explicitly specified. Support for mapped indexing (like the Tela language) would be useful. Perhaps it should be an attribute so as to allow assignment: Could you give an example for an application where this would be useful? Konrad> 1.2) Selecting arbitrary elements specified by an array of indices i: Konrad> + take(a, i, axis=0) Konrad> Conditions: i must be of an integer array type, Konrad> minimum.reduce(ravel(i)) >= 0, Konrad> maximum.reduce(ravel(i)) < a.shape[axis] As Paul suggested, perhaps gather() is a better name than take() (gather is a name used in vector libraries). But it really does not matter very much to me. I explicitly avoided "gather" for two reasons: 1) It is not a particularly clear description of what is happening. 2) What I describe is not the exact equivalent of what you find in most vector libraries, because the index array can have a higher rank than 1. There needs to be a dual capability for assigning the items specified by an array of indices (a completely general form of product indexing). I agree that this would be nice, but if I remember correctly there was some problem with putting this into indexing. An insertion method for inserting a subarray into a larger array would also be useful. Something like: The APL way of doing this is to use what I call repeat() to make room for the insertion and then assign to this part. Since this is not a very frequent operation, I don't think we need a special function for it. flatten() could also be a possible name. Why should this be a function rather than a method or attribute? For example, a.flat == ravel(a) As I pointed out in an earlier message, we should better avoid methods for array operations except for very specialized ones (like byteswap or typecode). There is simply no clean way to decide what should be a method and what a function, and besides all operations coded in Python have to be functions anyway. (This, by the way, is the current distinction: everything implemented in C is a method, everything else is a function. I think we agree that this is the worst possible distinction.) but a.flat[s] would allow subscription/assignment of 'a' in flattened form using a scalar, sequence object, or a slice. Of course a So will a function, if it doesn't copy the array but returns a reference. Konrad> 2.1.3) Combining a group of axes into one axis (see comment 2): Konrad> + a[i1, i2, ......, i3, i4] (see comment 3) Konrad> The double ellipsis works similar to the single one, but Konrad> contracts all the axes covered into a single axis. I do not like this. Perhaps an attribute 'collapse' would work? E.g. a.collapse[i1, i2, ..., i3, i4] If that can be done, fine. So what should the value of "collapse" be before subscripting? I can't see a way to make this work. Konrad> 2.1.4) Adding an axis of length 1: Konrad> a[..., NewAxis, ...] Of course the "..." are notational and have nothing to do with ellipses indexes. Otherwise, this does not make sense. Of course. I hadn't thought about this possible confusion. Yorick has a nice general form of the transpose() function. It allows a variable length argument list where the arguments are cyclic permutations of the indexes. It is actually a little more general. I think this is close to what I wanted to describe. I'll have to look up the Yorick documentation... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 21 11:06:52 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA05953 for matrix-sig-people; Wed, 21 Feb 1996 11:06:52 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id LAA05949 for ; Wed, 21 Feb 1996 11:06:47 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA13825; Wed, 21 Feb 96 11:00:03 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) id LAA24573; Wed, 21 Feb 1996 11:12:15 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199602211612.LAA24573@maigret> Subject: Re: [PYTHON MATRIX-SIG] Functions and names To: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Wed, 21 Feb 1996 11:12:15 -0500 (EST) Cc: chris.chase@jhuapl.edu, matrix-sig@python.org In-Reply-To: <199602202238.RAA24811@cyclone.ERE.UMontreal.CA> from "Konrad HINSEN" at Feb 20, 96 05:38:41 pm X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 506 Sender: owner-matrix-sig@python.org Precedence: bulk > An insertion method for inserting a subarray into a larger array would > also be useful. Something like: > > The APL way of doing this is to use what I call repeat() to make > room for the insertion and then assign to this part. Since this > is not a very frequent operation, I don't think we need a special > function for it. Actually, when I saw Chris' description of that I thought "yeah!", since I can see myself using it quite a bit. But I guess I can always write my own version. --da ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 21 15:01:23 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA00733 for matrix-sig-people; Wed, 21 Feb 1996 15:01:23 -0500 Received: from GS151.SP.CS.CMU.EDU (GS151.SP.CS.CMU.EDU [128.2.203.163]) by python.org (8.6.12/8.6.12) with SMTP id PAA00729 for ; Wed, 21 Feb 1996 15:01:17 -0500 Received: from LOCALHOST by GS151.SP.CS.CMU.EDU id aa03628; 21 Feb 96 12:59 EST To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] MatLab Reply-To: dredish@CS.cmu.edu Date: Wed, 21 Feb 1996 12:59:27 -0500 Message-ID: <3626.824925567@GS151.SP.CS.CMU.EDU> From: David Redish Sender: owner-matrix-sig@python.org Precedence: bulk I've just downloaded the 0.34 Numeric extension (looks really good so far -- compiled out of the box for HPUX), but MLab.py does not seem to work. e.g. >>> import MLab >>> MLab.tri(5) Traceback (innermost last): File "", line 1, in ? File "/users/adr/lib/python/numeric/MLab.py", line 55, in tri m = greaterEqual(subtract.outer(mrange(N), mrange(M)), -k) NameError: greaterEqual I tried adding "From Numeric import *" to MLab.py and then I get >>> import MLab >>> MLab.tri(5) Traceback (innermost last): File "", line 1, in ? File "/users/adr/lib/python/numeric/MLab.py", line 56, in tri m = greaterEqual(subtract.outer(mrange(N), mrange(M)), -k) AttributeError: outer Any help would be appreciated. thanx adr ------------------------------------------------------------ David Redish Computer Science Department CMU graduate student Neural Processes in Cognition Training Program Center for the Neural Basis of Cognition (CNBC) http://www.cs.cmu.edu/Web/People/dredish/home.html ------------------------------------------------------------ maintainer, CNBC website: http://www.cs.cmu.edu/Web/Groups/CNBC maintainer, Cog-Neuro Sites on the Internet, courtesy CNBC: http://www.cs.cmu.edu/Web/Groups/CNBC/other maintainer, NIPS*96 website: http://www.cs.cmu.edu/Web/Groups/NIPS ------------------------------------------------------------ ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 21 15:12:59 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA00848 for matrix-sig-people; Wed, 21 Feb 1996 15:12:59 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id PAA00844 for ; Wed, 21 Feb 1996 15:12:49 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA14377; Wed, 21 Feb 96 15:06:22 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) id PAA25500; Wed, 21 Feb 1996 15:18:32 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199602212018.PAA25500@maigret> Subject: Re: [PYTHON MATRIX-SIG] MatLab To: dredish@cs.cmu.edu Date: Wed, 21 Feb 1996 15:18:31 -0500 (EST) Cc: matrix-sig@python.org In-Reply-To: <3626.824925567@GS151.SP.CS.CMU.EDU> from "David Redish" at Feb 21, 96 12:59:27 pm X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 4056 Sender: owner-matrix-sig@python.org Precedence: bulk > I've just downloaded the 0.34 Numeric extension (looks really good so > far -- compiled out of the box for HPUX), but MLab.py does not seem to > work. I've hacked a version of MLab which mostly works -- no guarantees. --da """Matlab(tm) compatibility functions. This will hopefully become a complete set of the basic functions available in matlab. The syntax is kept as close to the matlab syntax as possible. One fundamental change is that the first index in matlab varies the fastest (as in FORTRAN). That means that it will usually perform reductions over columns, whereas with this object the most natural reductions are over rows. It's perfectly possible to make this work the way it does in matlab if that's desired. """ from Numeric import * # Elementary Matrices # zeros is from matrixmodule in C # ones is from Numeric.py ## This should be replaced by Paul Dubois' URNG very soon now. import Ranf def rand(*args): """rand(d1,...,dn, typecode='d') returns a matrix of the given dimensions which is initialized to random number in the range [0,1). """ return Ranf.random_sample(args) def eye(N, M=None, k=0, typecode=None): """eye(N, M=N, k=0, typecode=None) returns a N-by-M matrix where the k-th diagonal is all ones, and everything else is zeros. """ if M == None: M = N if type(M) == type('d'): typecode = M M = N if (typecode == None): m = outer(subtract, arange(N), arange(M)).equal(-k) else: i = arange(N, typecode=typecode) m = outer(subtract, i, range(M)).equal(-k) return m def tri(N, M=None, k=0, typecode=None): if M == None: M = N if type(M) == type('d'): typecode = M M = N if (typecode == None): m = outer(subtract, arange(N), arange(M)).greaterEqual(-k) else: i = arange(N, typecode=typecode) m = outer(subtract, i, arange(M)).greaterEqual(-k) return m # Matrix manipulation def diag(v, k=0): s = v.shape if len(s)==1: n = s[0]+abs(k) if k > 0: v = v.concat(zeros(k, v.typecode)) elif k < 0: v = zeros(-k, v.typecode).concat(v) return multiply[[1,0]](eye(n, k=k), v) elif len(s)==2: v = add.reduce(eye(s[0], s[1], k=k)*v) if k > 0: return v[:-k] elif k < 0: return v[-k:] else: return v def fliplr(m): return m[:, ::-1] def flipud(m): return m[::-1] # reshape(x, m, n) is not used, instead use reshape(x, (m, n)) def rot90(m, k=1): k = k % 4 if k == 0: return m elif k == 1: return m.transpose()[::-1,::-1] elif k == 2: return fliplr(m)[::-1,::-1] elif k == 3: return fliplr(m.transpose()) def tril(m, k=0): return tri(m.shape[0], m.shape[1], k=k, typecode=m.typecode())*m def triu(m, k=0): return (1-tri(m.shape[0], m.shape[1], k-1, m.typecode()))*m # Data analysis # Basic operations def max(m): return maximum.reduce(m) def min(m): return minimum.reduce(m) # Actually from BASIS, but it fits in so naturally here... def ptp(m): return max(m)-min(m) def mean(m): return add.reduce(m)/len(m) # sort is done in C but is done row-wise rather than column-wise def msort(m): return sort(m.transpose()).transpose() def median(m): return msort(m)[m.shape[0]/2] def std(m): mu = mean(m) return sqrt(add.reduce(pow(m-mu,2)))/sqrt(len(m)-1) def sum(m): return add.reduce(m) def cumsum(m): return add.accumulate(m) def prod(m): return multiply.reduce(m) def cumprod(m): return multiply.accumulate(m) def trapz(y, x=None): """Integrate f using the trapezoidal rule, where y is f(x). """ if x == None: d = 1 else: d = diff(x) return sum(d * (y[1:]+y[0:-1])/2) def diff(x, n=1): """Discrete difference approximation to the derivative """ if n > 1: return diff(x[1:]-x[:-1], n-1) else: return x[1:]-x[:-1] def dot(x, y): return add.reduce(x*y) def corrcoef(x, y=None): """The correlation coefficients """ c = cov(x, y) d = diag(c) return c/sqrt(outer(multiply, d,d)) def cov(m,y=None): if y != None: m = array([m,y], m.typecode()) mu = mean(m) sum_cov = 0.0 for v in m: sum_cov = sum_cov+outer(multiply, v,v) return (sum_cov-len(m)*outer(multiply,mu,mu))/(len(m)-1) ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 21 15:26:44 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA00983 for matrix-sig-people; Wed, 21 Feb 1996 15:26:44 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id PAA00979 for ; Wed, 21 Feb 1996 15:26:35 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA22323; Wed, 21 Feb 96 15:26:29 EST Date: Wed, 21 Feb 96 15:26:29 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9602212026.AA22323@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Version 0.35 Sender: owner-matrix-sig@python.org Precedence: bulk Well, I warned people that v0.34 would be buggy and people did a great job of finding bugs. Thanks to Konrad Hinsen, Perry Stoll and David Ascher in particular, v0.35 is now a little bit more solid. It's in the usual place ftp://sls-ftp.lcs.mit.edu/pub/jjh/patch-0.34-0.35 or ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericPython-0.35.tar.gz Warning: I did something to break the TestSuite fairly recently, so don't expect it to work for you. I thought it was more important to get these bug fixes out to people quickly than to dive into the TestSuite right now. Enjoy - Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 21 18:48:24 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id SAA02608 for matrix-sig-people; Wed, 21 Feb 1996 18:48:24 -0500 Received: from muecke.appl-math.tu-muenchen.de (muecke.appl-math.tu-muenchen.de [129.187.242.1]) by python.org (8.6.12/8.6.12) with ESMTP id SAA02604 for ; Wed, 21 Feb 1996 18:48:12 -0500 Received: from hamster.appl-math.tu-muenchen.de by muecke.appl-math.tu-muenchen.de via ESMTP (940816.SGI.8.6.9/911001.SGI) for id AAA22328; Thu, 22 Feb 1996 00:47:20 +0100 Received: by hamster.appl-math.tu-muenchen.de (940816.SGI.8.6.9/911001.SGI) for matrix-sig@python.org id AAA18770; Thu, 22 Feb 1996 00:47:16 +0100 From: "Thomas Schwaller" Message-Id: <9602220047.ZM18768@hamster.appl-math.tu-muenchen.de> Date: Thu, 22 Feb 1996 00:47:09 +0100 X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Announement: trimodule.tgz Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-matrix-sig@python.org Precedence: bulk Hi all, Here's a first release of trimodule, a standalone module for triangulation of 2-dimensional domains.You can get it at: ftp://ftp.appl-math.tu-muenchen.de/pub/et/trimodule.tgz I also sent it to Jim by mail and it should appear in the usual place at the matrixmodule site. Read trimodule.html (With Netscape it looks nice) to get a first impression, (look especially at the last pictures) Hope you enjoy it. I'm sorry for not responding immediatly to incoming mail, I read them just every second or third day at the moment, so I'm not that reactive! :-) I tried to do my best documenting the stuff I did, but be prepared for crashes when doing your first tests. The Input has to be correct! BTW. The Lapack Module of Doug works very fine. On My Linux box I use it with a static double Lapack library only (also built statically into the Interpreter, not as dynamically loadable module for maximal speed) When using it dynamically the import time is not as good, as usual. The big documentation modules are are a problem. One should put the docstrings in *.py modules and load them when needed, I think that would be the best solution (one *.py documentation module for each function). So we have a small c-module, speed and the documentation) Tom ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 22 12:14:12 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA08513 for matrix-sig-people; Thu, 22 Feb 1996 12:14:12 -0500 Received: from GS151.SP.CS.CMU.EDU (GS151.SP.CS.CMU.EDU [128.2.203.163]) by python.org (8.6.12/8.6.12) with SMTP id MAA08509 for ; Thu, 22 Feb 1996 12:14:07 -0500 Received: from LOCALHOST by GS151.SP.CS.CMU.EDU id aa04506; 22 Feb 96 12:13 EST To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Applying a general function to an array Reply-To: dredish@CS.cmu.edu Date: Thu, 22 Feb 1996 12:13:40 -0500 Message-ID: <4504.825009220@GS151.SP.CS.CMU.EDU> From: David Redish Sender: owner-matrix-sig@python.org Precedence: bulk Is it possible to apply a general python function (that takes one input and produces one output to an array? For example, I'd like to have a python function (implemented in C), "y = f(x)" and then be able to say Y = f(X) and get Y set to an array where each element in Y is f applied to the corresponding element in X. I guess I could get the shape, ravel the array, apply the function using map, convert the list to an array, and reshape it, but that seems like it would be very slow. In a similar vein, is there documentation for 1. the Array C API 2. ofuncs While, I'm at it, I'd like to complement Paul Dubois and David Ascher for their documentation efforts. It's been incredibly helpful. (And of course to Jim and Konrad for all their work putting this together.) adr ------------------------------------------------------------ David Redish Computer Science Department CMU graduate student Neural Processes in Cognition Training Program Center for the Neural Basis of Cognition (CNBC) http://www.cs.cmu.edu/Web/People/dredish/home.html ------------------------------------------------------------ maintainer, CNBC website: http://www.cs.cmu.edu/Web/Groups/CNBC maintainer, Cog-Neuro Sites on the Internet, courtesy CNBC: http://www.cs.cmu.edu/Web/Groups/CNBC/other maintainer, NIPS*96 website: http://www.cs.cmu.edu/Web/Groups/NIPS ------------------------------------------------------------ ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 22 12:35:32 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA08624 for matrix-sig-people; Thu, 22 Feb 1996 12:35:32 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id MAA08620 for ; Thu, 22 Feb 1996 12:35:30 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA28770; Thu, 22 Feb 96 12:35:06 EST Date: Thu, 22 Feb 96 12:35:06 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9602221735.AA28770@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: dredish@CS.cmu.edu Cc: matrix-sig@python.org In-Reply-To: <4504.825009220@GS151.SP.CS.CMU.EDU> (message from David Redish on Thu, 22 Feb 1996 12:13:40 -0500) Subject: Re: [PYTHON MATRIX-SIG] Applying a general function to an array Sender: owner-matrix-sig@python.org Precedence: bulk An example of the sort of function you want to do this with would be very helpful in answering this. Many python functions already work as you describe. ie. (This is from my memory of how RBF NN's work) Say I have a Radial Basis Function Neural Net and I want the user to be able to give a python function the input RBF (actually this would be nice thing to have): The standard RBF function is (from memory, I last worked on these things years ago): def gaussian(x, A, B, C): return A*exp(B*(x-C)**2) This function wil normally just work for any reasonable combination of 1d, 2d arrays and scalars. > In a similar vein, is there documentation for > 1. the Array C API arrayobject.h provides some minimal documentation on this. I'd look at the various modules that people have produced for examples. > 2. ofuncs I doubt that there will be much need for additional ofuncs in the future. They are currently only really useful for binary functions that need to operate over a large variety of array types, and need to be applied in numerous interesting ways (pseudo indices, reduction, etc.) If you have something that you really think should be an ofunc, I'd like to know some more of the details. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 22 13:03:54 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA08820 for matrix-sig-people; Thu, 22 Feb 1996 13:03:54 -0500 Received: from GS151.SP.CS.CMU.EDU (GS151.SP.CS.CMU.EDU [128.2.203.163]) by python.org (8.6.12/8.6.12) with SMTP id NAA08816 for ; Thu, 22 Feb 1996 13:03:51 -0500 Received: from LOCALHOST by GS151.SP.CS.CMU.EDU id aa00370; 22 Feb 96 13:03 EST To: James Hugunin cc: dredish@CS.cmu.edu, matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Applying a general function to an array In-reply-to: Your message of "Thu, 22 Feb 1996 12:35:06 EST." <9602221735.AA28770@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Reply-To: dredish@CS.cmu.edu Date: Thu, 22 Feb 1996 13:02:50 -0500 Message-ID: <368.825012170@GS151.SP.CS.CMU.EDU> From: David Redish Sender: owner-matrix-sig@python.org Precedence: bulk >An example of the sort of function you want to do this with would be >very helpful in answering this. Many python functions already work as >you describe. This is fine for already-coded python functions (like exp), but what I really want to know how to add new C-coded functions. For example, I have a gaussian function coded up in C that takes care of normalization, random-sampling with a gaussian distribution, etc. So I'd like to be able to apply that function to each element of an array. I could recode the function in python, but that would be much slower. Also, it's already nicely coded-up as a C-coded python object and software-reuse is the word here, right? e.g. I can currently do: x = 10 G = Gaussian(wt, mean, variance) y = G(x) A = arange(10) G = Gaussian(wt, mean, variance) Y = G(A) where G is a C-coded python object that defines a __call__ function. thanx adr PS. I am using these for NN's similar to RBFs. (: ------------------------------------------------------------ David Redish Computer Science Department CMU graduate student Neural Processes in Cognition Training Program Center for the Neural Basis of Cognition (CNBC) http://www.cs.cmu.edu/Web/People/dredish/home.html ------------------------------------------------------------ maintainer, CNBC website: http://www.cs.cmu.edu/Web/Groups/CNBC maintainer, Cog-Neuro Sites on the Internet, courtesy CNBC: http://www.cs.cmu.edu/Web/Groups/CNBC/other maintainer, NIPS*96 website: http://www.cs.cmu.edu/Web/Groups/NIPS ------------------------------------------------------------ ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 22 13:25:30 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA09007 for matrix-sig-people; Thu, 22 Feb 1996 13:25:30 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id NAA09003 for ; Thu, 22 Feb 1996 13:25:28 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA29237; Thu, 22 Feb 96 13:25:04 EST Date: Thu, 22 Feb 96 13:25:04 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9602221825.AA29237@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: dredish@CS.cmu.edu Cc: matrix-sig@python.org In-Reply-To: <368.825012170@GS151.SP.CS.CMU.EDU> (message from David Redish on Thu, 22 Feb 1996 13:02:50 -0500) Subject: Re: [PYTHON MATRIX-SIG] Applying a general function to an array Sender: owner-matrix-sig@python.org Precedence: bulk I assume that you have a python function of the form: PyObject *gaussian_call(PyObject *self, PyObject *args) { double x, r; PyArg_ParseTuple("d", &x); r = calculate(self, x); return Py_BuildValue("d", r); } if you convert this to the following, it will work on arbitrary arrays PyObject *gaussian_call(PyObject *self, PyObject *args) { int n, i; PyObject *op; PyArrayObject *ap, *rp; PyArg_ParseTuple("O", &op); ap = PyArray_ContiguousFromObject(op, PyArray_DOUBLE, 0, 0); rp = PyArray_FromDims(ap->nd, ap->dimensions, PyArray_DOUBLE); n = PyArray_Size(ap); for(i=0; idata)[i] = calculate(self, ((double *)ap->data)[i]); } return PyArray_Return(rp); } I admit this looks a little bit more complicated, but this new version will work on any size/shape of array as well as on python scalars. Any other solution will involve converting the elements of the array into and out of python objects, and the overhead of doing this would probably totally swap the computation. I don't know if this is what you're looking for, but I hope it's helpful. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 22 13:26:14 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA09018 for matrix-sig-people; Thu, 22 Feb 1996 13:26:14 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id NAA09014 for ; Thu, 22 Feb 1996 13:26:10 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id NAA12720 (8.6.11/IDA-1.6); Thu, 22 Feb 1996 13:24:30 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA24683; Thu, 22 Feb 1996 13:24:29 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA05399; Thu, 22 Feb 1996 13:24:27 -0500 Date: Thu, 22 Feb 1996 13:24:27 -0500 Message-Id: <199602221824.NAA05399@cyclone.ERE.UMontreal.CA> To: dredish@CS.cmu.edu CC: matrix-sig@python.org In-reply-to: <4504.825009220@GS151.SP.CS.CMU.EDU> (message from David Redish on Thu, 22 Feb 1996 12:13:40 -0500) Subject: Re: [PYTHON MATRIX-SIG] Applying a general function to an array From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk Is it possible to apply a general python function (that takes one input and produces one output to an array? For example, I'd like to have a python function (implemented in C), "y = f(x)" and then be able to say Y = f(X) and get Y set to an array where each element in Y is f applied to the corresponding element in X. There is no real support for this right now. It has to be added, and I am trying to make such things part of my function set proposal (progressingly slowly due to lots of other work). If your function f is written in terms of functions from umath, then it should work automatically, but in the case of a C function or a more complicated Python function (with tests, loops, etc.), you have no other way than to write explicit for-loops or some messy map() application. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Feb 22 16:04:18 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA10318 for matrix-sig-people; Thu, 22 Feb 1996 16:04:18 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id QAA10314 for ; Thu, 22 Feb 1996 16:04:15 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA20094; Thu, 22 Feb 96 13:03:58 PST Received: by kristen.llnl.gov (5.x/SMI-SVR4) id AA11572; Thu, 22 Feb 1996 13:03:58 -0800 Date: Thu, 22 Feb 1996 13:03:58 -0800 Message-Id: <9602222103.AA11572@kristen.llnl.gov> From: "P. Dubois" To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] URNG-2.2 Sender: owner-matrix-sig@python.org Precedence: bulk URNG-2.2 is available at ftp-icf.llnl.gov/pub/basis/URNG-2.2.tar.gz. This fixes an error in memory management. This was the first C extension I wrote. I'd like to describe the error to make sure I have it right this time. I have a routine that allocates a new "urngobject", and I was using it like this: static PyObject * URNG_CreateGenerator(self, args) PyObject *self; /* Not used */ PyObject *args; { int seed; PyObject *result; if (!PyArg_ParseTuple(args, "i", &seed)) return NULL; result = (PyObject *) newurngobject(seed); return result; } I was getting a core dump when the program ended if I created such an object. As a newbie, I assumed I had the reference count stuff messed up, so I tried returning Py_BuildValue("O", result) instead of just result. I stopped dumping core and went on. But this is wrong, of course, since I now have a reference count of two on the object and it is never deleted. I had made the skeleton using modulator and had not checked the "dealloc" option. When I add such a routine: static void urng_dealloc(self) urngobject *self; { PyMem_DEL(self); } and put it in the object's table, all is well. So is this right, you *have* to supply a dealloc method when you write an extension object? If so, maybe modulator shouldn't give you a choice. Or, am I still confused? ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Feb 23 02:37:50 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id CAA13717 for matrix-sig-people; Fri, 23 Feb 1996 02:37:50 -0500 Received: from harfang.CC.UMontreal.CA ([132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id CAA13713 for ; Fri, 23 Feb 1996 02:37:45 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id RAA21727 (8.6.11/IDA-1.6); Thu, 22 Feb 1996 17:30:21 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA23407; Thu, 22 Feb 1996 17:29:41 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA27584; Thu, 22 Feb 1996 17:29:40 -0500 Date: Thu, 22 Feb 1996 17:29:40 -0500 Message-Id: <199602222229.RAA27584@cyclone.ERE.UMontreal.CA> To: dubois@kristen.llnl.gov CC: matrix-sig@python.org In-reply-to: <9602222103.AA11572@kristen.llnl.gov> (dubois@kristen.llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] URNG-2.2 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk So is this right, you *have* to supply a dealloc method when you write an extension object? If so, maybe modulator shouldn't give you a choice. That is also my understanding. It also makes sense, because Python can't know how to deallocate your object. It can only keep track of the reference count and call the dealloc method when it reaches zero. So even if it were allowed not to have a dealloc method, this would mean a permanent memory leak. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Feb 23 17:27:56 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA20191 for matrix-sig-people; Fri, 23 Feb 1996 17:27:56 -0500 Received: from bartel.jhuapl.edu (bartel.jhuapl.edu [128.244.146.143]) by python.org (8.6.12/8.6.12) with SMTP id RAA20187 for ; Fri, 23 Feb 1996 17:27:54 -0500 Message-Id: <199602232227.RAA20187@python.org> Received: by bartel.jhuapl.edu (1.38.193.4/16.2) id AA18709; Fri, 23 Feb 1996 17:34:13 -0500 Date: Fri, 23 Feb 1996 17:34:13 -0500 From: Chris Chase S1A To: matrix-sig@python.org Cc: hinsenk@ere.umontreal.ca (Konrad HINSEN) Subject: Re: [PYTHON MATRIX-SIG] Functions and names In-Reply-To: <199602202238.RAA24811@cyclone.ERE.UMontreal.CA> References: <199602202220.RAA23965@cyclone.ERE.UMontreal.CA> <199602202238.RAA24811@cyclone.ERE.UMontreal.CA> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk I apologize that this response is delayed. My regular work takes priority over my attention to the SIG mailing list. >>>>> "Konrad" == Konrad HINSEN writes: Chase> Support for mapped indexing (like the Tela language) would Chase> be useful. Perhaps it should be an attribute so as to Chase> allow assignment: Konrad> Could you give an example for an application where this would Konrad> be useful? One application where I use this: I often use map projections (e.g. azimuthal) to represent regularly sampled data on a sphere (latitude/longitude or azimuth/elevation coordinates), i.e. data is a 2D array on a regularly sampled lat/lon grid. One way to map data onto the map projection is to compute the lat/lon coordinate of each pixel in the map. Suppose the map is to be an NxM image for a particular projection and that the NxM arrays 'lat' and 'lon' contain the latitude and longitude coordinates of each pixel in map. 'lat' and 'lon' then are converted to indexes according to the grid sampling. Then mapped indexing produces the desired map image: map = data.mapped(lat,lon) As I mentioned previously, I use this kind of indexing when dealing with coordinates for 2D image maps or 3D volume data. The coordinates are often used as part of a transformation (as above) or a path through the data (i.e. a line-of-sight). For more info, mapped indexing is supported in both IDL and Tela. See http://www.geo.fmi.fi/prog/tela/telahelp-5.html#ss5.18. Chase> There needs to be a dual capability for assigning the items Chase> specified by an array of indices (a completely general form Chase> of product indexing). Konrad> I agree that this would be nice, but if I remember correctly Konrad> there was some problem with putting this into indexing. Originally, James Hugunin had implemented general index vectors. However, he stated that they were removed when he decided to have array indexing return by reference. Index vectors do not fit well into this implementation. Returning by reference would require keeping a copy of the index vector and would require a rewrite of the current array representation. Besides, I think it is more natural to return a copy of the indexed elements rather than a reference when using a general index vector. When the operation is assignment there is no reason to not support index vectors since array references are not being created. Just because general product indexing does not completely fit into the current implementation is not a good reason to not support it. Rather than mess around with the current indexing that returns by reference, I suggested a 'subarray' attribute that would support general product indexing for assignment and selection (returning a copy). Chase> An insertion method for inserting a subarray into a larger Chase> array would also be useful. Something like: Konrad> The APL way of doing this is to use what I call repeat() to make Konrad> room for the insertion and then assign to this part. Since this Konrad> is not a very frequent operation, I don't think we need a special Konrad> function for it. I specifically said that the function I was describing overwrites elements in the target array rather than making room. True it is not as frequent of an operation as some others, but it is one that I use other with matrix linear algebra (manipulating block matrices) and for cut-away's and slices in 3D volume data. It is a general operation supported by other languages and not application specific. Konrad> 2.1.3) Combining a group of axes into one axis (see comment 2): Konrad> + a[i1, i2, ......, i3, i4] (see comment 3) Konrad> The double ellipsis works similar to the single one, but Konrad> contracts all the axes covered into a single axis. Chase> I do not like this. Perhaps an attribute 'collapse' would Chase> work? E.g. Chase> a.collapse[i1, i2, ..., i3, i4] Konrad> If that can be done, fine. So what should the value of "collapse" be Konrad> before subscripting? I can't see a way to make this work. It would have to be an object containing a reference to the array 'a'. It would have the special property that the ellipses index, '...', collapses missing dimensions. Otherwise it would act like a normal array. Since this is a special kind of access to the array object, a 'collapse' method would be better, but then it could not support the indexing syntax for slices and ellipses. I personally think that the best solution would be a syntactical symbol like "*", but that idea was shot down. Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Feb 23 19:00:09 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id TAA20625 for matrix-sig-people; Fri, 23 Feb 1996 19:00:09 -0500 Received: from bartel.jhuapl.edu (bartel.jhuapl.edu [128.244.146.143]) by python.org (8.6.12/8.6.12) with SMTP id TAA20621 for ; Fri, 23 Feb 1996 19:00:04 -0500 Message-Id: <199602240000.TAA20621@python.org> Received: by bartel.jhuapl.edu (1.38.193.4/16.2) id AA18848; Fri, 23 Feb 1996 19:06:19 -0500 Date: Fri, 23 Feb 1996 19:06:19 -0500 From: Chris Chase S1A Cc: hinsenk@ere.umontreal.ca (Konrad HINSEN) To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] RubberIndex In-Reply-To: <199602141831.NAA04933@cyclone.ERE.UMontreal.CA> References: <199602141643.LAA28371@cyclone.ERE.UMontreal.CA> <199602141831.NAA04933@cyclone.ERE.UMontreal.CA> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk I made a suggestion about having array indexing generate an error when too few indexes are used. Konrad felt that the analogy to lists was more important: >>>>> "Konrad" == Konrad HINSEN writes: Konrad> That wasn't my point either. My point is that since arrays and nested Konrad> lists are treated as interchangeable in many situations, it makes Konrad> sense to use this analogy as much as possible and allow an indexing Konrad> scheme for arrays that matches that for nested arrays. WHich means Konrad> that it should always be allowed to have just one index, independent Konrad> of the rank of the array. For an array 'a' (rank > 1), the current implementation of a[0] adds no functionality since it is equivalent to a[0,...]. I would like to make another suggestion along the lines my original one that also handles the purpose of ravel() for both assignment and selection. In IDL, Tela and Yorick using a single index or slice defaults to indexing an array in flattened form. Otherwise, the number of actual indexes used must equal the rank. A similar implementation could be supported for the array object. This obviates 'flat' attribute that I suggested earlier. a[i] indexes 'a' in flattened form. 'i' can be a scalar or a slice. Then, a[:] == ravel(a) and assignment to 'a' in flattened form would also be supported: a[i] = 2 which eliminates the need for a "flattened" assignment function or for ravel(). Like other array languages, if more than one index is used without a rubber index and there are fewer indexes than the array rank then an error should result. If 'a' has rank 3, then a[0] = 1 b = a[0,...] b = a[0,0,0] would be valid. But a[0,0] would generate an error. This would be consistent with the fact that a[0,0,0,0] also generates an error. This implementation still allows an array to be used anywhere where a sequence object (e.g. list) is used. Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Feb 24 09:40:55 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA23650 for matrix-sig-people; Sat, 24 Feb 1996 09:40:55 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id JAA23646 for ; Sat, 24 Feb 1996 09:40:52 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id JAA11992 (8.6.11/IDA-1.6); Sat, 24 Feb 1996 09:39:00 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA16785; Sat, 24 Feb 1996 09:38:59 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA20228; Sat, 24 Feb 1996 09:38:58 -0500 Date: Sat, 24 Feb 1996 09:38:58 -0500 Message-Id: <199602241438.JAA20228@cyclone.ERE.UMontreal.CA> To: chris.chase@jhuapl.edu CC: matrix-sig@python.org In-reply-to: <199602232226.RAA24062@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Fri, 23 Feb 1996 17:34:13 -0500) Subject: Re: [PYTHON MATRIX-SIG] Functions and names From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk I often use map projections (e.g. azimuthal) to represent regularly sampled data on a sphere (latitude/longitude or azimuth/elevation coordinates), i.e. data is a 2D array on a regularly sampled lat/lon grid. One way to map data onto the map projection is to compute the lat/lon coordinate of each pixel in the map. Suppose the map is to be an NxM image for a particular projection and that the NxM arrays 'lat' and 'lon' contain the latitude and longitude coordinates of each pixel in map. 'lat' and 'lon' then are converted to indexes according to the grid sampling. Then mapped indexing produces the desired map image: map = data.mapped(lat,lon) I think I understand this now. In fact, I was planning to include this feature in my function proposal, but under "mapping functions" (yet to be written) rather than "indexing", because it is just a special case of array mapping with simple indexing as the function to be applied. current array representation. Besides, I think it is more natural to return a copy of the indexed elements rather than a reference when using a general index vector. Indeed, but this creates the unpleasant situation that an expression like a[i,j] returns either a copy or a reference, depending on the values of i and j. Having references at all is confusing enough, but this would certainly cause some headaches. Personally I'd prefer to have standard indexing always return a copy (that is the safer option) and provide an alternative syntax for obtaining a reference. Then standard indexing could again allow general index vectors. I specifically said that the function I was describing overwrites elements in the target array rather than making room. True it is not Then allowing general index vectors in indexed assignments should be all you need, right? Chase> I do not like this. Perhaps an attribute 'collapse' would Chase> work? E.g. Chase> a.collapse[i1, i2, ..., i3, i4] Konrad> If that can be done, fine. So what should the value of "collapse" be Konrad> before subscripting? I can't see a way to make this work. It would have to be an object containing a reference to the array 'a'. It would have the special property that the ellipses index, '...', collapses missing dimensions. Otherwise it would act like a normal array. Since this is a special kind of access to the array object, a So effectively you want to introduce a second array type with a slightly different behaviour. I don't think this is a good idea. There would be no way to stop users from just writing a.collapse and assign the result to variables, return it from library functions etc. Suddenly there are two array types in free circulation that behave almost, but not quite, identically. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Feb 24 09:52:26 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA23740 for matrix-sig-people; Sat, 24 Feb 1996 09:52:26 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id JAA23736 for ; Sat, 24 Feb 1996 09:52:23 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id JAA12066 (8.6.11/IDA-1.6); Sat, 24 Feb 1996 09:50:41 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA17996; Sat, 24 Feb 1996 09:50:40 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA20452; Sat, 24 Feb 1996 09:50:39 -0500 Date: Sat, 24 Feb 1996 09:50:39 -0500 Message-Id: <199602241450.JAA20452@cyclone.ERE.UMontreal.CA> To: chris.chase@jhuapl.edu CC: matrix-sig@python.org In-reply-to: <199602232358.SAA29109@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Fri, 23 Feb 1996 19:06:19 -0500) Subject: Re: [PYTHON MATRIX-SIG] RubberIndex From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk In IDL, Tela and Yorick using a single index or slice defaults to indexing an array in flattened form. Otherwise, the number of actual indexes used must equal the rank. I won't deny that this has some practical advantages, but it violates the principle that an array is first of all a structured collection of data with a definite shape. Changing this structure should always be an explicit operation, not hidden in a special rule for indexing. For me it still makes more sense to consider an array equivalent to a nested list with the same structure. Like other array languages, if more than one index is used without a rubber index and there are fewer indexes than the array rank then an error should result. I certainly agree on that. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Feb 24 13:43:30 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA24853 for matrix-sig-people; Sat, 24 Feb 1996 13:43:30 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id NAA24849 for ; Sat, 24 Feb 1996 13:43:25 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id NAA14863 (8.6.11/IDA-1.6 for ); Sat, 24 Feb 1996 13:41:41 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA11071; Sat, 24 Feb 1996 13:41:41 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id NAA28353; Sat, 24 Feb 1996 13:41:40 -0500 Date: Sat, 24 Feb 1996 13:41:40 -0500 Message-Id: <199602241841.NAA28353@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Two weeks' experience From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk During the last two weeks I have been very busy with my real work, but most of this has been done in Python, with lots of arrays. I was helping a colleague to locate a bug in his Fortran code that is very important for both of us. Python has been incredibly useful for this task, but I have also discovered a few serious problems, which I would like to report here. My point is mostly to show how important practical applications are to find weak points in the current design and implementation. My first and most frustrating experience was that it is very difficult to write code that works for both arrays and scalars (or rank-0 arrays). Sure, a one-line expression using operators and mathematical formulas is no problem. But as soon as there are conditionals, loops, etc., the easiest way is often to have two separate functions. All this is caused mostly by conditional expressions. To check whether all elements of a are greater than the corresponding elements of b you can write andLogical.reduce(ravel(a.greater(b))). But if a is a scalar, this has to be andLogical.reduce(ravel(b.less(a))), and if both a and b are scalars it must be a > b. I ended up writing a special comparison function that uses one of these expressions depending on the type of the arguments. This shows that no array operations that make sense for rank-0 arrays and scalars (i.e. most) should be implemented as a method of the array type. I had reached this conclusion before on the basis of other arguments, but this practical frustration was much more convincing! Another problem that cost me two hours to solve was matrix multiplication. I had a three-dimensional wave function evaluated on a grid, and therefore stored in the array psi of rank 3. I wanted to apply the operator for kinetic energy in one direction, which is a rank-2 array, t. My first attempt was t.matrixMultiply(psi), and I expected this to do a dot product using the last axis of tx and the first of psi. But no, all I got was an exception. So I had to write this function myself, which I still considered a minor exercise. More than an hour later, I had arrived at the following code: def dot(a1, a2): r1 = len(a1.shape)-1 r2 = len(a2.shape)-1 axes = [r1] + range(r1) a1 = a1.transpose(axes).copy() f = a1.reshape a1 = apply(f, a1.shape + r2*(1,)) a2 = a2.copy() f = a2.reshape a2 = apply(f, a2.shape[0:1] + r1*(1,) + a2.shape[1:]) return add.reduce(a1*a2) If you think that this ought to be easier, I agree. The basic idea of this function is to reshape both arrays to make the axis to be summed over the first, and to add new axes of length one to make things match. All the trouble is caused by two unpleasant features of the method reshape(): 1) It works only on contiguous arrays, which makes it necessary to copy the arguments first. 2) It wants the new shape in the form of several integer arguments, which causes the weird construction used above to obtain a rank that is calculated rather than a known constant. The final operation I want to complain about is fromFunction(). I had always thought that it calls the supplied function once for each element, supplying integer indices as arguments. But no, it tries to be smarter, by calling the function only once with suitably shaped arrays as arguments. That is of course much more efficient, but it limits the function to a straightforward sequence of ofuncs, in which case there is hardly any need to use the fromFunction() constructor. Most cases I can think of where this constructor makes sense involve functions with conditional expressions, like my kinetic energy operator: def tij(i,j): if i == j: return pi**2/3 else: d = i-j return 2.*(-1.)**d/d**2 So instead of the straightforward and readable expression t = (hbar**2/(2.*mass*delta**2)) * fromFunction([ngrid,ngrid], tij) I had to write the very Fortran-like code t = zeros(ngrid,ngrid) for i in range(ngrid): for j in range(ngrid): t[i,j] = tij(i,j) t = (hbar**2/(2.*mass*delta**2)) * t There were a few more minor problems, but I think I can stop now. The direct consequences of the problems described here will appear in my proposal for reorganising the array functions, but most of all I'd like to urge all of you to use what we have for as many applications you can think of. And we should seriously consider a public alpha release to get more testers. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Feb 24 17:38:17 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA26172 for matrix-sig-people; Sat, 24 Feb 1996 17:38:17 -0500 Received: from estel.uindy.edu (ESTEL.UINDY.EDU [192.146.191.197]) by python.org (8.6.12/8.6.12) with SMTP id RAA26168 for ; Sat, 24 Feb 1996 17:38:12 -0500 Received: by estel.uindy.edu (NX5.67d/NX3.0M) id AA26596; Sat, 24 Feb 96 17:40:21 -0500 Date: Sat, 24 Feb 96 17:40:21 -0500 From: Steve Spicklemire Message-Id: <9602242240.AA26596@estel.uindy.edu> To: matrix-sig@python.org Cc: jjh@Goldilocks.LCS.MIT.EDU Subject: [PYTHON MATRIX-SIG] Pretty Printer Sender: owner-matrix-sig@python.org Precedence: bulk I'm probably doing something wrong but I couldn't get PrettyPrinter to work with shape (1,) float arrays. I made this change to mine... (there's probably a much better way to do this... but at least I'm back up and running...) RCS file: PrettyPrinter.py,v retrieving revision 1.1 retrieving revision 1.2 diff -c5 -r1.1 -r1.2 *** /tmp/,RCSt1026586 Sat Feb 24 17:36:04 1996 --- /tmp/,RCSt2026586 Sat Feb 24 17:36:05 1996 *************** *** 86,96 **** format = format + str(max_str_len) + '.' + str(precision) + 'e' if large_exponent: format = format + '3' item_length = max_str_len + 1 else: format = '%.' + str(precision) + 'f' ! precision = min(precision, apply(max, tuple(map(lambda x, p=precision, f=format: _digits(x,p,f), data)))) max_str_len = len(str(int(max_val))) + precision + 2 if sign: format = '%#+' else: format = '%#' --- 86,99 ---- format = format + str(max_str_len) + '.' + str(precision) + 'e' if large_exponent: format = format + '3' item_length = max_str_len + 1 else: format = '%.' + str(precision) + 'f' ! if len(data.shape) == 1 and data.shape[0] == 1: ! precision = min(precision, _digits(data[0],precision,format)) ! else: ! precision = min(precision, apply(max, tuple(map(lambda x, p=precision, f=format: _digits(x,p,f), data)))) max_str_len = len(str(int(max_val))) + precision + 2 if sign: format = '%#+' else: format = '%#' ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Feb 25 09:38:09 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA29809 for matrix-sig-people; Sun, 25 Feb 1996 09:38:09 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id JAA29805 for ; Sun, 25 Feb 1996 09:38:02 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id JAA26443 (8.6.11/IDA-1.6); Sun, 25 Feb 1996 09:36:00 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA24108; Sun, 25 Feb 1996 09:35:59 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id JAA24211; Sun, 25 Feb 1996 09:35:58 -0500 Date: Sun, 25 Feb 1996 09:35:58 -0500 Message-Id: <199602251435.JAA24211@cyclone.ERE.UMontreal.CA> To: steve@estel.uindy.edu CC: matrix-sig@python.org, jjh@Goldilocks.LCS.MIT.EDU In-reply-to: <9602242240.AA26596@estel.uindy.edu> (message from Steve Spicklemire on Sat, 24 Feb 96 17:40:21 -0500) Subject: Re: [PYTHON MATRIX-SIG] Pretty Printer From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk I'm probably doing something wrong but I couldn't get PrettyPrinter to work with shape (1,) float arrays. I made this change to mine... (there's A better patch is precision = min(precision, max(tuple(map(lambda x, p=precision, f=format: _digits(x,p,f), data)))) There seem to be too many versions of the pretty printer around. I had assumed that everyone by now should have the latest one, but appearantly that's not true. I am even more surprised that there are no more serious complaints. When I installed version 0.35 of the numerics package, I would always get error messages when typing "from Numeric import *" due to recursive imports of Numeric and PrettyPrinter. Am I the only one with this problem? Or has everyone silently fixed this? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Feb 25 14:24:22 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA31086 for matrix-sig-people; Sun, 25 Feb 1996 14:24:22 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id OAA31082 for ; Sun, 25 Feb 1996 14:24:17 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id OAA02575 (8.6.11/IDA-1.6 for ); Sun, 25 Feb 1996 14:22:19 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA24739; Sun, 25 Feb 1996 14:22:18 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id OAA02089; Sun, 25 Feb 1996 14:22:17 -0500 Date: Sun, 25 Feb 1996 14:22:17 -0500 Message-Id: <199602251922.OAA02089@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Updated function list From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk Thanks to a rainy weekend, I have completed the proposed function list. It now covers all operations that I can think of (which doesn't mean that nothing is missing!). The descriptions of array() and reshape() have changed with respect to the first draft. Enjoy and ciriticize! --------------------------------------------------------------------------- This list is an attempt to put some order into the array functions. I have made a list of functions that should be available and proposed a name for each function. Lines that describe new or modified operations are marked with a '+'. Comments are at the end of the file. As I have suggested before, it is useful to have an additional module defining abbreviations and/or more meaningful names for often-used combinations. These should not be part of the standard module to reduce name space pollution and learning overhead, but they should nevertheless be part of the standard distribution to ensure uniformity. I'll include recommended abbreviations in the respective sections. The following principles have been applied: 1) Every operation that is not closely related to the implementation details of the array object is implemented as a function, not as a method. This is necessary because methods don't work on scalars, which also serve as rank-0 arrays. It is also necessary to allow the implementation of every operation in C or Python without changing the interface. And this decision also ensures consistency with additional modules (linear algebra etc.); such modules can only add functions, not methods. 2) There are no functions with a variable number of array arguments. Such functions are difficult to apply to a number of arrays that is calculated in the code rather than constant. Instead, a sequence of arrays is passed. This has the added advantage that a higher-dimensional array can be passed, implying an iteration over the first axis. Constructors ============ 1) Construct an array from an arbitrary sequence object or a function: + array(object, shape=None, type=None) Creates an array from the given object with a given shape and type. If the shape and/or the type are not specified, they are inferred from the data. The object providing the data can be of the following types: 1) An object with a method "toArray". This method is called, and the resulting array is converted to the given shape and type (if not None). 2) A nested list or tuple, or an array. The shape is inferred from the nesting, the type from the data, unless space and/or type are specified. 3) A file object. The file is read as a rank-2 array. If a shape and/or type is specified, the result is changed to that shape/type. 4) A callable object. This object is called once for each array element with the indices as arguments. In reshaping to an explicitly given shape, the input is reused from the beginning if necessary. It is also allowed not to use all the input data. This constructor replaces the current constructors array(), copyAndReshape(), and fromFunction(), and makes special cases like zeros() so easy that they don't have to be standard functions any more. Note: the current fromFunction() behaves in a subtly different way: it calls the function only once with index arrays as arguments. This is equivalent if the function consists only of operators and ofuncs, and of course much more efficient, but for most interesting functions it doesn't work at all. In this case I prefer generality to efficiency. 2) Construct a rank-1 array of equally spaced points: + arrayRange(x1, x2=None, x3=None) Currently: function arange() does exactly the same. Abbreviations: zeros(n1,n2,...) equals array(0,(n1,n2,...)) ones(n1,n2,...) equals array(1,(n1,n2,...)) unitMatrix(n) equals array([1]+n*[0], (n,n)) arange equals arrayRange Structural array operations =========================== 1) Selecting subarrays 1.1) Selecting on a "raster" along each coordinate: Done by indexing with integers and slices 1.2) Selecting arbitrary elements specified by an array of indices i: + take(a, i, axis=0) Conditions: i must be of an integer array type, minimum.reduce(ravel(i)) >= 0, maximum.reduce(ravel(i)) < a.shape[axis] Currently: method a.take(i, axis=0), i can be of any type, but must have rank 1. 1.3) Selecting the diagonal, i.e. those values where the indices along two axes are equal (see comment 1): + diagonal(a, axis1=0, axis2=1) Conditions: axis1 < len(a.shape), axis2 < len(a.shape), a.shape[axis1] == a.shape[axis2] 2) Rearranging array elements 2.1) Changing the shape 2.1.1) General reshaping: + reshape(a,shape, copy=1) shape can be of any non-nested sequence type Condition: multiply.reduce(shape) == multiply.reduce(a.shape) If copy==1, the returned array is a new copy, otherwise a reference to a. Currently: method a.reshape(length1, length2, ...) This syntax does not provide an easy way to specify a non-constant shape. Also, the current behaviour is like copy=0, which can lead to unpleasant surprises (since otherwise only indexing returns references) and limits reshaping to contiguous arrays. 2.1.2) Reshaping to a rank-1 array: + ravel(a, copy=1) equivalent to reshape(a,(multiply.reduce(a.shape),), copy) 2.1.3) Combining a group of axes into one axis (see comment 2): + a[i1, i2, ......, i3, i4] (see comment 3) The double ellipsis works similar to the single one, but contracts all the axes covered into a single axis. 2.1.4) Adding an axis of length 1: a[..., NewAxis, ...] 2.2) Transposition: + transpose(a, axes) axes is a non-nested sequence of non-negative integers with maximum.reduce(axes) < len(a.shape) Currently: method a.transpose(axes) does the same, but there is a bug that sometimes gives wrong results if an axis is used more than once in the list. It also insists that len(axes) == len(a.shape), although there is no need for this restriction. 3) Replicating and combining array elements 3.1) Replicating elements along an axis: + repeat(a, n, axis=0) n is a non-nested integer sequence with len(n) == a.shape[axis] Each element in a is repeated as often as indicated by the corresponding element in n. The length of the result along the specified axis is add.reduce(n). Currently: function compress(n, a, axis=0) is a special case limited to boolean (0/1) elements in n. 3.2) Concatenation of arrays along an axis: + concatenate((a1,a2,a3,...), axis=0) Condition: all arrays must have the same shape for the remaining axes. Currently: method a.concat(a1,a2,a3,...) works only along first axis and is difficult to apply to a variable number of arguments. 3.3) Concatenation of arrays along a new axis: Can be done by combining concatenate() and indexing with NewAxis. (see comment 4) Abbreviation: reverse(a) = a[::-1] Arithmetic and logical operations ================================= 1) Binary elementwise arithmetic operators + - * / % ** as functions: add, subtract, multiply, divide, remainder, power 2) Standard "mathematical" functions arccos, arccosh, arcsin, arcsinh, arctan, arctanh, cos, cosh, exp, log, log10, sin, sinh, sqrt, tan, tanh 3) Other elementwise arithmetic functions maximum, minimum, clip + conjugate Currently, conjugate() is implemented as a method both on complex objects and array objects. It should be available as a function, because then it can be applied to other number objects as well. There are many algorithms that work for both real and complex numbers providing that conjugates are used here and there. It ought to be possible to implement these algorithms and make them work for float and complex objects as well as array objects. The method we have now doesn't work on real and integer objects. 4) Comparison functions + equal(a,b), notEqual(a,b), greater(a,b), greaterEqual(a,b), + less(a,b), lessEqual(a,b) Currently all these are methods. Apart from introducing an arbitrary asymmetry, this makes it impossible to write code that works both for scalars and arrays. 5) Logical and boolean operators + logicalAnd(a,b), logicalOr(a,b), logicalXor(a,b), logicalNot(a), + booleanAnd(a,b), booleanOr(a,b), booleanXor(a,b), booleanNot(a,b) Currently there are equivalent methods with "inversed" names. I think these names are more intuitive. About the problem of methods, see above. For some strange reason there is currently no logical xor operation. 6) Non-elementwise operations 6.1) Attribute functions of binary operator functions (see 1 and 5) reduce, accumulate, + outer + Note: reduce and accumulate should return the neutral + element of their base operation, reshaped to agree with the + remaining axes, if used along an axis of length zero. + Example: + add.reduce(array(42, (2,0,1)), 1) + should be equal to + array(0, (2,1)) 6.2) Dot/matrix product + dot(a, b, axis_a = -1, axis_b = 0) equivalent to: if axis_a < 0: axis_a = len(a.shape)+axis_a if axis_b >= 0: axis_b = axis_b + len(a.shape) add.reduce(diagonal(multiply.outer(a,b), axis_a, axis_b)) (but of course done more efficiently) Currently: method matrixMultiply, restricted to rank-2 arrays. Redundant method: nonzero I can't see what this operation is good for. It seems to be equivalent to ravel(notEqual(a,0)), which is not an extremely frequent combination. Abbreviations: sum = add.reduce cumsum = add.accumulate product = multiply.reduce cumproduct = multiply.accumulate allTrue = logicalAnd.reduce someTrue = logicalOr.reduce ptp(a) = maximum.reduce(a)-minimum.reduce(a) Looping, mapping, filtering, etc. ================================= 1) Looping over array elements for element in a: ... Looping over anything else than the first axis can be achieved by indexing. 2) Mapping a function on one or more arrays + arrayMap(function, (a1, a2, a3, ..., an), ranks = None) function must be a callable object. If ranks is specified, it must be a sequence of integers of length n; if ranks is None, all ranks are assumed to be one. The frames of all arrays with respect to the rank specification must match. 3) Filtering + arrayFilter(function, a, axis=0) collects all elements along the given axis for which the supplied function returns non-0. See also repeat() in section "structural operations". 4) Conditional selection + choose(a, (b0, b2, b3, ..., bn)) a must be an array of integers between 0 and n. The result has the same shape as a with elements selected from b0...bn according to the value of the corresponding element of a. Currently: method a.choose(b0, ..., bn) Passing the arrays b0...bn as a tuple has the advantage that this tuple can be constructed arbitrarily in the code. 5) Sorting 5.1) Sort an array + sortArray(a, function=lambda x:x, axis=0) Sorts the elements along the specified axis according to the return value of the supplied function. Note: the default function makes sense only for rank-1 arrays. 5.2) Sort index + sortIndex(a, function=lambda x:x, axis=0) returns an index array i such that take(a,i) == sort(a, function, axis) Redundant function: where(condition, x, y) equals choose(condition, (x,y)) Other operations ================ 1) Type casts 1.1) General cast + a.asType(typecode) 1.2) Special cast functions that also work for scalars + integerArray(a) + floatArray(a) + complexArray(a) 2) Implementation dependent functions: 2.1) Obtain the amount of bytes occupied by each element a.itemsize() 2.2) Return a byte-swapped copy + a.byteswapped() 2.3) Return the element type code a.typecode() 2.4) Return the element type + a.elementType() returns the type object of the Python scalar object that results if an element is extracted. Returns None for a general object (typecode 'O'). 2.5) Checks if an array is contiguous + a.isContiguous() 2.1) Return data as a Python string a.toString() Redundant function: a.copy() Copying should be done with copy.copy(), just as for any other object. Comments ======== 1) APL uses a special case of transposition for selecting a diagonal, but this is often confusing. 2) In J this is done by using ravel() with reduced rank, but as long as we don't have functions with bounded ranks, we need a special function. 3) I tried to find a construction that does not require yet another syntax and this is the best I could think of. But I am not particularly attached to it, so feel free to think of something better. 4) APL does this with fractional indices, which is probably the weirdest feature in APL. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Feb 26 11:30:01 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA05717 for matrix-sig-people; Mon, 26 Feb 1996 11:30:01 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id LAA05706 for ; Mon, 26 Feb 1996 11:29:53 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA26473; Mon, 26 Feb 96 11:28:48 EST Date: Mon, 26 Feb 96 11:28:48 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9602261628.AA26473@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: hinsenk@ere.umontreal.ca Cc: matrix-sig@python.org In-Reply-To: <199602241841.NAA28353@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Subject: Re: [PYTHON MATRIX-SIG] Two weeks' experience Sender: owner-matrix-sig@python.org Precedence: bulk As the only person here with several months of experience writing pratical code, I'd like to comment on a few of Konrad's points. 1) You can in fact write greater(a, b) already, today, using the current package. In fact, I personally do this all this time. Once again, I wish that people would at least try things before complaining that they can't be done. 2) reshape does clearly need work, no arguments here 3) fromFunction was only written to prove to Tom Schwaller that it could be done. It is in fact very useful in it's current form. I have made every efort to avoid having to loop over python functions in the array code as this is ridiculously slow. If you want these functions they can always be easily written in python. I really feel that once this gets released beyond this list it's going to get difficult to make the kinds of major changes that Konrad is advocating, so I'm reluctant to go forward on a public alpha release until these issues are at least somewhat resolved. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Feb 26 11:34:31 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA05776 for matrix-sig-people; Mon, 26 Feb 1996 11:34:31 -0500 Received: from netcomsv.netcom.com (uucp2.netcom.com [163.179.3.2]) by python.org (8.6.12/8.6.12) with ESMTP id LAA05772 for ; Mon, 26 Feb 1996 11:34:28 -0500 Received: from busti.blueskyprod.com by netcomsv.netcom.com with UUCP (8.6.12/SMI-4.1) id IAA11559; Mon, 26 Feb 1996 08:18:18 -0800 Received: from blueskyprod.com by blueskyprod.com with smtp (Smail3.1.29.1 #3) id m0tr4dC-000FmGC; Mon, 26 Feb 96 10:14 EST Received: from troy by blueskyprod.com with smtp (Smail3.1.28.1 #8) id m0tr4Xk-0000tYC; Mon, 26 Feb 96 10:08 EST Message-ID: <3131CD91.1CFB@blueskyprod.com> Date: Mon, 26 Feb 1996 10:11:13 -0500 From: Maurice Van Swaaij X-Mailer: Mozilla 2.0 (X11; I; IRIX 5.3 IP22) MIME-Version: 1.0 To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] openGL Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk I've downloaded the alpha version of openGL from http://maigret.cog.brown.edu/python/opengl/. It says that it needs the python matrix module. Is there an alpha available to test this openGL module with ? -- Maurice Van Swaaij System Architect Blue Sky Productions maurice@blueskyprod.com 1 914 941 5260 ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Feb 26 11:39:24 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA05826 for matrix-sig-people; Mon, 26 Feb 1996 11:39:24 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id LAA05821 for ; Mon, 26 Feb 1996 11:39:21 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA26505; Mon, 26 Feb 96 11:38:14 EST Date: Mon, 26 Feb 96 11:38:14 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9602261638.AA26505@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: chris.chase@jhuapl.edu Cc: matrix-sig@python.org In-Reply-To: <199602240000.TAA20621@python.org> (message from Chris Chase S1A on Fri, 23 Feb 1996 19:06:19 -0500) Subject: Re: [PYTHON MATRIX-SIG] RubberIndex Sender: owner-matrix-sig@python.org Precedence: bulk Just a quick point: If 'a' has rank 3, then a[0] = 1 This implementation still allows an array to be used anywhere where a sequence object (e.g. list) is used. This is not true. In order to be able to use an array anywhere a list is used, a[0] for a rank 3 array must return a rank 2 array, because this is how it works for multidimensional lists currently. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Feb 26 12:35:29 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA06309 for matrix-sig-people; Mon, 26 Feb 1996 12:35:29 -0500 Received: from bartel.jhuapl.edu (bartel.jhuapl.edu [128.244.146.143]) by python.org (8.6.12/8.6.12) with SMTP id MAA06305 for ; Mon, 26 Feb 1996 12:35:26 -0500 Message-Id: <199602261735.MAA06305@python.org> Received: by bartel.jhuapl.edu (1.38.193.4/16.2) id AA01519; Mon, 26 Feb 1996 12:41:00 -0500 Date: Mon, 26 Feb 1996 12:41:00 -0500 From: Chris Chase S1A To: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] RubberIndex In-Reply-To: <9602261638.AA26505@baribal.LCS.MIT.EDU.LCS.MIT.EDU> References: <199602240000.TAA20621@python.org> <9602261638.AA26505@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk >>>>> "James" == James Hugunin writes: James> Just a quick point: James> If 'a' has rank 3, then James> a[0] = 1 James> This implementation still allows an array to be used anywhere where a James> sequence object (e.g. list) is used. James> This is not true. My statement was that it could be used anywhere a sequence object is used, a list being an example of a sequence object. James> In order to be able to use an array anywhere a list is used, You will never be able to use the current array implementation everywhere that a list is used because array has different definitions for operators (e.g. *, +) and array can not support 'del' without having to copy all of its elements. If you want to use an array as a list then use a.toList() (the toList() method would not be needed if arrays were a subclass of lists). James> a[0] for a rank 3 array must return a rank 2 array, because James> this is how it works for multidimensional lists currently. "must"? Why? As far as I can tell the hierarchical indexing for arrays was a convenience for those used to working lists, because it duplicates behavior already provided by a[0,...] indexing. I was suggesting a different behavior for arrays as sequence objects. My suggestion regarded supporting the use of flattened indexing for both selection and assignment. This specific solution for flattened indexing is supported in all the array languages that I have had exposure to (e.g. Matlab, IDL, Tela, Yorick). To me this is more important than the hierarchical indexing scheme so that an array looks like a list (but does not act like a list for other purposes). As an alternative, a 'flat' (or 'flattened') array attribute could be supported. An aside: I know that Konrad Hinsen prefers functions to methods and attributes so that the same code will work for both scalars and arrays. But this is similar to the list analogy above. scalars will never work in all array code because scalars are neither sequences nor dictionary types. The first time one attempts to subscript a scalar an error will be produced. Certainly we don't want subscription to be a function rather than a method. Rather than jump onto the everything is a function bandwagon we may need to rethink the relationship between scalars and arrays. What is the entire scope of code where scalars and arrays need to be interchangeable? Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Feb 26 14:25:16 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA07213 for matrix-sig-people; Mon, 26 Feb 1996 14:25:16 -0500 Received: from bartel.jhuapl.edu (bartel.jhuapl.edu [128.244.146.143]) by python.org (8.6.12/8.6.12) with SMTP id OAA07208 for ; Mon, 26 Feb 1996 14:25:12 -0500 Message-Id: <199602261925.OAA07208@python.org> Received: by bartel.jhuapl.edu (1.38.193.4/16.2) id AA01904; Mon, 26 Feb 1996 14:30:43 -0500 Date: Mon, 26 Feb 1996 14:30:43 -0500 From: Chris Chase S1A To: hinsenk@ere.umontreal.ca (Konrad HINSEN) Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Functions and names In-Reply-To: <199602241438.JAA20228@cyclone.ERE.UMontreal.CA> References: <199602232226.RAA24062@cyclone.ERE.UMontreal.CA> <199602241438.JAA20228@cyclone.ERE.UMontreal.CA> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk >>>>> "Konrad" == Konrad HINSEN writes: Chase> current array representation. Besides, I think it is more Chase> natural to return a copy of the indexed elements rather Chase> than a reference when using a general index vector. Konrad> Indeed, but this creates the unpleasant situation that an expression Konrad> like a[i,j] returns either a copy or a reference, depending on the Konrad> values of i and j. Having references at all is confusing enough, but Konrad> this would certainly cause some headaches. This was not my suggestion. My suggestion kept the current behavior. I suggested a different access method or attribute called 'subarray' allowing the use of a general index vector that would return a copy. Konrad> Personally I'd prefer to have standard indexing always return a copy Konrad> (that is the safer option) and provide an alternative syntax for Konrad> obtaining a reference. Then standard indexing could again allow Konrad> general index vectors. Personally, I currently only care that general index vectors be supported for both subscription and assignment. Chase> I specifically said that the function I was describing overwrites Chase> elements in the target array rather than making room. True it is not Konrad> Then allowing general index vectors in indexed assignments should Konrad> be all you need, right? One only needs slices. The insertion functionality I suggested only requires one to specify the starting position. Normally, the user would have to compute the slice ending position and check to see if the input array will fit (if not the input array could be sliced to fit). I was suggesting a insertion function that would handle options of wrap-around versus truncation versus just error signaling. It is very convenient for manipulating block regions of arrays (like placing one image inside another image). Perhaps a better name than "insert" would be "place"? Chase> I do not like this. Perhaps an attribute 'collapse' would Chase> work? E.g. Chase> a.collapse[i1, i2, ..., i3, i4] Konrad> If that can be done, fine. So what should the value of "collapse" be Konrad> before subscripting? I can't see a way to make this work. Chase> It would have to be an object containing a reference to the Chase> array 'a'. It would have the special property that the Chase> ellipses index, '...', collapses missing dimensions. Chase> Otherwise it would act like a normal array. Since this is a Chase> special kind of access to the array object, a Konrad> So effectively you want to introduce a second array type with Konrad> a slightly different behaviour. Konrad> I don't think this is a good idea. There would be no way to Konrad> stop users from just writing a.collapse and assign the result Konrad> to variables, return it from library functions etc. Suddenly Konrad> there are two array types in free circulation that behave Konrad> almost, but not quite, identically. I was not sure that it was a good solution for this reason. I suggested an attribute because attributes can support the indexing syntax whereas functions can not. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Feb 26 16:12:28 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA08275 for matrix-sig-people; Mon, 26 Feb 1996 16:12:28 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id QAA08271 for ; Mon, 26 Feb 1996 16:12:08 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id QAA04496 (8.6.11/IDA-1.6); Mon, 26 Feb 1996 16:09:40 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA06581; Mon, 26 Feb 1996 16:09:39 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA23356; Mon, 26 Feb 1996 16:09:38 -0500 Date: Mon, 26 Feb 1996 16:09:38 -0500 Message-Id: <199602262109.QAA23356@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: matrix-sig@python.org In-reply-to: <9602261628.AA26473@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] Two weeks' experience From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk 3) fromFunction was only written to prove to Tom Schwaller that it could be done. It is in fact very useful in it's current form. I have made every efort to avoid having to loop over python functions in the array code as this is ridiculously slow. If you want these functions they can always be easily written in python. Of course, but something straightforward should be in the standard function set. The reason why I think the current fromFunction() is not so important is that it can easily be replaced: fromFunction([1,2], f) is the same as apply(f, indices(1,2)) I really feel that once this gets released beyond this list it's going to get difficult to make the kinds of major changes that Konrad is advocating, so I'm reluctant to go forward on a public alpha release until these issues are at least somewhat resolved. Of course it would be necessary to warn people that things are going to change. It would even make sense to include a list of proposals and ask for comments. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Feb 26 16:29:40 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA08463 for matrix-sig-people; Mon, 26 Feb 1996 16:29:40 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id QAA08459 for ; Mon, 26 Feb 1996 16:29:36 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id QAA05172 (8.6.11/IDA-1.6); Mon, 26 Feb 1996 16:27:07 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA09903; Mon, 26 Feb 1996 16:27:06 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA24193; Mon, 26 Feb 1996 16:27:05 -0500 Date: Mon, 26 Feb 1996 16:27:05 -0500 Message-Id: <199602262127.QAA24193@cyclone.ERE.UMontreal.CA> To: chris.chase@jhuapl.edu CC: jjh@Goldilocks.LCS.MIT.EDU, matrix-sig@python.org In-reply-to: <199602261735.MAA06305@python.org> (message from Chris Chase S1A on Mon, 26 Feb 1996 12:41:00 -0500) Subject: Re: [PYTHON MATRIX-SIG] RubberIndex From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk You will never be able to use the current array implementation everywhere that a list is used because array has different definitions for operators (e.g. *, +) and array can not support 'del' without True, but that is no reason to introduce additional incompatibilities. "must"? Why? As far as I can tell the hierarchical indexing for arrays was a convenience for those used to working lists, because it duplicates behavior already provided by a[0,...] indexing. Historically it was the other way round, but all that doesn't matter. The question is what behaviour is the most useful. both selection and assignment. This specific solution for flattened indexing is supported in all the array languages that I have had exposure to (e.g. Matlab, IDL, Tela, Yorick). To me this is more important than the hierarchical indexing scheme so that an array looks like a list (but does not act like a list for other purposes). The array languages I am familiar with (APL and J) do not permit this. I suppose that the languages you mention support flattened access because Fortran programmers are used to it, not because it is important in real applications. I still think that a structural modification like flattening should be explicit, not implied. And let's not forget that our goal is not to write a Matlab/IDL/whatever clone, but to provide arrays for Python. We should stick as close as possible to established Python habits, which is why I insist so much on the analogy with nested lists. As an alternative, a 'flat' (or 'flattened') array attribute could be supported. Why? What's wrong with ravel(a)? You can index its result if you want. An aside: I know that Konrad Hinsen prefers functions to methods and attributes so that the same code will work for both scalars and arrays. But this is similar to the list analogy above. scalars will never work in all array code because scalars are neither sequences nor dictionary types. The first time one attempts to subscript a scalar an error will be produced. Certainly we don't want subscription to be True, but that makes sense. Scalars are considered rank-0 arrays, and indexing rank-0 arrays makes no sense. So the behaviour is consistent. And again, the fact that something can't be done perfectly is no argument for not doing it as good as possible. My point about functions vs. methods is that many algorithms can reasonably applied to arrays of all ranks, including rank 0. Of course such algorithms cannot include indexing. But all the operations that are rank-independent should be implemented in such a way that they accept both arrays and scalars. a function rather than a method. Rather than jump onto the everything is a function bandwagon we may need to rethink the relationship between scalars and arrays. What is the entire scope of code where scalars and arrays need to be interchangeable? Everything except indexing. Some care must be taken with functions that require an axis specification (like add.reduce). Obviously a rank-0 array does not have axes, but add.reduce(x) still makes sense if x is a scalar. Ideally, a call to add.reduce with an explicit axis specification should raise an exception if given a scalar, but do the right thing with the default axis. BTW, there are more arguments not to implement array operations as methods, as I have outlined in my proposal. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Feb 26 16:37:30 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA08547 for matrix-sig-people; Mon, 26 Feb 1996 16:37:30 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id QAA08543 for ; Mon, 26 Feb 1996 16:37:27 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id QAA05445 (8.6.11/IDA-1.6); Mon, 26 Feb 1996 16:34:57 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA11230; Mon, 26 Feb 1996 16:34:56 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id QAA24610; Mon, 26 Feb 1996 16:34:55 -0500 Date: Mon, 26 Feb 1996 16:34:55 -0500 Message-Id: <199602262134.QAA24610@cyclone.ERE.UMontreal.CA> To: chris.chase@jhuapl.edu CC: matrix-sig@python.org In-reply-to: <199602261922.OAA18677@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Mon, 26 Feb 1996 14:30:43 -0500) Subject: Re: [PYTHON MATRIX-SIG] Functions and names From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk One only needs slices. The insertion functionality I suggested only requires one to specify the starting position. Normally, the user would have to compute the slice ending position and check to see if the input array will fit (if not the input array could be sliced to fit). I was suggesting a insertion function that would handle options of wrap-around versus truncation versus just error signaling. It is very convenient for manipulating block regions of arrays (like placing one image inside another image). Perhaps a better name than "insert" would be "place"? I see. That looks indeed useful; I remember having once written an APL function to do more or less the same. It is however not so easy to find a nice syntax for this. Right now all operations that change array elements are assignments, and I'd like to keep it that way. One possibility would be to allow somthing like this: a = array([1,2,3,4,5,6]) b = array([0,0]) a[2] = b with the result a == array([1,2,0,0,5,6]) I just don't like the error checking that gets lost in the process. Maybe the alternative a[2::] = b (assignment of a vector that is too short) would be a lesser evil. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 28 10:50:42 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA26789 for matrix-sig-people; Wed, 28 Feb 1996 10:50:42 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id KAA26785 for ; Wed, 28 Feb 1996 10:50:37 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA09113; Wed, 28 Feb 96 10:47:58 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) for matrix-sig@python.org id KAA12674; Wed, 28 Feb 1996 10:54:59 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199602281554.KAA12674@maigret> Subject: [PYTHON MATRIX-SIG] fft's To: matrix-sig@python.org Date: Wed, 28 Feb 1996 10:54:59 -0500 (EST) X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 2801 Sender: owner-matrix-sig@python.org Precedence: bulk I've been working on python interface to the FFT routines in the complib.sgimath library (since those are the ones I have optimized binaries for). This has been educational, but it has raised several questions which I can't answer. First I'll talk about a specific one related to 2-d and 3-d FFT's, then ask a few questions about interfacing to fortran in general. The complib.sgimath (based on stuff from netlib) has a set of fft calls, 1, 2 and 3-d. Things are working fine in 1-D, but I have to admit that my grokking of 2-D and 3-D FFT's is rather limited. When doing a 1-d fft, the output array is bigger than the input array by enough to fit an aligned complex. Fine. The 2-d fft calls in the fft library I'm using have the following syntax: int dfft2du( int job, int n1, int n2, double *sequence, int lda, double *workspace) job specifies direction (forward vs. backward) sequence is where the data is stored (both before and after the fft) workspace is the sines & cosines and the factorization of n1 and n2. All that is fine. n1 and n2 are defined as follows: N1 - INTEGER. On entry, N1 specifies the number of elements in the first dimension of the sequence. Unchanged on exit. N2 - INTEGER. On entry, N2 specifies the number of elements in the second dimension of the sequence. Unchanged on exit. and LDA is defined as: LDA - INTEGER. On entry, LDA specifies the first dimension of the array SEQUENCE as declared in the calling (sub)program. LDA must be at least max( 1, 2*((N1+2)/2) ). Unchanged on exit. This is where I get lost. I assumed that if I had a n1 x n2 array to start with, I'd have an array which would be slightly bigger in, say, n1 and n2 (enough to store a complex?). But this LDA parameter is completely confusing me. I have similar problems in 3d of course, where LDA is replaced by ld1 and ld2. I suspect if I grok LDA i'll grok LD1 and LD2. =) I realize that fortran and C/Python index into arrays differently (row vs. column first). What is a good way of dealing with this? This brings to mind a different issue regarding interfaces to existing libraries. The library I'm using returns rank 1 arrays, which are really rank 1, 2, or 3 but designed for antiquated languages. Should I do the conversion to the appropriate rank (e.g. the assignment of shape) in a .py wrapper, so that the library module itself is as compatible w/ the fortran and C interfaces, or should I make the library python-friendly, and have it return appropriately sized arrays? Thanks for reading this far. --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 28 12:21:10 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA27538 for matrix-sig-people; Wed, 28 Feb 1996 12:21:10 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id MAA27533 for ; Wed, 28 Feb 1996 12:21:01 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA03053; Wed, 28 Feb 96 09:19:29 PST Received: from kristen (localhost) by kristen.llnl.gov (5.x/SMI-SVR4) id AA01069; Wed, 28 Feb 1996 09:19:27 -0800 Message-Id: <31348E9F.8EC@llnl.gov> Date: Wed, 28 Feb 1996 09:19:27 -0800 From: "Paul. Dubois" Organization: Lawrence Livermore National Laboratory X-Mailer: Mozilla 2.0 (X11; I; SunOS 5.4 sun4m) Mime-Version: 1.0 To: David Ascher Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] fft's References: <199602281554.KAA12674@maigret> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk Most routines of this type written in C or Fortran allow for the possibility that an array has been declared of a larger size than it actually is. For example #define N 32 #define M 20 int n, m, c[N]{M]; n = something m = something ...now you want to do something with c and as far as you are concerned it is n by m. In C, the arrays are stored so that in considering c[i][j] and c[i][j+1], they are 1 storage unit apart, but c[i][j] and c[i+1][j] are M apart. For Fortran, it works the other way, and it is c(i,j) and c(i, j+1) that are N apart. When you pass such a beast to a general routine, therefore, it is necessary to tell it N in Fortran or M in C. So in your fft what they want for lda is M. However, when working with an object-oriented language, this kind of partially full matrix just doesn't arise very often, and in most of your work the python array x with shape (n,m) will be passed to the library fft routine with lda = m. Hope this helps. -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 28 17:05:51 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA30866 for matrix-sig-people; Wed, 28 Feb 1996 17:05:51 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id RAA30862 for ; Wed, 28 Feb 1996 17:05:46 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id RAA20394 (8.6.11/IDA-1.6); Wed, 28 Feb 1996 17:02:57 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA01082; Wed, 28 Feb 1996 17:02:56 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA24236; Wed, 28 Feb 1996 17:02:55 -0500 Date: Wed, 28 Feb 1996 17:02:55 -0500 Message-Id: <199602282202.RAA24236@cyclone.ERE.UMontreal.CA> To: da@maigret.cog.brown.edu CC: matrix-sig@python.org In-reply-to: <199602281554.KAA12674@maigret> (da@maigret.cog.brown.edu) Subject: Re: [PYTHON MATRIX-SIG] fft's From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > n1 and n2 are defined as follows: > > N1 - INTEGER. > On entry, N1 specifies the number of elements in the > first dimension of the sequence. > Unchanged on exit. > N2 - INTEGER. > On entry, N2 specifies the number of elements in the > second dimension of the sequence. > Unchanged on exit. So these describe the number of data points... > and LDA is defined as: > > LDA - INTEGER. > On entry, LDA specifies the first dimension of the > array SEQUENCE as declared in the calling (sub)program. > LDA must be at least max( 1, 2*((N1+2)/2) ). > Unchanged on exit. and this the physical size of the grid. The reason for having both is that you might want to dimension some arrays larger than necessary to avoid recompilation for every different value of N1/N2. Remember, we are talking about a library designed for a language without dynamic allocation! In the case of Python, you would most certainly allocate an array of the correct size and LDA would be equal to N1. You will find these "LDA" parameters everywhere in Fortran libraries. > I realize that fortran and C/Python index into arrays differently (row > vs. column first). What is a good way of dealing with this? Your interface should make sense to Python users. They won't see the Fortran source code anyway and probably don't even care whether it's Fortran. > really rank 1, 2, or 3 but designed for antiquated languages. Should I > do the conversion to the appropriate rank (e.g. the assignment of shape) > in a .py wrapper, so that the library module itself is as compatible w/ > the fortran and C interfaces, or should I make the library > python-friendly, and have it return appropriately sized arrays? It matters only that the function that end users call uses correctly shaped arrays. Personally, I'd use the correct shape also for the low-level interface, because there is no possible use for 1d access. But if you prefer otherwise, I don't care. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 28 17:39:28 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA31288 for matrix-sig-people; Wed, 28 Feb 1996 17:39:28 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id RAA31284 for ; Wed, 28 Feb 1996 17:39:26 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa15732; 28 Feb 96 17:36 EST Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id RAA23873; Wed, 28 Feb 1996 17:36:03 -0500 Message-Id: <199602282236.RAA23873@monty> To: Konrad HINSEN cc: da@maigret.cog.brown.edu, matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] fft's In-reply-to: Your message of "Wed, 28 Feb 1996 17:02:55 EST." <199602282202.RAA24236@cyclone.ERE.UMontreal.CA> References: <199602282202.RAA24236@cyclone.ERE.UMontreal.CA> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Wed, 28 Feb 1996 17:36:03 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk > > LDA - INTEGER. > > On entry, LDA specifies the first dimension of the > > array SEQUENCE as declared in the calling (sub)program. > > LDA must be at least max( 1, 2*((N1+2)/2) ). > > Unchanged on exit. > > and this the physical size of the grid. The reason for having both > is that you might want to dimension some arrays larger than necessary > to avoid recompilation for every different value of N1/N2. This is what I thought, except that the formula for LDA's minimum size seems to imply that it must be at least *1 larger* than N1? Perhaps the library module uses this as temp space? --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Feb 28 17:45:20 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA31369 for matrix-sig-people; Wed, 28 Feb 1996 17:45:20 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id RAA31364 for ; Wed, 28 Feb 1996 17:45:17 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id RAA21408 (8.6.11/IDA-1.6); Wed, 28 Feb 1996 17:41:19 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA07938; Wed, 28 Feb 1996 17:41:18 -0500 Received: by cyclone.ERE.UMontreal.CA (950221.405.SGI.8.6.10/5.17) id RAA26182; Wed, 28 Feb 1996 17:41:18 -0500 Date: Wed, 28 Feb 1996 17:41:18 -0500 Message-Id: <199602282241.RAA26182@cyclone.ERE.UMontreal.CA> To: guido@CNRI.Reston.VA.US CC: da@maigret.cog.brown.edu, matrix-sig@python.org In-reply-to: <199602282236.RAA23873@monty> (message from Guido van Rossum on Wed, 28 Feb 1996 17:36:03 -0500) Subject: Re: [PYTHON MATRIX-SIG] fft's From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > > > LDA - INTEGER. > > > On entry, LDA specifies the first dimension of the > > > array SEQUENCE as declared in the calling (sub)program. > > > LDA must be at least max( 1, 2*((N1+2)/2) ). > > > Unchanged on exit. > > > > and this the physical size of the grid. The reason for having both > > is that you might want to dimension some arrays larger than necessary > > to avoid recompilation for every different value of N1/N2. > > This is what I thought, except that the formula for LDA's minimum size > seems to imply that it must be at least *1 larger* than N1? Perhaps > the library module uses this as temp space? The minimum value is nonsense. The smallest reasonable value for N1 is 1, which makes 2*((N1+2)/2) = 2, which is always larger than 1. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Mar 4 11:04:22 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA06319 for matrix-sig-people; Mon, 4 Mar 1996 11:04:22 -0500 Received: from hurlbut. (hurlbut.jhuapl.edu [128.244.146.28]) by python.org (8.6.12/8.6.12) with SMTP id LAA06315 for ; Mon, 4 Mar 1996 11:04:11 -0500 Received: by hurlbut. (5.0/SMI-SVR4) id AA21958; Mon, 4 Mar 1996 11:01:34 +0500 Date: Mon, 4 Mar 1996 11:01:34 +0500 Message-Id: <9603041601.AA21958@hurlbut.> From: Chris Chase S1A To: hinsenk@ere.umontreal.ca (Konrad HINSEN) Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] RubberIndex In-Reply-To: <199602262127.QAA24193@cyclone.ERE.UMontreal.CA> References: <199602261735.MAA06305@python.org> <199602262127.QAA24193@cyclone.ERE.UMontreal.CA> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII content-length: 1324 Sender: owner-matrix-sig@python.org Precedence: bulk >>>>> "Konrad" == Konrad HINSEN writes: Chase> both selection and assignment. This specific solution for Chase> flattened indexing is supported in all the array languages Chase> that I have had exposure to (e.g. Matlab, IDL, Tela, Chase> Yorick). To me this is more important than the hierarchical Chase> indexing scheme so that an array looks like a list (but does Chase> not act like a list for other purposes). Konrad> The array languages I am familiar with (APL and J) do not Konrad> permit this. I forgot about my brief experience with APL-PLUS. Chase> As an alternative, a 'flat' (or 'flattened') array attribute Chase> could be supported. Konrad> Why? What's wrong with ravel(a)? You can index its result if Konrad> you want. The reason for a 'flat' attribute is that ravel(a) does not support _assignment_ to 'a' in flattened form. An attribute or method is a more natural interface for changing an objects data members. However, ravel(a) could provide this functionality if ravel(a) returned a reference instead of a copy. The attribute would not fit your functions for everything so that scalars can be substituted for arrays. But if the flattened array is to be indexed (for assignment or selection) then scalars could not be supported anyway. Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Mar 4 11:41:31 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA06736 for matrix-sig-people; Mon, 4 Mar 1996 11:41:31 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id LAA06730 for ; Mon, 4 Mar 1996 11:41:27 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id LAA28237 (8.6.11/IDA-1.6); Mon, 4 Mar 1996 11:37:26 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id LAA03488; Mon, 4 Mar 1996 11:37:24 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id LAA25988; Mon, 4 Mar 1996 11:37:23 -0500 Date: Mon, 4 Mar 1996 11:37:23 -0500 Message-Id: <199603041637.LAA25988@cyclone.ERE.UMontreal.CA> To: chris.chase@aplmail.jhuapl.edu CC: matrix-sig@python.org In-reply-to: <9603041601.AA21958@hurlbut.> (message from Chris Chase S1A on Mon, 4 Mar 1996 11:01:34 +0500) Subject: Re: [PYTHON MATRIX-SIG] RubberIndex From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > The reason for a 'flat' attribute is that ravel(a) does not support > _assignment_ to 'a' in flattened form. An attribute or method is a > more natural interface for changing an objects data members. However, > ravel(a) could provide this functionality if ravel(a) returned a > reference instead of a copy. I think that right now it does, but in my opinion it shouldn't, at least not by default. For assignment in flattened form, my proposal contains: 2.1.3) Combining a group of axes into one axis (see comment 2): + a[i1, i2, ......, i3, i4] (see comment 3) The double ellipsis works similar to the single one, but contracts all the axes covered into a single axis. This is more flexible, since it allows partially flattened forms, and follows the principle "assignment only via indexing", which I might add to the list of principles... > The attribute would not fit your functions for everything so that > scalars can be substituted for arrays. But if the flattened array is > to be indexed (for assignment or selection) then scalars could not be > supported anyway. Which is another reason to do all assignments in the form of indexing, such that indexing is the only operation not supported on scalars. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Mar 4 13:53:23 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA07690 for matrix-sig-people; Mon, 4 Mar 1996 13:53:23 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id NAA07685 for ; Mon, 4 Mar 1996 13:53:08 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id NAA02673 (8.6.11/IDA-1.6 for ); Mon, 4 Mar 1996 13:49:16 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id NAA25490; Mon, 4 Mar 1996 13:49:14 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id NAA03129; Mon, 4 Mar 1996 13:49:13 -0500 Date: Mon, 4 Mar 1996 13:49:13 -0500 Message-Id: <199603041849.NAA03129@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Proposal: upward cast operation From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk Problem: How do you write a mathematical function that works for integers, floats, complex numbers, and number classes implemented in Python? If you think that's easy, look at this example: def f(a, b): return a/(a+b) If you call this with integer arguments, it won't do what you think, because the division operation for integers is different. You might of course simply say "don't call this with integer arguments", but you might have to call it with numbers coming from, say, a library function whose types are not so easy to predict. You could also modify the code to def f(a, b): return float(a)/(a+b) but then it doesn't work for complex arguments, because a gets replaced by its real part. And for other number types, you don't even know what float() does. What is needed is a type cast that acts only "if necessary". I propose to add two such functions to umath, one to cast to float and the other to cast to complex. The cast to float should work like this: 1) If the argument is an integer, cast it to float. 2) If the argument has a suitably named method, call this method to do the cast. 3) If neither 1) or 2) apply, return the argument unchanged. The cast to complex would work similarly, with only the first step different. Any comments? Is there a problem I have overlooked? Or is there a simpler solution to the problem? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Mar 5 14:51:38 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA17451 for matrix-sig-people; Tue, 5 Mar 1996 14:51:38 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id OAA17443 for ; Tue, 5 Mar 1996 14:51:29 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA17071; Tue, 5 Mar 96 14:47:16 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) for matrix-sig@python.org id OAA28722; Tue, 5 Mar 1996 14:44:04 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199603051944.OAA28722@maigret> Subject: [PYTHON MATRIX-SIG] which fft library should I do? To: matrix-sig@python.org Date: Tue, 5 Mar 1996 14:44:03 -0500 (EST) X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 1343 Sender: owner-matrix-sig@python.org Precedence: bulk While I have started work on a module to the fft routines in the SGI complib.sgimath library, I'm thinking that maybe I should do something which is 1) better tested (there are things like "array of size 2*((n+2)/2)" in the man pages), and 2) more portable (e.g. would work on my PC). I've looked around on the web, and there are more FFT packages out there than there should be. =) Netlib's FFTPACK is fine, but it's only in Fortran (the fft.c file doesn't seem to be a straight port -- it seems to change the function names, for example). I think that the library used should come in C at least, and in Fortran if possible, mostly due to the portability problem. While "serious" fft users would probably do it on a fast machine w/ a fortran library, I think that I want to be able to do FFT on MacPython or PythonWin without a fortran compiler. I think the first goal of an fft module for Python would be to be relatively generic and relatively simple. It should do at least 1, 2 and 3 dims. It should deal with floats, doubles, and float and double complex. I don't think that I want to worry about routines which are optimized for, say, multiple even sequences with only odd wave numbers. Any suggestions? --david PS: How bad of a performance hit does one take w/ f2c-output code as opposed to handcoded C on code like this? ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Mar 5 16:30:05 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA18216 for matrix-sig-people; Tue, 5 Mar 1996 16:30:05 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id QAA18211 for ; Tue, 5 Mar 1996 16:30:00 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id QAA14069 (8.6.11/IDA-1.6); Tue, 5 Mar 1996 16:25:38 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id QAA22798; Tue, 5 Mar 1996 16:25:38 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id QAA12725; Tue, 5 Mar 1996 16:25:36 -0500 Date: Tue, 5 Mar 1996 16:25:36 -0500 Message-Id: <199603052125.QAA12725@cyclone.ERE.UMontreal.CA> To: da@maigret.cog.brown.edu CC: matrix-sig@python.org In-reply-to: <199603051944.OAA28722@maigret> (da@maigret.cog.brown.edu) Subject: Re: [PYTHON MATRIX-SIG] which fft library should I do? From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > I think that the library used should come in C at least, and in Fortran > if possible, mostly due to the portability problem. While "serious" fft > users would probably do it on a fast machine w/ a fortran library, I > think that I want to be able to do FFT on MacPython or PythonWin without > a fortran compiler. I agree. I remember having seen a list of mathematical libraries in C somewhere, which is where you should start looking for C libraries. I suppose it was in news.answers, but I am not sure. > PS: How bad of a performance hit does one take w/ f2c-output code as > opposed to handcoded C on code like this? That depends a lot on your C compiler and on your application. Many C compilers are not very good at standard optimizations for numerical code. The difficulties caused by the language itself are few: pointer aliasing (a Fortran compiler may assume that two array arguments passed to a subroutine do not refer to overlapping memory blocks; a C compiler may not), the power operator (few C compilers can optimize e.g. small integer powers) and intrinsic functions (although some C compilers already come with a good substitute in the form of #pragmas in math.h and corresponding optimizations). Another question is whether the code produced by f2c is sufficiently portable. After all, f2c has to be configured for some features of the C compiler, and there are C compilers for which this is impossible. For example, the C compiler must provide two floating-point precisions of which the larger one uses exactly twice as many bytes as the smaller one. I remember one compiler (Pure-C for the Atari ST) for which I could not configure f2c, because "float" takes 4 bytes and "double" 10. Another problem is the f2c library. The generic source code is very Unix-specific. I don't know whether there are any ports to other operating systems. All that means that a real C library is best. One can still have an alternate Fortran library binding for people with a Fortran compiler and a need for speed. As long as the high-level interface is the same, few users should see the difference. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Mar 6 00:02:58 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id AAA21350 for matrix-sig-people; Wed, 6 Mar 1996 00:02:58 -0500 Received: from galaxies.plk.af.mil (ip40.abq-dialin.hollyberry.com [165.247.201.40]) by python.org (8.6.12/8.6.12) with SMTP id AAA21346 for ; Wed, 6 Mar 1996 00:02:35 -0500 Received: by galaxies.plk.af.mil (NX5.67d/NX3.0M) id AA00271; Tue, 5 Mar 96 21:59:45 -0700 Date: Tue, 5 Mar 96 21:59:45 -0700 From: Earl Spillar Message-Id: <9603060459.AA00271@galaxies.plk.af.mil> Received: by NeXT.Mailer (1.100) Received: by NeXT Mailer (1.100) To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Efficient data reading Reply-To: spillar@uwyo.edu Sender: owner-matrix-sig@python.org Precedence: bulk Gentlepeople: I'm going to implement some astronomical image processing stuff using the Matrix extensions. I think it's a perfect application for Python- I need to perform arithmatic on 2MB numerical arrays quickly and efficiently, while dealing with some simple variables associated with each array. I could post more details if there is interest- Anyway, the question is how to most efficiently read in data already written to disk in a standard format (FITS). I know how to parse the headers, its the 2 MB data arrays I'm worried about. I'm thinking about reading the data using the array module, and feeding the result of that to the Numerical module. Is that reasonably efficient? It seems like I'll have 2 copies floating around for a moment. Is there another mechanism in the Numerical module that I'm missing? Thanks for any help- this stuff looks great, and answers needs I've had for some time! Earl Spillar spillar@uwyo.edu ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Mar 6 02:59:47 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id CAA22282 for matrix-sig-people; Wed, 6 Mar 1996 02:59:47 -0500 Received: from grz08u.unileoben.ac.at (grz08u.unileoben.ac.at [192.82.157.120]) by python.org (8.6.12/8.6.12) with SMTP id CAA22277 for ; Wed, 6 Mar 1996 02:59:39 -0500 Received: from g4305m.unileoben.ac.at by grz08u.unileoben.ac.at (AIX 3.2/UCB 5.64/4.03) id AA13795; Wed, 6 Mar 1996 08:56:29 +0100 Date: Wed, 6 Mar 96 08:54:56 +0100 From: "Herbert Weinhandl" Message-Id: <45891.weinhand@unileoben.ac.at> X-Minuet-Version: Minuet1.0_Beta_17A Reply-To: X-Popmail-Charset: IBM 8-Bit To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] which fft library should I do? Sender: owner-matrix-sig@python.org Precedence: bulk [snip] >All that means that a real C library is best. One can still >have an alternate Fortran library binding for people with >a Fortran compiler and a need for speed. As long as the >high-level interface is the same, few users should see the >difference. You should look at th URL http://www.tu-chemnitz.de/~arndt/joerg.html there is e.g. fxt.tgz (C source for FFT and Fast Hartley Transform). You can find a lot of links to other sites related to this matter. I hope this helps Herbert Weinhandl Institute for Solid State Physics, Austrian Academy of Sciences, Institute for Metal Physics, University for Mining & Metallurgy, Jahnstr. 12, ! It is A-8700 Leoben, Austria ! NOT Tel: (+43) 3842-45511-37 ! my bailiwick ! Fax: (+43) 3842-45512-16 ! from Cliff Stoll's "The Cuckoo's Egg" ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Mar 6 10:16:07 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA00265 for matrix-sig-people; Wed, 6 Mar 1996 10:16:07 -0500 Received: from bartel.jhuapl.edu (bartel.jhuapl.edu [128.244.146.143]) by python.org (8.6.12/8.6.12) with SMTP id KAA00253 for ; Wed, 6 Mar 1996 10:15:59 -0500 Message-Id: <199603061515.KAA00253@python.org> Received: by bartel.jhuapl.edu (1.38.193.4/16.2) id AA09894; Wed, 6 Mar 1996 09:12:28 -0500 Date: Wed, 6 Mar 1996 09:12:28 -0500 From: Chris Chase S1A To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] index of C and C++ numerical software Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk Below is the header of a file containing info on numerical C/C++ software on the net. It contains pointers to lots of FFT software among other things. Chris Welcome to of the index of resources for numerical computation in C or C++. It is a collection of pointers to: - free source code available on the net, - articles and documents, especially those available over the net. This file is ftp://usc.edu/pub/C-numanal/numcomp-free-c.gz ftp://ftp.math.psu.edu/pub/FAQ/numcomp-free-c or c/numcomp-free-c on netlib (slightly outdated). Also see the last item on http://www.math.psu.edu/OtherMath.html ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Mar 6 10:33:48 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA00444 for matrix-sig-people; Wed, 6 Mar 1996 10:33:48 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id KAA00437 for ; Wed, 6 Mar 1996 10:33:43 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA06526; Wed, 6 Mar 96 08:29:41 EST Date: Wed, 6 Mar 96 08:29:41 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9603061329.AA06526@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: spillar@uwyo.edu Cc: matrix-sig@python.org In-Reply-To: <9603060459.AA00271@galaxies.plk.af.mil> (message from Earl Spillar on Tue, 5 Mar 96 21:59:45 -0700) Subject: Re: [PYTHON MATRIX-SIG] Efficient data reading Sender: owner-matrix-sig@python.org Precedence: bulk From: Earl Spillar I'm going to implement some astronomical image processing stuff using the Matrix extensions. I think it's a perfect application for Python- I need to perform arithmatic on 2MB numerical arrays quickly and efficiently, while dealing with some simple variables associated with each array. I could post more details if there is interest- BTW - has anybody heard anything more about the image module discussed a while back on this list? It sounds like it might be very helpful here. Anyway, the question is how to most efficiently read in data already written to disk in a standard format (FITS). I know how to parse the headers, its the 2 MB data arrays I'm worried about. I'm thinking about reading the data using the array module, and feeding the result of that to the Numerical module. Is that reasonably efficient? It seems like I'll have 2 copies floating around for a moment. Is there another mechanism in the Numerical module that I'm missing? I explicitly designed the numeric module to not access file objects (well, there are a couple of hidden methods lying around, but those wil be going away very soon). This makes safe code (things like grail) much easier to think about. It also should strongly encourage people to use pickling as their "native" format for storing arrays which I think is almost always the best choice. Also, for every case I've had to deal with, reading data into a string, and then converting the string to an array works just fine. ie. data = Numeric.fromString(fp.read(n_bytes), type_of_data) It is true, this does temporarily require twice as much memory as data "should" require, and it memcpy's all of that data once. However, in comparision to the time required for reading 2MB of data off even a fast local disk, the memory copies/allocations are negligible. Almost anything that you do with arrays (like a = a + 2) will temporarily require this double memory. (Note to experts, this can actually be written as add(a, 2, a) in which case you don't need that extra memory, but if you want to write a lot of code like this, then you should probably be working in C anyway). -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Mar 6 11:07:45 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA00759 for matrix-sig-people; Wed, 6 Mar 1996 11:07:45 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id LAA00755 for ; Wed, 6 Mar 1996 11:07:42 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id JAA00801 (8.6.11/IDA-1.6); Wed, 6 Mar 1996 09:14:42 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id JAA02685; Wed, 6 Mar 1996 09:14:41 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id JAA24129; Wed, 6 Mar 1996 09:14:41 -0500 Date: Wed, 6 Mar 1996 09:14:41 -0500 Message-Id: <199603061414.JAA24129@cyclone.ERE.UMontreal.CA> To: spillar@uwyo.edu CC: matrix-sig@python.org In-reply-to: <9603060459.AA00271@galaxies.plk.af.mil> (message from Earl Spillar on Tue, 5 Mar 96 21:59:45 -0700) Subject: Re: [PYTHON MATRIX-SIG] Efficient data reading From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > the headers, its the 2 MB data arrays I'm worried about. I'm thinking about > reading the data using the array module, and feeding the result of that to the > Numerical module. Is that reasonably efficient? It seems like I'll have 2 > copies floating around for a moment. Is there another mechanism > in the Numerical module that I'm missing? There is no difference between the "array module" and the "numerical module". The module "Numerical" simply imports objects from the array module. Or are you referring to the old array module? I see no problem in writing a C function that allocates the array and loads the data without any copying. However, you will have to take care that this array is not copied during later processing. Python is not exactly designed to avoid copying objects. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Mar 6 16:41:55 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA03628 for matrix-sig-people; Wed, 6 Mar 1996 16:41:55 -0500 Received: from catch.ivab.se (root@catch.ivab.se [193.180.173.22]) by python.org (8.6.12/8.6.12) with ESMTP id QAA03624 for ; Wed, 6 Mar 1996 16:41:50 -0500 Received: from arnold.image.ivab.se (arnold.image.ivab.se [193.180.173.119]) by catch.ivab.se (8.6.12/8.6.12) with SMTP id WAA18382 for ; Wed, 6 Mar 1996 22:41:49 +0100 Received: by arnold.image.ivab.se; (5.65v3.2/1.1.8.2/16Nov95-0104AM) id AA20782; Wed, 6 Mar 1996 22:41:43 +0100 Date: Wed, 6 Mar 1996 22:41:43 +0100 Message-Id: <9603062141.AA20782@arnold.image.ivab.se> From: Fredrik Lundh To: matrix-sig@python.org In-Reply-To: <9603061329.AA06526@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] Efficient data reading Reply-To: fredrik_lundh@ivab.se Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk > BTW - has anybody heard anything more about the image module discussed > a while back on this list? It sounds like it might be very helpful > here. An image-sig have been formed, but not yet activated. I've been a bit too busy writing articles and holding seminars on Python... Anyway, feel free to sign up (image-sig-request@python.org), but don't expect any activity (not this week, at least :-). A formal announcement will be posted to the news group real soon now. The Python Imaging Library, release 0.1, is nearly completed, and I hope to release it within a few weeks. I've even written a handbook for it... Regards /F ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Mar 6 23:28:46 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id XAA06682 for matrix-sig-people; Wed, 6 Mar 1996 23:28:46 -0500 Received: from roper.uwyo.edu (roper.uwyo.edu [129.72.60.8]) by python.org (8.6.12/8.6.12) with ESMTP id XAA06678 for ; Wed, 6 Mar 1996 23:28:43 -0500 Received: from plains.uwyo.edu (plains.uwyo.edu) by ROPER.UWYO.EDU (PMDF V5.0-4 #14244) id <01I20XJXV7TC0028TI@ROPER.UWYO.EDU> for matrix-sig@python.org; Wed, 06 Mar 1996 16:56:17 -0700 (MST) Received: from 129.238.158.202 ("port 3924"@129.238.158.202) by PLAINS.UWYO.EDU (PMDF V5.0-4 #14244) id <01I20XJNI9I8001QIK@PLAINS.UWYO.EDU> for matrix-sig@python.org; Wed, 06 Mar 1996 16:56:14 -0700 (MST) Date: Wed, 06 Mar 1996 16:56:14 -0700 (MST) From: Earl J Spillar Subject: [PYTHON MATRIX-SIG] Efficient data reading To: matrix-sig@python.org Reply-to: spillar@uwyo.edu Message-id: MIME-version: 1.0 Content-type: text/plain; charset=US-ASCII; X-MAPIextension=".TXT" Content-transfer-encoding: quoted-printable Priority: Normal Sender: owner-matrix-sig@python.org Precedence: bulk Gentlemen- thanks for your answers. James Hugunin said: BTW - has anybody heard anything more about the image module discussed a while back on this list? It sounds like it might be very helpful here. Probably will be helpful. A lot of what astronomers call "image processi= ng" is actually numerical analysis though, so the other Matrix stuff is very important. I explicitly designed the numeric module to not access file objects... strongly encourage people to use pickling as their "native" format for = storing arrays which I think is almost always the best choice..." I agree completely with this philosophy. I will certainly keep my data = in pickled format. I do have a need to get the data in and out to use "lega= cy" apps though, hence the question. (Note to experts, this can actually be written as add(a, 2, a) in which case you don't need that extra memory, but if you want to write a lot of code like this, then you should probably be working in C anyway). Thanks for the trick. I WILL be doing a lot of code like this, but it = needs to be "reconfigured" frequently. I want to prototype in the fluid Python = environment and then freeze bottlenecks into C. Konrad said: There is no difference between the "array module" and the "numerical module". The module "Numerical" simply imports objects from the array module. Or are you referring to the old array mod= ule? Actually, was thinking about the old array module. I guess when I saw = the Numeric.fromString method, I thought C string, and CR/LF etc, NULLs, and = thought it wouldn't work... primitive thinking on my part. Thanks for the help! Earl Spillar spillar@uwyo.edu http://plains.uwyo.edu/~spillar/ ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Mar 7 01:55:10 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id BAA07511 for matrix-sig-people; Thu, 7 Mar 1996 01:55:10 -0500 Received: from catch.ivab.se (root@catch.ivab.se [193.180.173.22]) by python.org (8.6.12/8.6.12) with ESMTP id BAA07502 for ; Thu, 7 Mar 1996 01:55:07 -0500 Received: from arnold.image.ivab.se (arnold.image.ivab.se [193.180.173.119]) by catch.ivab.se (8.6.12/8.6.12) with SMTP id HAA18789; Thu, 7 Mar 1996 07:54:57 +0100 Received: by arnold.image.ivab.se; (5.65v3.2/1.1.8.2/16Nov95-0104AM) id AA22905; Thu, 7 Mar 1996 07:54:56 +0100 Date: Thu, 7 Mar 1996 07:54:56 +0100 Message-Id: <9603070654.AA22905@arnold.image.ivab.se> From: Fredrik Lundh To: spillar@uwyo.edu Cc: matrix-sig@python.org In-Reply-To: (message from Earl J Spillar on Wed, 06 Mar 1996 16:56:14 -0700 (MST)) Subject: Re: [PYTHON MATRIX-SIG] Efficient data reading Reply-To: fredrik_lundh@ivab.se Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk > Probably will be helpful. A lot of what astronomers call "image > processing" is actually numerical analysis though, so the other > Matrix stuff is very important. Indeed. One important future task is to provide tight interfaces between PIL and the matrix extensions. I'd rather not duplicate stuff like frequency domain filters, for example. On the other hand, a FITS reader should probably be part of PIL rather than some other module. I've already hacked a simple version, but haven't tested it much. Maybe your module could be plugged in instead? /F ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Mar 7 11:48:02 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA11163 for matrix-sig-people; Thu, 7 Mar 1996 11:48:02 -0500 Received: from roper.uwyo.edu ([129.72.60.8]) by python.org (8.6.12/8.6.12) with ESMTP id LAA11159; Thu, 7 Mar 1996 11:47:58 -0500 Received: from plains.uwyo.edu (plains.uwyo.edu) by ROPER.UWYO.EDU (PMDF V5.0-4 #14244) id <01I21W09COXC0081OX@ROPER.UWYO.EDU>; Thu, 07 Mar 1996 09:22:11 -0700 (MST) Received: from 129.238.158.202 ("port 3953"@129.238.158.202) by PLAINS.UWYO.EDU (PMDF V5.0-4 #14244) id <01I21W06YDOW002F1Q@PLAINS.UWYO.EDU>; Thu, 07 Mar 1996 09:22:09 -0700 (MST) Date: Thu, 07 Mar 1996 09:22:21 -0700 (MST) From: Earl J Spillar Subject: [PYTHON MATRIX-SIG] FITS images place in Module Space To: Python Matrix sig , Python Image sig Reply-to: spillar@uwyo.edu Message-id: MIME-version: 1.0 Content-type: text/plain; charset=US-ASCII; X-MAPIextension=".TXT" Content-transfer-encoding: 7bit Priority: Normal Sender: owner-matrix-sig@python.org Precedence: bulk Fredrik Lundh said- On the other hand, a FITS reader should probably be part of PIL rather than some other module. I've already hacked a simple version, but haven't tested it much. Maybe your module could be plugged in instead? This makes sense. I wasn't aware of PIL until these posts. I'm not sure my module is all that useful yet- it mostly deals with the header parts so far, and calls some legacy free standing executables for data operations. The Python Imaging Library, release 0.1, is nearly completed, and I hope to release it within a few weeks. I've even written a handbook for it... I'ld be very interested in seeing how the handbook! Earl Spillar spillar@uwyo.edu http://plains.uwyo.edu/~spillar/ ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Mar 7 16:39:54 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA13740 for matrix-sig-people; Thu, 7 Mar 1996 16:39:54 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id QAA13736 for ; Thu, 7 Mar 1996 16:39:49 -0500 Received: from kristen.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19) id AA26095; Thu, 7 Mar 96 13:39:30 PST Received: from kristen (localhost) by kristen.llnl.gov (5.x/SMI-SVR4) id AA09510; Thu, 7 Mar 1996 13:39:28 -0800 Message-Id: <313F5790.68C0@llnl.gov> Date: Thu, 07 Mar 1996 13:39:28 -0800 From: "Paul. Dubois" Organization: Lawrence Livermore National Laboratory X-Mailer: Mozilla 2.0 (X11; I; SunOS 5.4 sun4m) Mime-Version: 1.0 To: Konrad HINSEN Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Functions and names References: <199602202012.PAA17939@cyclone.ERE.UMontreal.CA> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk I finally got time to take a look at Konrad's memo about functions and names for the matrix class. I have no objection to it. Except, I still want to be able to distinguish those actions which modify self and those which return a new array, so that x.reshape(s) modifies x, (or else the only way to do it is assignment to x.shape) x.shaped(s) returns some new value. Likewise with transpose. In short, if the name is or is most likely to be interpreted as an imperative verb, it means that self is modified, and no value is returned. -- Paul F. Dubois, L-472 (510)-422-5426 Lawrence Livermore National Laboratory FAX (510)-423-9969 Livermore, CA 94550 dubois1@llnl.gov Consulting: PaulDubois@aol.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Mar 7 19:09:45 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id TAA00929 for matrix-sig-people; Thu, 7 Mar 1996 19:09:45 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id TAA00924 for ; Thu, 7 Mar 1996 19:09:40 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id TAA28722 (8.6.11/IDA-1.6); Thu, 7 Mar 1996 19:08:11 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id TAA15895; Thu, 7 Mar 1996 19:08:10 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id TAA21569; Thu, 7 Mar 1996 19:08:08 -0500 Date: Thu, 7 Mar 1996 19:08:08 -0500 Message-Id: <199603080008.TAA21569@cyclone.ERE.UMontreal.CA> To: dubois1@llnl.gov CC: matrix-sig@python.org In-reply-to: <313F5790.68C0@llnl.gov> (dubois1@llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] Functions and names From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > > I finally got time to take a look at Konrad's memo about functions > and names for the matrix class. I have no objection to it. > Except, I still want to be able to distinguish those actions which > modify self and those which return a new array, so that In my proposal there are no operations that modify an array. I am not sure they are needed at all. You can change elements by assignment with indexing, and you can assign to shape. In-place transposition is not essential, because transposition returns a reference anyway (which I discovered only today; I'll update my proposal to make this behaviour optional, as for reshape()). I'd like to establish the principle that with exception of indexing, no operation returns a reference by default. Making a copy leads to fewer surprises and is therefore the better default option. In-place operations are more important for add-on packages like linear algebra, and I agree that the naming scheme should make the distinction clear. For example, one could have invert(m) and inverse(m). But for other operations the choice is less clear, e.g. for singular value decomposition. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Mar 9 16:10:28 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA15478 for matrix-sig-people; Sat, 9 Mar 1996 16:10:28 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id QAA15474 for ; Sat, 9 Mar 1996 16:10:24 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA00330; Sat, 9 Mar 96 16:05:26 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) for matrix-sig@python.org id QAA09084; Sat, 9 Mar 1996 16:05:20 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199603092105.QAA09084@maigret> Subject: [PYTHON MATRIX-SIG] missing (?) feature To: matrix-sig@python.org Date: Sat, 9 Mar 1996 16:05:19 -0500 (EST) X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 753 Sender: owner-matrix-sig@python.org Precedence: bulk I came accross a feature I'd like to see in the "choose" command/function (or whatever else it gets called =): Basically, my problem is that I want to do something like: ranf = Ranf.ranf age = age + 1 x = age.greater(MAX_AGE).choose(x, ranf) y = age.greater(MAX_AGE).choose(y, ranf) age = age.greater(MAX_AGE).choose(age, 0) so that the x and y values of 'old' coordinates get replaced w/ new coordinates. The new coordinates are typically based on random numbers. With the current implementation I have to do: x = age.greater(MAX_AGE).choose(x, random_sample(NUM_DOTS)) which is wasteful since on a given iteration, only a few of the dots are likely to be "too old". But a new random_sample() array is generated each time anyway. --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Mar 11 12:54:51 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA02266 for matrix-sig-people; Mon, 11 Mar 1996 12:54:51 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id MAA02261 for ; Mon, 11 Mar 1996 12:54:46 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id JAA19777 (8.6.11/IDA-1.6); Mon, 11 Mar 1996 09:08:57 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id JAA08734; Mon, 11 Mar 1996 09:08:56 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id JAA11002; Mon, 11 Mar 1996 09:08:55 -0500 Date: Mon, 11 Mar 1996 09:08:55 -0500 Message-Id: <199603111408.JAA11002@cyclone.ERE.UMontreal.CA> To: da@maigret.cog.brown.edu CC: matrix-sig@python.org In-reply-to: <199603092105.QAA09084@maigret> (da@maigret.cog.brown.edu) Subject: Re: [PYTHON MATRIX-SIG] missing (?) feature From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > Basically, my problem is that I want to do something like: > > ranf = Ranf.ranf > age = age + 1 > x = age.greater(MAX_AGE).choose(x, ranf) > y = age.greater(MAX_AGE).choose(y, ranf) > age = age.greater(MAX_AGE).choose(age, 0) If you are really worried about efficiency, you should certainly not recalculate age.greater(MAX_AGE) three times ;-) > With the current implementation I have to do: > > x = age.greater(MAX_AGE).choose(x, random_sample(NUM_DOTS)) > > which is wasteful since on a given iteration, only a few of the dots are > likely to be "too old". But a new random_sample() array is generated > each time anyway. Which "waste" are you worried about? The waste in CPU time caused by the random generator? Unless your array is really big, this is probably not worth mentioning. On the other hand, a rather concise solution could be obtained if the idea of allowing general list indices is taken up again. Then you could write: update = repeat(greater(age, MAX_AGE), arrayRange(len(age))) x[update] = random_sample(len(update)) y[update] = random_sample(len(update)) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Mar 12 05:27:25 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id FAA09668 for matrix-sig-people; Tue, 12 Mar 1996 05:27:25 -0500 Received: from trulli.imc.akh-wien.ac.at (trulli.imc.akh-wien.ac.at [149.148.60.30]) by python.org (8.6.12/8.6.12) with SMTP id FAA09663 for ; Tue, 12 Mar 1996 05:26:50 -0500 Received: by trulli.imc.akh-wien.ac.at (AIX 3.2/UCB 5.64/4.03) id AA12133; Tue, 12 Mar 1996 11:25:04 GMT Date: Tue, 12 Mar 1996 11:25:04 GMT From: g.k@trulli.imc.akh-wien.ac.at (Guenter Kolousek) Message-Id: <9603121125.AA12133@trulli.imc.akh-wien.ac.at> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Extending NumericalPython by fuzzy numbers? Sender: owner-matrix-sig@python.org Precedence: bulk Hi everybody! I recently looked at the new NumericalPython. It looks great. For my purposes I need fuzzy numbers. Is it easy to extend the library by new functions and especially by new data types, like fuzzy numbers? There is just a little problem: No experience in extending Python with C functions! Is there a way for me to implement it with a reasonable effort and contribute it to the matrix lib? Is it possible to implement the extensions in C++? I appreciate any helpful answers. Guenter Kolousek -- ---------------------------------------------------- DI Guenter Kolousek Department for Medical Computer Sciences Medical School, University of Vienna mail: Spitalgasse 23, 1090 Vienna, Austria email: g.k@trulli.imc.akh-wien.ac.at phone: +43-1-40400/6664 fax: +43-1-40400/6667 ---------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Mar 12 11:40:25 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA11794 for matrix-sig-people; Tue, 12 Mar 1996 11:40:25 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id LAA11790 for ; Tue, 12 Mar 1996 11:40:20 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA16842; Tue, 12 Mar 96 11:38:41 EST Date: Tue, 12 Mar 96 11:38:41 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9603121638.AA16842@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: g.k@trulli.imc.akh-wien.ac.at Cc: matrix-sig@python.org In-Reply-To: <9603121125.AA12133@trulli.imc.akh-wien.ac.at> (g.k@trulli.imc.akh-wien.ac.at) Subject: Re: [PYTHON MATRIX-SIG] Extending NumericalPython by fuzzy numbers? Sender: owner-matrix-sig@python.org Precedence: bulk At the moment, I am probably the only person in the world who can extend the array module with new data types. This is obviously not the way I'd like things to be. If there is an important data type, with a nice implementation in C, it shouldn't take me much time to add the appropriate hooks. I happen to strongly dislike C++, so adding a C++ object is unlikely. On the other hand... How exactly would you define a fuzzy number? My guess is that a fuzzy array could probably be implemented fairly easily as a subclass of UserArray in python, and that you wouldn't take too much of a performance hit. Of course, this is just a guess. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Mar 12 14:01:26 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA13061 for matrix-sig-people; Tue, 12 Mar 1996 14:01:26 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id OAA13057 for ; Tue, 12 Mar 1996 14:01:22 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id MAA06750 (8.6.11/IDA-1.6); Tue, 12 Mar 1996 12:19:28 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id MAA19783; Tue, 12 Mar 1996 12:19:21 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id MAA10705; Tue, 12 Mar 1996 12:05:05 -0500 Date: Tue, 12 Mar 1996 12:05:05 -0500 Message-Id: <199603121705.MAA10705@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: g.k@trulli.imc.akh-wien.ac.at, matrix-sig@python.org In-reply-to: <9603121638.AA16842@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] Extending NumericalPython by fuzzy numbers? From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > At the moment, I am probably the only person in the world who can > extend the array module with new data types. This is obviously not Sorry to disappoint you, but what do you think about this: Python 1.3 (Mar 4 1996) [C] Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> from Numeric import * >>> from Derivatives import * >>> a = array([DerivVar(-2.9,0), DerivVar(3.2,1)]) >>> multiply.reduce(a) (-9.28, [3.2, -2.9]) As long as speed is not of prime importance, using general arrays with arbitrary numeric types seems a very acceptable solution to me. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Mar 13 10:08:18 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA20354 for matrix-sig-people; Wed, 13 Mar 1996 10:08:18 -0500 Received: from GS151.SP.CS.CMU.EDU (GS151.SP.CS.CMU.EDU [128.2.203.163]) by python.org (8.6.12/8.6.12) with SMTP id KAA20349 for ; Wed, 13 Mar 1996 10:08:12 -0500 Received: from LOCALHOST by GS151.SP.CS.CMU.EDU id aa25551; 13 Mar 96 10:07 EST To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] atanh Reply-To: dredish@CS.cmu.edu Date: Wed, 13 Mar 1996 10:07:15 -0500 Message-ID: <25549.826729635@GS151.SP.CS.CMU.EDU> From: David Redish Sender: owner-matrix-sig@python.org Precedence: bulk I have been unable to find atanh in the python Numeric libraries. Is it there? adr ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Mar 13 11:35:47 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA21210 for matrix-sig-people; Wed, 13 Mar 1996 11:35:47 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id LAA21204 for ; Wed, 13 Mar 1996 11:35:42 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id LAA07125 (8.6.11/IDA-1.6); Wed, 13 Mar 1996 11:33:16 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id LAA26892; Wed, 13 Mar 1996 11:33:15 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id LAA18578; Wed, 13 Mar 1996 11:33:14 -0500 Date: Wed, 13 Mar 1996 11:33:14 -0500 Message-Id: <199603131633.LAA18578@cyclone.ERE.UMontreal.CA> To: dredish@CS.cmu.edu CC: matrix-sig@python.org In-reply-to: <25549.826729635@GS151.SP.CS.CMU.EDU> (message from David Redish on Wed, 13 Mar 1996 10:07:15 -0500) Subject: Re: [PYTHON MATRIX-SIG] atanh From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > > I have been unable to find atanh in the python Numeric libraries. > Is it there? I can't find it either. There is one in cmath, which also works for floats, but of course not for arrays. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Mar 13 12:26:19 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA21668 for matrix-sig-people; Wed, 13 Mar 1996 12:26:19 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id MAA21663 for ; Wed, 13 Mar 1996 12:26:13 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id MAA08641 (8.6.11/IDA-1.6 for ); Wed, 13 Mar 1996 12:24:20 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id MAA02027; Wed, 13 Mar 1996 12:24:19 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id MAA23017; Wed, 13 Mar 1996 12:24:18 -0500 Date: Wed, 13 Mar 1996 12:24:18 -0500 Message-Id: <199603131724.MAA23017@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] ArrayPrinter.py From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk # Array printing function # # Written by Konrad Hinsen # last revision: 1996-3-13 # import sys from fast_umath import * import Numeric def arrayToString(a, max_line_width = None, precision = None, suppress_small = None): if len(a.shape) == 0: return str(a[0]) if max_line_width is None: try: max_line_width = sys.output_line_width except AttributeError: max_line_width = 77 if precision is None: try: precision = sys.float_output_precision except AttributeError: precision = 8 if suppress_small is None: try: suppress_small = sys.float_output_suppress_small except AttributeError: suppress_small = 0 if a.contiguous(): data = a.reshape(None) else: data = a.copy().reshape(None) type = a.typecode() items_per_line = a.shape[-1] if type == 'b' or type == '1' or type == 's' or type == 'i' \ or type == 'l': max_str_len = max(len(str(maximum.reduce(data))), len(str(minimum.reduce(data)))) format = '%' + str(max_str_len) + 'd' item_length = max_str_len+1 format_function = lambda x, f = format: _formatInteger(x, f) elif type == 'f' or type == 'd': format, item_length = _floatFormat(data, precision, suppress_small) format_function = lambda x, f = format: _formatFloat(x, f) elif type == 'F' or type == 'D': real_format, real_item_length = _floatFormat(data.real, precision, suppress_small, sign=0) imag_format, imag_item_length = _floatFormat(data.imaginary, precision, suppress_small, sign=1) item_length = real_item_length + imag_item_length + 2 format_function = lambda x, f1 = real_format, f2 = imag_format: \ _formatComplex(x, f1, f2) elif type == 'c': item_length = 1 format_function = lambda x: x elif type == 'O': item_length = max(map(lambda x: len(str(x)), data)) + 1 format_function = _formatGeneral else: return str(a) final_spaces = (type != 'c') line_width = item_length*items_per_line - final_spaces if line_width > max_line_width: indent = 6 if indent == item_length: indent = 8 items_first = (max_line_width+final_spaces)/item_length if items_first < 1: items_first = 1 items_continuation = (max_line_width+final_spaces-indent)/item_length if items_continuation < 1: items_continuation = 1 line_width = max(item_length*items_first, item_length*items_continuation+indent) - final_spaces number_of_lines = 1 + (items_per_line-items_first + items_continuation-1)/items_continuation line_format = (number_of_lines, items_first, items_continuation, indent, line_width) else: line_format = (1, items_per_line, 0, 0, line_width) return _arrayToString(a, format_function, len(a.shape), line_format)[:-1] def _floatFormat(data, precision, suppress_small, sign = 0): exp_format = 0 non_zero = abs(Numeric.compress(data.notEqual(0), data)) if len(non_zero) == 0: max_val = 0. min_val = 0. else: max_val = maximum.reduce(non_zero) min_val = minimum.reduce(non_zero) if max_val >= 1.e12: exp_format = 1 if not suppress_small and (min_val < 0.0001 or max_val/min_val > 1000.): exp_format = 1 if exp_format: large_exponent = 0 < min_val < 1e-99 or max_val >= 1e100 max_str_len = 8 + precision + large_exponent if sign: format = '%+' else: format = '%' format = format + str(max_str_len) + '.' + str(precision) + 'e' if large_exponent: format = format + '3' item_length = max_str_len + 1 else: format = '%.' + str(precision) + 'f' precision = min(precision, max(tuple(map(lambda x, p=precision, f=format: _digits(x,p,f), data)))) max_str_len = len(str(int(max_val))) + precision + 2 if sign: format = '%#+' else: format = '%#' format = format + str(max_str_len) + '.' + str(precision) + 'f' item_length = max_str_len + 1 return (format, item_length) def _digits(x, precision, format): s = format % x zeros = len(s) while s[zeros-1] == '0': zeros = zeros-1 return precision-len(s)+zeros def _arrayToString(a, format_function, rank, line_format): if rank == 0: return str(a[0]) elif rank == 1: s = '' items = line_format[1] indent = 0 index = 0 for j in range(line_format[0]): s = s + indent * ' ' for i in range(items): s = s + format_function(a[index]) index = index + 1 if index == a.shape[0]: break if s[-1] == ' ': s = s[:-1] s = s + '\n' items = line_format[2] indent = line_format[3] else: s = '' for i in range(a.shape[0]-1): s = s + _arrayToString(a[i], format_function, rank-1, line_format) if rank == 3: s = s + '\n' elif rank > 3: s = s + (rank-3)*(line_format[4]*'-'+'\n') s = s + _arrayToString(a[a.shape[0]-1], format_function, rank-1, line_format) return s def _formatInteger(x, format): return format % x + ' ' def _formatFloat(x, format, strip_zeros = 1): if format[-1] == '3': format = format[:-1] s = format % x third = s[-3] if third == '+' or third == '-': s = s[1:-2] + '0' + s[-2:] elif format[-1] == 'f': s = format % x if strip_zeros: zeros = len(s) while s[zeros-1] == '0': zeros = zeros-1 s = s[:zeros] + (len(s)-zeros)*' ' else: s = format % x return s + ' ' def _formatComplex(x, real_format, imag_format): r = _formatFloat(x.real, real_format)[:-1] i = _formatFloat(x.imag, imag_format, 0)[:-1] spaces = 0 while r[spaces] == ' ': spaces = spaces + 1 r = spaces*' ' + '(' + r[spaces:] if imag_format[-1] == 'f': zeros = len(i) while i[zeros-1] == '0': zeros = zeros-1 i = i[:zeros] + 'j)' + (len(i)-zeros)*' ' else: i = i + 'j)' return r + i + ' ' def _formatGeneral(x): return str(x) + ' ' ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Mar 13 12:33:53 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA21768 for matrix-sig-people; Wed, 13 Mar 1996 12:33:53 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id MAA21762 for ; Wed, 13 Mar 1996 12:33:49 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id MAA08816 (8.6.11/IDA-1.6 for ); Wed, 13 Mar 1996 12:31:58 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id MAA02686; Wed, 13 Mar 1996 12:31:58 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id MAA23007; Wed, 13 Mar 1996 12:23:51 -0500 Date: Wed, 13 Mar 1996 12:23:51 -0500 Message-Id: <199603131723.MAA23007@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] New array pretty printer From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk I have added two new features to the array printer, and I will send the new version to everyone in a separate message. Since the last NumericPython distribution did not contain the most recent version (there were still a few minor bugs), I recommend the current update to everyone, even those who are not interested in the new features. The first new feature is support for general object arrays. This is rather elementary (str() is called on each element, and the resulting strings are printed in a neat arrangement), but it is very useful and also sufficient for any general array with reasonable elements. Of course in extreme cases (e.g. when str() for one element returns a multi-line string) the array output will be a complete mess. The second new feature is an additional optional parameter suppress_small that defaults to "false". If set to true (1), then elements with a very small absolute value in float arrays will not cause a switch to exponential format (they will be printed as zeros). This results in a much neater output if these elements are in fact "numerical zeros", but of course some information is lost. I'd also like to propose to change the name of the module "PrettyPrinter" to "ArrayPrinter". After someone sent me another module called "PrettyPrinter" (which prints lists etc. in a nice way), I had some problems for a while until I found out what went wrong. To avoid such problems it makes sense to choose a very specific name for this module, and "ArrayPrinter" is certainly much more specific than "PrettyPrinter". ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Mar 13 15:58:57 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA23604 for matrix-sig-people; Wed, 13 Mar 1996 15:58:57 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id PAA23599 for ; Wed, 13 Mar 1996 15:58:50 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA09651; Wed, 13 Mar 96 15:58:14 EST Date: Wed, 13 Mar 96 15:58:14 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9603132058.AA09651@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: hinsenk@ere.umontreal.ca Cc: dredish@CS.cmu.edu, matrix-sig@python.org In-Reply-To: <199603131633.LAA18578@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Subject: Re: [PYTHON MATRIX-SIG] atanh Sender: owner-matrix-sig@python.org Precedence: bulk This is an obvious oversight. As far as I can tell, the inverse hyperbolic trig functions are part of the standard math library for doubles, and Konrad's provided them for complex numbers, so I'll just add a couple lines to my ufunc generator code and they'll be in the next version. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Mar 14 17:45:45 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA32636 for matrix-sig-people; Thu, 14 Mar 1996 17:45:45 -0500 Received: from lemur.magnet.com (davem@lemur.magnet.com [199.125.236.2]) by python.org (8.6.12/8.6.12) with ESMTP id RAA32632 for ; Thu, 14 Mar 1996 17:45:39 -0500 Received: (from davem@localhost) by lemur.magnet.com (8.6.12/MAGNET) id RAA19827 for matrix-sig@python.org; Thu, 14 Mar 1996 17:43:48 -0500 From: Dave Mitchell Message-Id: <199603142243.RAA19827@lemur.magnet.com> Subject: [PYTHON MATRIX-SIG] matrix module To: matrix-sig@python.org Date: Thu, 14 Mar 1996 17:43:48 -0500 (EST) X-Mailer: ELM [version 2.4 PL24 PGP3 *ALPHA*] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 127 Sender: owner-matrix-sig@python.org Precedence: bulk > > > Could someone please give me an url for the current matrix stuff? > > thanks-Dave > davem@magnet.com ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Mar 14 20:04:43 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id UAA00662 for matrix-sig-people; Thu, 14 Mar 1996 20:04:43 -0500 Received: from netcom4.netcom.com (root@netcom4.netcom.com [192.100.81.107]) by python.org (8.6.12/8.6.12) with ESMTP id UAA00658 for ; Thu, 14 Mar 1996 20:04:38 -0500 Received: from [10.0.2.15] by netcom4.netcom.com (8.6.13/Netcom) id RAA09393; Thu, 14 Mar 1996 17:03:47 -0800 X-Sender: demars@10.0.2.2 Message-Id: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Thu, 14 Mar 1996 17:03:53 -0800 To: matrix-sig@python.org From: demars@netcom.com (Dennis C. De Mars) Subject: Re: [PYTHON MATRIX-SIG] matrix module Sender: owner-matrix-sig@python.org Precedence: bulk >> >> >> Could someone please give me an url for the current matrix stuff? >> >> thanks-Dave >> davem@magnet.com > > >================= >MATRIX-SIG - SIG on Matrix Math for Python > >send messages to: matrix-sig@python.org >administrivia to: matrix-sig-request@python.org >================= Same here; perhaps the location could be posted to the mailing list for those of us who have signed up recently? - Dennis D. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Mar 16 14:55:07 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA12953 for matrix-sig-people; Sat, 16 Mar 1996 14:55:07 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id OAA12949 for ; Sat, 16 Mar 1996 14:55:02 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA15069; Sat, 16 Mar 96 14:49:19 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) for matrix-sig@python.org id OAA27675; Sat, 16 Mar 1996 14:49:17 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199603161949.OAA27675@maigret> Subject: [PYTHON MATRIX-SIG] Any interest in PC binaries? To: matrix-sig@python.org Date: Sat, 16 Mar 1996 14:49:16 -0500 (EST) X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 748 Sender: owner-matrix-sig@python.org Precedence: bulk I've recently gotten a PC, so the first thing I've done (naturally!) is to get the Numeric extension to work with Python under Windows95, along w/ the alpha OpenGL, GLU and Glut extensions. I will also be looking into integrating the Numeric extension w/ Jim Ahlstrom's DOS version (though clearly not the OpenGL stuff), as I hate having a ">>>" prompt but being unable to do "from Numeric import *". Would there be any interest in my making binaries of these ports available, or is everyone on the list a pure unixhead? --david PS: Thanks to Python, the boundary between unix workstations and pc's is blurring -- a fast pentium w/ Python and OpenGL is looking more and more like my (r3000) SGI at school. Well, actually, it's faster. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Mar 16 15:12:22 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA13087 for matrix-sig-people; Sat, 16 Mar 1996 15:12:22 -0500 Received: from phoenix.geog.ubc.ca (phoenix.geog.ubc.ca [137.82.51.1]) by python.org (8.6.12/8.6.12) with ESMTP id PAA13083 for ; Sat, 16 Mar 1996 15:12:15 -0500 Received: from whimbrel.geog.ubc.ca (whimbrel [137.82.51.7]) by phoenix.geog.ubc.ca (8.7.3/8.7.3) with ESMTP id MAA08642 for ; Sat, 16 Mar 1996 12:10:56 -0800 (PST) Received: (phil@localhost) by whimbrel.geog.ubc.ca (8.6.12/8.6.12) id MAA07296; Sat, 16 Mar 1996 12:10:55 -0800 Date: Sat, 16 Mar 1996 12:10:55 -0800 Message-Id: <199603162010.MAA07296@whimbrel.geog.ubc.ca> From: Phil Austin To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Any interest in PC binaries? In-Reply-To: <199603161949.OAA27675@maigret> References: <199603161949.OAA27675@maigret> Sender: owner-matrix-sig@python.org Precedence: bulk David Ascher writes: > I've recently gotten a PC, so the first thing I've done (naturally!) > is to get the Numeric extension to work with Python under Windows95, > along w/ the alpha OpenGL, GLU and Glut extensions. I will also be > looking into integrating the Numeric extension w/ Jim Ahlstrom's DOS > version (though clearly not the OpenGL stuff), as I hate having a ">>>" > prompt but being unable to do "from Numeric import *". > > Would there be any interest in my making binaries of these ports > available, or is everyone on the list a pure unixhead? > > --david One of the main attractions of Python (to us) is the cross-platform development, and we'd appreciate access to the binaries. Our hope is that it can be used as an Octave/Matlab-like teaching tool in our undergraduate PC lab. Regards, Phil Austin INTERNET: phil@geog.ubc.ca (604) 822-2175 FAX: (604) 822-6150 http://www.geog.ubc.ca/~phil Associate Professor Atmospheric Sciences Programme Geography #217 University of British Columbia 1984 W Mall Vancouver, BC V6T 1Z2 CANADA ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Mar 17 00:07:12 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id AAA15610 for matrix-sig-people; Sun, 17 Mar 1996 00:07:12 -0500 Received: from netcom4.netcom.com (root@netcom4.netcom.com [192.100.81.107]) by python.org (8.6.12/8.6.12) with ESMTP id AAA15606 for ; Sun, 17 Mar 1996 00:07:02 -0500 Received: from [10.0.2.15] by netcom4.netcom.com (8.6.13/Netcom) id VAA00180; Sat, 16 Mar 1996 21:05:42 -0800 X-Sender: demars@10.0.2.2 Message-Id: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Sat, 16 Mar 1996 21:05:47 -0800 To: matrix-sig@python.org From: demars@netcom.com (Dennis C. De Mars) Subject: Re: [PYTHON MATRIX-SIG] matrix module Sender: owner-matrix-sig@python.org Precedence: bulk >>> >>> >>> Could someone please give me an url for the current matrix stuff? >>> >>> thanks-Dave >>> davem@magnet.com >> >> > >Same here; perhaps the location could be posted to the mailing list for >those of us who have signed up recently? > >- Dennis D. > As Rosanne Rosanadana used to say, "Never mind." I discovered that an archive of the entire content of the mailing list (as well as the other Python lists) is available from Majordomo, which contains the desired URL plus all of the discussion of Numerical Python which provides very useful background -- now I won't feel as if I've come in on the middle of a conversation while reading the traffic on this SIG. I would have realized this was available if I'd read the info on the Python web site more carefully. Anyhow, I suggest to any new subscribers that haven't already done it that they download the archive for matrix-sig -- over a megabyte and well worth the effort (...and my apologies to other members of the SIG who will find this message superfluous, but I think there may be a few other subscribers out there who are as confused as I was and will find this helpful). - Dennis D. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Mar 17 23:06:13 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id XAA21503 for matrix-sig-people; Sun, 17 Mar 1996 23:06:13 -0500 Received: from galaxies.plk.af.mil (ip20.abq-dialin.hollyberry.com [165.247.201.20]) by python.org (8.6.12/8.6.12) with SMTP id XAA21498 for ; Sun, 17 Mar 1996 23:05:57 -0500 Received: by galaxies.plk.af.mil (NX5.67d/NX3.0M) id AA00503; Sun, 17 Mar 96 21:03:48 -0700 Date: Sun, 17 Mar 96 21:03:48 -0700 From: Earl Spillar Message-Id: <9603180403.AA00503@galaxies.plk.af.mil> Received: by NeXT.Mailer (1.100) Received: by NeXT Mailer (1.100) To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] PC binaries Reply-To: spillar@uwyo.edu Sender: owner-matrix-sig@python.org Precedence: bulk David Ascher asked: "Would there be any interest in my making binaries of these ports available, or is everyone on the list a pure unixhead?" Well, I'ld like to be a pure unix head, but the reality is I need to work with people who aren't- so I would like to have the ports available! Earl Spillar spillar@uwyo.edu http://plains.uwyo.edu/~spillar/ ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Mar 19 06:30:29 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id GAA32465 for matrix-sig-people; Tue, 19 Mar 1996 06:30:29 -0500 Received: from sol.cc.u-szeged.hu (sol.cc.u-szeged.hu [160.114.8.24]) by python.org (8.6.12/8.6.12) with SMTP id GAA32460 for ; Tue, 19 Mar 1996 06:30:23 -0500 Received: from szbk.u-szeged.hu (rosi.szbk.u-szeged.hu) by sol.cc.u-szeged.hu (5.0/SMI-SVR4) id AA02777; Tue, 19 Mar 1996 12:23:56 +0100 Received: from everx.szbk.u-szeged.hu by szbk.u-szeged.hu (5.0/SMI-SVR4) id AA22880; Tue, 19 Mar 1996 12:27:08 --100 Received: from SZBK_EVERX/MERCURY by everx.szbk.u-szeged.hu (Mercury 1.21); 19 Mar 96 12:24:00 MET DST Received: from MERCURY by SZBK_EVERX (Mercury 1.21); 19 Mar 96 12:23:43 MET DST From: "Groma Geza" Organization: Biological Research Center To: matrix-sig@python.org Date: Tue, 19 Mar 1996 12:23:34 MET Subject: [PYTHON MATRIX-SIG] pickle error on Win95 Priority: normal X-Mailer: Pegasus Mail for Windows (v2.01) Message-Id: <9A3698A0A36@everx.szbk.u-szeged.hu> Content-Length: 1204 Sender: owner-matrix-sig@python.org Precedence: bulk I built v0.35 on Win95 by Symantec C++ 7.3. No compilation error occured. I realized that TestSuite was not updated to this version. The following error, however, probably is not related to that. Running TestPickle resulted in: pickle.load(fp) -> Bad result: EOFError: None Traceback (innermost last): File "", line 1, in ? File "c:\Python\Lib\ArrayTest\TestPickle.py", line 31, in ? do_eval(test, initialize = init) File "c:\Python\Lib\ArrayTest\TestUtils.py", line 51, in do_eval raise ValueError, "Bad result" ValueError: Bad result Is that a bug in the pickling routines, or can be fixed by a proper compiler flag? This is the hex dump of file 'matrix.save' created by TestPickle: 4D 69 4C 34 20 32 20 33 20 34 20 0D 0A 00 00 00 00 01 00 00 00 02 00 00 00 03 00 00 00 0D 0A 00 00 00 0B 00 00 00 0C 00 00 00 0D 00 00 00 14 00 00 00 15 00 00 00 16 00 00 00 17 00 00 00 64 00 00 00 65 00 00 00 66 00 00 00 67 00 00 00 6E 00 00 00 6F 00 00 00 70 00 00 00 71 00 00 00 78 00 00 00 79 00 00 00 7A 00 00 00 7B 00 00 00 2E Geza I. Groma Institute of Biophysics Biological Research Centre of Hungarian Academy of Sciences Szeged, Hungary ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Mar 19 08:22:29 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id IAA00305 for matrix-sig-people; Tue, 19 Mar 1996 08:22:29 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id IAA00301 for ; Tue, 19 Mar 1996 08:22:23 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA19874; Tue, 19 Mar 96 08:16:08 EST Date: Tue, 19 Mar 96 08:16:08 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9603191316.AA19874@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: GROMA@everx.szbk.u-szeged.hu Cc: matrix-sig@python.org In-Reply-To: <9A3698A0A36@everx.szbk.u-szeged.hu> (GROMA@everx.szbk.u-szeged.hu) Subject: Re: [PYTHON MATRIX-SIG] pickle error on Win95 Sender: owner-matrix-sig@python.org Precedence: bulk Your hex dump looks right to me, so I'd guess that the error is something else. I'm surprised that v0.35 compiled on Win95 without any compilation errors. I haven't yet implemented any of the patches that others have told me are needed to get things to compile under windows. Did you have to make any changes to the code? -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Mar 19 08:47:51 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id IAA00508 for matrix-sig-people; Tue, 19 Mar 1996 08:47:51 -0500 Received: from estel.uindy.edu (ESTEL.UINDY.EDU [192.146.191.197]) by python.org (8.6.12/8.6.12) with SMTP id IAA00504 for ; Tue, 19 Mar 1996 08:47:45 -0500 Received: by estel.uindy.edu (NX5.67d/NX3.0M) id AA03021; Tue, 19 Mar 96 08:49:12 -0500 Date: Tue, 19 Mar 96 08:49:12 -0500 From: Steve Spicklemire Message-Id: <9603191349.AA03021@estel.uindy.edu> To: jjh@Goldilocks.LCS.MIT.EDU Cc: GROMA@everx.szbk.u-szeged.hu, matrix-sig@python.org In-Reply-To: <9603191316.AA19874@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] pickle error on Win95 Sender: owner-matrix-sig@python.org Precedence: bulk Jim, >>>>> "James" == James Hugunin writes: James> Your hex dump looks right to me, so I'd guess that the James> error is something else. James> I'm surprised that v0.35 compiled on Win95 without any James> compilation errors. I haven't yet implemented any of the James> patches that others have told me are needed to get things James> to compile under windows. Did you have to make any changes James> to the code? Note that he's using the Symantec compiler, rather than MSoft. I don't have that compiler, so I can't be sure, but all of the patches I made could've easily been OK on another compiler (e.g., MSoft seems to define 'complex', and M_PI was not defined.. etc.) -steve James> -Jim James> ================= MATRIX-SIG - SIG on Matrix Math for James> Python James> send messages to: matrix-sig@python.org administrivia to: James> matrix-sig-request@python.org ================= ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Mar 19 17:15:30 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id RAA00635 for matrix-sig-people; Tue, 19 Mar 1996 17:15:30 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id RAA00631 for ; Tue, 19 Mar 1996 17:15:25 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id RAA13136 (8.6.11/IDA-1.6 for ); Tue, 19 Mar 1996 17:13:43 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id RAA11909; Tue, 19 Mar 1996 17:13:43 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id RAA23768; Tue, 19 Mar 1996 17:13:42 -0500 Date: Tue, 19 Mar 1996 17:13:42 -0500 Message-Id: <199603192213.RAA23768@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Wanted: testers for a module From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk This has no direct relation to the Matrix SIG, but since the code I am talking about works only with the numerics package, I might just as well restrict the distribution to those who have it. I have written a module that handles physical quantities with units. It allows all reasonable mathematical operations on them (while checking unit compatibility and doing automatic unit conversions) and can convert between compatible units. I am looking for people willing to test this. The code itself is not so much of a problem, but the biggest part of the file is the unit table, and I am not 100% sure that all entries are correct and that my collection is sufficiently complete. Anyone interested please mail me. Oh yes, there is a relation to arrays: you can use arrays for the values if you want to :-) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Mar 20 05:49:18 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id FAA05325 for matrix-sig-people; Wed, 20 Mar 1996 05:49:18 -0500 Received: from sol.cc.u-szeged.hu (sol.cc.u-szeged.hu [160.114.8.24]) by python.org (8.6.12/8.6.12) with SMTP id FAA05321 for ; Wed, 20 Mar 1996 05:49:09 -0500 Received: from szbk.u-szeged.hu (rosi.szbk.u-szeged.hu) by sol.cc.u-szeged.hu (5.0/SMI-SVR4) id AA02151; Wed, 20 Mar 1996 11:49:25 +0100 Received: from everx.szbk.u-szeged.hu by szbk.u-szeged.hu (5.0/SMI-SVR4) id AA28477; Wed, 20 Mar 1996 11:52:47 --100 Received: from SZBK_EVERX/MERCURY by everx.szbk.u-szeged.hu (Mercury 1.21); 20 Mar 96 11:49:29 MET DST Received: from MERCURY by SZBK_EVERX (Mercury 1.21); 20 Mar 96 11:49:22 MET DST From: "Groma Geza" Organization: Biological Research Center To: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Date: Wed, 20 Mar 1996 11:49:19 MET Subject: Re: [PYTHON MATRIX-SIG] pickle error on Win95 Cc: matrix-sig@python.org Priority: normal X-Mailer: Pegasus Mail for Windows (v2.01) Message-Id: <9BAD7A75EB3@everx.szbk.u-szeged.hu> Content-Length: 1510 Sender: owner-matrix-sig@python.org Precedence: bulk > Your hex dump looks right to me, so I'd guess that the error is > something else. Meanwhile I fround that pickle.load() fails if the value of any element of an integer array is 13 (CR!?). This is demonstrated in the following example: --------------------------------------------------- >>> import pickle >>> from Numeric import * >>> def CreateBug(m): ... fp=open("test","w") ... pickle.dump(m,fp) ... fp.close() ... fp=open("test","r") ... y=pickle.load(fp) ... print y ... >>> CreateBug(array([12,14,15])) 12 14 15 >>> CreateBug(array([12,13,14])) Traceback (innermost last): File "", line 1, in ? File "", line 6, in CreateBug File "c:\Python\Lib\pickle.py", line 533, in load return Unpickler(file).load() File "c:\Python\Lib\pickle.py", line 375, in load self.dispatch[key](self) File "c:\Python\Lib\pickle.py", line 387, in load_eof raise EOFError EOFError >>> ---------------------------------------------------------------------- If you have any idea how to fix this error, I am eager to check it on my system. > I'm surprised that v0.35 compiled on Win95 without any compilation > errors. I haven't yet implemented any of the patches that others have > told me are needed to get things to compile under windows. Did you > have to make any changes to the code? Absolutely no change was necessery for Symantec C. Geza I. Groma Institute of Biophysics Biological Research Centre of Hungarian Academy of Sciences Szeged, Hungary ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Mar 20 06:08:30 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id GAA05407 for matrix-sig-people; Wed, 20 Mar 1996 06:08:30 -0500 Received: from sol.cc.u-szeged.hu (sol.cc.u-szeged.hu [160.114.8.24]) by python.org (8.6.12/8.6.12) with SMTP id GAA05383 for ; Wed, 20 Mar 1996 06:04:08 -0500 Received: from szbk.u-szeged.hu (rosi.szbk.u-szeged.hu) by sol.cc.u-szeged.hu (5.0/SMI-SVR4) id AA03629; Wed, 20 Mar 1996 12:04:03 +0100 Received: from everx.szbk.u-szeged.hu by szbk.u-szeged.hu (5.0/SMI-SVR4) id AA28544; Wed, 20 Mar 1996 12:07:24 --100 Received: from SZBK_EVERX/MERCURY by everx.szbk.u-szeged.hu (Mercury 1.21); 20 Mar 96 12:04:07 MET DST Received: from MERCURY by SZBK_EVERX (Mercury 1.21); 20 Mar 96 12:03:52 MET DST From: "Groma Geza" Organization: Biological Research Center To: Steve Spicklemire Date: Wed, 20 Mar 1996 12:03:51 MET Subject: Re: [PYTHON MATRIX-SIG] pickle error on Win95 Cc: matrix-sig@python.org Priority: normal X-Mailer: Pegasus Mail for Windows (v2.01) Message-Id: <9BB158C6B31@everx.szbk.u-szeged.hu> Content-Length: 527 Sender: owner-matrix-sig@python.org Precedence: bulk > Note that he's using the Symantec compiler, rather than MSoft. I don't > have that compiler, so I can't be sure, but all of the patches I made could've > easily been OK on another compiler (e.g., MSoft seems to define 'complex', and > M_PI was not defined.. etc.) I offer my help in checking your patches on Symantec C. Could you send me the code and the list of compiler errors you have got with MSoft? Geza I. Groma Institute of Biophysics Biological Research Centre of Hungarian Academy of Sciences Szeged, Hungary ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Mar 20 06:09:05 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id GAA05419 for matrix-sig-people; Wed, 20 Mar 1996 06:09:05 -0500 Received: from catch.ivab.se (root@catch.ivab.se [193.180.173.22]) by python.org (8.6.12/8.6.12) with ESMTP id GAA05415 for ; Wed, 20 Mar 1996 06:09:01 -0500 Received: from arnold.image.ivab.se (arnold.image.ivab.se [193.180.173.119]) by catch.ivab.se (8.6.12/8.6.12) with SMTP id MAA09848; Wed, 20 Mar 1996 12:08:36 +0100 Received: by arnold.image.ivab.se; (5.65v3.2/1.1.8.2/16Nov95-0104AM) id AA12490; Wed, 20 Mar 1996 12:08:34 +0100 Date: Wed, 20 Mar 1996 12:08:34 +0100 Message-Id: <9603201108.AA12490@arnold.image.ivab.se> From: Fredrik Lundh To: GROMA@everx.szbk.u-szeged.hu Cc: jjh@Goldilocks.LCS.MIT.EDU, matrix-sig@python.org In-Reply-To: <9BAD7A75EB3@everx.szbk.u-szeged.hu> (GROMA@everx.szbk.u-szeged.hu) Subject: Re: [PYTHON MATRIX-SIG] pickle error on Win95 Reply-To: fredrik_lundh@ivab.se Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk ... fp=open("test","w") ... pickle.dump(m,fp) ... fp.close() ... fp=open("test","r") ... y=pickle.load(fp) ... print y Hey, you've forgot the "b" flag! The following should work: ... fp=open("test","wb") ... pickle.dump(m,fp) ... fp.close() ... fp=open("test","rb") ... y=pickle.load(fp) Since text and binary files are not the same thing under DOS (and some other systems as well), you should always make sure to add "b" when dealing with binary data. See open() in the python manual for details. /F ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Mar 20 10:12:48 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA06952 for matrix-sig-people; Wed, 20 Mar 1996 10:12:48 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id KAA06948 for ; Wed, 20 Mar 1996 10:12:45 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa07012; 20 Mar 96 10:05 EST Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id KAA07841; Wed, 20 Mar 1996 10:05:00 -0500 Message-Id: <199603201505.KAA07841@monty> To: fredrik_lundh@ivab.se cc: GROMA@everx.szbk.u-szeged.hu, jjh@goldilocks.lcs.mit.edu, matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] pickle error on Win95 In-reply-to: Your message of "Wed, 20 Mar 1996 12:08:34 +0100." <9603201108.AA12490@arnold.image.ivab.se> References: <9603201108.AA12490@arnold.image.ivab.se> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Wed, 20 Mar 1996 10:04:59 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk > Hey, you've forgot the "b" flag! The following should work: [...] > Since text and binary files are not the same thing under DOS (and > some other systems as well), you should always make sure to add "b" > when dealing with binary data. See open() in the python manual for > details. On the other hand, pickle *used* to be a text-only format, which is fine with 'r' and 'b'. But I presume that numerical arrays may be written in a more efficient binary form? --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Mar 20 10:23:32 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA07073 for matrix-sig-people; Wed, 20 Mar 1996 10:23:32 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id KAA07068 for ; Wed, 20 Mar 1996 10:23:29 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA06306; Wed, 20 Mar 96 10:22:55 EST Date: Wed, 20 Mar 96 10:22:55 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9603201522.AA06306@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: guido@CNRI.Reston.VA.US Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] pickle error on Win95 Sender: owner-matrix-sig@python.org Precedence: bulk > Hey, you've forgot the "b" flag! The following should work: [...] > Since text and binary files are not the same thing under DOS (and > some other systems as well), you should always make sure to add "b" > when dealing with binary data. See open() in the python manual for > details. On the other hand, pickle *used* to be a text-only format, which is fine with 'r' and 'b'. But I presume that numerical arrays may be written in a more efficient binary form? --Guido van Rossum URL: I've been waiting for somebody to notice this and properly castigate me for violating the beauty of pickle's text-only format. There are some major size and speed savings to be realized by dumping large arrays in a binary format. Since dumping 1 MB arrays to disk is not all that uncommon, these things are unfortunately an issue. This is the first time that anybody's noticed problems with the binary format, and it seems a simple enough matter to add warnings that matrices should only be pickled to binary format files. Other than this, I've found the binary format to work quite well for archiving arbitrary objects that contain matrices across many platforms (big/little endian) and even for passing these objects across sockets. Anybody know a way I can add a check to the matrix pickling routines to determine if they're writing to/reading from a binary format file and raise an exception otherwise? -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Mar 20 10:33:58 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA07226 for matrix-sig-people; Wed, 20 Mar 1996 10:33:58 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id KAA07222 for ; Wed, 20 Mar 1996 10:33:55 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa07458; 20 Mar 96 10:27 EST Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id KAA08036; Wed, 20 Mar 1996 10:27:15 -0500 Message-Id: <199603201527.KAA08036@monty> To: James Hugunin cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] pickle error on Win95 In-reply-to: Your message of "Wed, 20 Mar 1996 10:22:55 EST." <9603201522.AA06306@baribal.LCS.MIT.EDU.LCS.MIT.EDU> References: <9603201522.AA06306@baribal.LCS.MIT.EDU.LCS.MIT.EDU> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Wed, 20 Mar 1996 10:27:15 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk > I've been waiting for somebody to notice this and properly castigate > me for violating the beauty of pickle's text-only format. > > There are some major size and speed savings to be realized by dumping > large arrays in a binary format. Since dumping 1 MB arrays to disk is > not all that uncommon, these things are unfortunately an issue. Agreed. For large strings, I was going to add this (eventually) as a speed up extension as well. > This is the first time that anybody's noticed problems with the binary > format, and it seems a simple enough matter to add warnings that > matrices should only be pickled to binary format files. Other than > this, I've found the binary format to work quite well for archiving > arbitrary objects that contain matrices across many platforms > (big/little endian) and even for passing these objects across sockets. You must've patched the pickle module, right? > Anybody know a way I can add a check to the matrix pickling routines > to determine if they're writing to/reading from a binary format file > and raise an exception otherwise? That's easy. A file object has an attribute 'mode' which is the read/write open mode. Of course, pickle can also be used on pseudo file objects (like StringIO) which don't have this attribute. --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Mar 20 12:07:57 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA08168 for matrix-sig-people; Wed, 20 Mar 1996 12:07:57 -0500 Received: from oasis.unl.edu (oasis.unl.edu [129.93.34.17]) by python.org (8.6.12/8.6.12) with ESMTP id MAA08164 for ; Wed, 20 Mar 1996 12:07:53 -0500 Received: by oasis.unl.edu (950911.SGI.8.6.12.PATCH825/940406.SGI) for matrix-sig@python.org id LAA23052; Wed, 20 Mar 1996 11:07:36 -0600 From: drh@oasis.unl.edu (Doug Heisterkamp) Message-Id: <199603201707.LAA23052@oasis.unl.edu> Subject: [PYTHON MATRIX-SIG] LAPACK access module v.0.02 is available To: matrix-sig@python.org Date: Wed, 20 Mar 1996 11:07:35 -0600 (CST) X-Mailer: ELM [version 2.4 PL23] Content-Type: text Content-Length: 624 Sender: owner-matrix-sig@python.org Precedence: bulk Hi, The second version of the python modules for the low-level access to the LAPACK libraries is now available at ftp.cs.unl.edu in /pub/drh/python/pylapack.0.02.tar.gz Some of the changes from version 0.01 * Removed reference to f2c.h -- easier to link to fortran libraries * Fixed a mistake in reference counting. * Doc strings have been removed. * Modules have been split up based on first three letters of function names. * Functions return an exception if the arrays are not contiguous or of the wrong type * LinAlg.py from Konrad Hinsen has been added to the distribution. Doug Heisterkamp drh@oasis.unl.edu ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Mar 21 10:51:44 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA00780 for matrix-sig-people; Thu, 21 Mar 1996 10:51:44 -0500 Received: from sol.cc.u-szeged.hu (sol.cc.u-szeged.hu [160.114.8.24]) by python.org (8.6.12/8.6.12) with SMTP id KAA00733 for ; Thu, 21 Mar 1996 10:38:40 -0500 Received: from szbk.u-szeged.hu (rosi.szbk.u-szeged.hu) by sol.cc.u-szeged.hu (5.0/SMI-SVR4) id AA22416; Thu, 21 Mar 1996 16:15:51 +0100 Received: from everx.szbk.u-szeged.hu by szbk.u-szeged.hu (5.0/SMI-SVR4) id AA07989; Thu, 21 Mar 1996 16:18:59 --100 Received: from SZBK_EVERX/MERCURY by everx.szbk.u-szeged.hu (Mercury 1.21); 21 Mar 96 16:15:55 MET DST Received: from MERCURY by SZBK_EVERX (Mercury 1.21); 21 Mar 96 16:15:33 MET DST From: "Groma Geza" Organization: Biological Research Center To: Steve Spicklemire Date: Thu, 21 Mar 1996 16:15:30 MET Subject: [PYTHON MATRIX-SIG] compiling v0.35 by Symantec C Cc: matrix-sig@python.org Priority: normal X-Mailer: Pegasus Mail for Windows (v2.01) Message-Id: <9D7483D064B@everx.szbk.u-szeged.hu> Content-Length: 1550 Sender: owner-matrix-sig@python.org Precedence: bulk Steve Spicklemire wrote: > Here is my original note to Jim. I also have diffs around > somewhere. > > -steve > > -------- > > OK... well it took a lot longer than I would've figured > to get python compiling and running under windows.. but > once I got that.. the matrix module worked without much > change. The one that stumped me for a bit was endless > complaints about the 'complex' type. After much gnashing > of teeth it turns out that MSoft defines it's own complex > type in so..... if you include "mymath.h" before > "allobjects.h" in any module/object that deals with the complex > type, then all is well. Math.h of Symantec C includes complex in the form: struct complex { double x,y; }; This did not conflict with Konrad Hinsen's allobjects.h and complexobject.h distributed in v0.35. > Also, the Windows port needs > > HAVE_HYPOT > > defined in config.h for the math modules to build without > complaint. I used config.h distributed in the latest version of PythonWin. That defines HAVE_HYPOT. > Finally, M_PI needs to be defined as well. Symantec math.h includes #define PI 3.14159265358979323846 #define M_PI PI In addition M_PI is defined in umathmodule.c, fast_umathmodule.c and cmathmodule.c of v0.35. Geza I. Groma Institute of Biophysics Biological Research Centre of Hungarian Academy of Sciences Szeged, Hungary ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Mar 21 10:56:51 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA00827 for matrix-sig-people; Thu, 21 Mar 1996 10:56:51 -0500 Received: from sol.cc.u-szeged.hu (sol.cc.u-szeged.hu [160.114.8.24]) by python.org (8.6.12/8.6.12) with SMTP id KAA00819 for ; Thu, 21 Mar 1996 10:56:43 -0500 Received: from szbk.u-szeged.hu (rosi.szbk.u-szeged.hu) by sol.cc.u-szeged.hu (5.0/SMI-SVR4) id AA14257; Thu, 21 Mar 1996 14:51:51 +0100 Received: from everx.szbk.u-szeged.hu by szbk.u-szeged.hu (5.0/SMI-SVR4) id AA07007; Thu, 21 Mar 1996 14:54:04 --100 Received: from SZBK_EVERX/MERCURY by everx.szbk.u-szeged.hu (Mercury 1.21); 21 Mar 96 14:51:54 MET DST Received: from MERCURY by SZBK_EVERX (Mercury 1.21); 21 Mar 96 14:50:37 MET DST From: "Groma Geza" Organization: Biological Research Center To: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Date: Thu, 21 Mar 1996 14:50:36 MET Subject: Re: [PYTHON MATRIX-SIG] pickle error on Win95 Cc: matrix-sig@python.org Priority: normal X-Mailer: Pegasus Mail for Windows (v2.01) Message-Id: <9D5DDCA3BB6@everx.szbk.u-szeged.hu> Content-Length: 1589 Sender: owner-matrix-sig@python.org Precedence: bulk > I've been waiting for somebody to notice this and properly castigate > me for violating the beauty of pickle's text-only format. > > There are some major size and speed savings to be realized by dumping > large arrays in a binary format. Since dumping 1 MB arrays to disk is > not all that uncommon, these things are unfortunately an issue. > > This is the first time that anybody's noticed problems with the binary > format, and it seems a simple enough matter to add warnings that > matrices should only be pickled to binary format files. Other than > this, I've found the binary format to work quite well for archiving > arbitrary objects that contain matrices across many platforms > (big/little endian) and even for passing these objects across sockets. > > Anybody know a way I can add a check to the matrix pickling routines > to determine if they're writing to/reading from a binary format file > and raise an exception otherwise? > > -Jim I completely agree with the binary format for arrays. Is there any system where the "b" flag in open() causes problem? If not, I suggest to include this flag in TestPickle.py. BTW could you upgrade TestSuite to be compatible with v0.35? I realized that in this version nonzero() does not return None. This makes sense, but in this case I really miss a special object representing an empty array, like [] or () for an empty list or tuple. Matlab introduces [] for that, and it is very extensively used there. Geza I. Groma Institute of Biophysics Biological Research Centre of Hungarian Academy of Sciences Szeged, Hungary ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Mar 21 16:16:01 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA04129 for matrix-sig-people; Thu, 21 Mar 1996 16:16:01 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id QAA04125 for ; Thu, 21 Mar 1996 16:15:56 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id LAA19844 (8.6.11/IDA-1.6); Thu, 21 Mar 1996 11:24:50 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id LAA09980; Thu, 21 Mar 1996 11:24:11 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id LAA29075; Thu, 21 Mar 1996 11:24:09 -0500 Date: Thu, 21 Mar 1996 11:24:09 -0500 Message-Id: <199603211624.LAA29075@cyclone.ERE.UMontreal.CA> To: GROMA@everx.szbk.u-szeged.hu CC: jjh@Goldilocks.LCS.MIT.EDU, matrix-sig@python.org In-reply-to: <9D5DDCA3BB6@everx.szbk.u-szeged.hu> (GROMA@everx.szbk.u-szeged.hu) Subject: Re: [PYTHON MATRIX-SIG] pickle error on Win95 From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > None. This makes sense, but in this case I really miss a special object > representing an empty array, like [] or () for an empty list or tuple. > Matlab introduces [] for that, and it is very extensively used there. array() should accept empty lists in its arguments. Then array([]) would be a rank-1 array of length zero, array([[],[]]) a rank-2 array of shape (2,0) and so on. A single "empty array" does not make sense, since there can be many empty arrays that differ in shape. Matlab can get away with that because its matrix concept is much more limited. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Mar 25 11:52:57 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA01077 for matrix-sig-people; Mon, 25 Mar 1996 11:52:57 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id LAA01072 for ; Mon, 25 Mar 1996 11:52:52 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id IAA06662 (8.6.11/IDA-1.6); Mon, 25 Mar 1996 08:29:38 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id IAA25375; Mon, 25 Mar 1996 08:29:37 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id IAA16483; Mon, 25 Mar 1996 08:29:36 -0500 Date: Mon, 25 Mar 1996 08:29:36 -0500 Message-Id: <199603251329.IAA16483@cyclone.ERE.UMontreal.CA> To: GROMA@everx.szbk.u-szeged.hu CC: matrix-sig@python.org In-reply-to: <9EF360E2F63@everx.szbk.u-szeged.hu> (GROMA@everx.szbk.u-szeged.hu) Subject: [PYTHON MATRIX-SIG] Re: empty array From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > > > array() should accept empty lists in its arguments. Then array([]) > > would be a rank-1 array of length zero, array([[],[]]) a rank-2 array > > of shape (2,0) and so on. > > That is what I expected, too, but it does not work, at least on my > system (built from v0.35). What is wrong in this code? I know it doesn't work. When I wrote "it should work" I didn't mean "I think it works", but "I think it ought to work". Keep in mind that the array module is neither finished nor bug-free. > Even if array([]) were correct, that would be useful only to create > an empty array, and not to test it. In TestUtils.py I found a simple > and natural way to check the equality of array a and b: > > a.notEqual(b).nonzero() == None I must confess that I never understood what nonzero() is good for. Now I am beginning to realize its use. The name is rather misleading... Anyway, to test the equality of two arrays, of course a == b ought to work. Currently it seems to be equivalent to a is b which is against standard Python conventions. Unfortunately there is no way to give a reasonable meaning to a > b and a < b, but they are much less important in practice. Supposing that a == b does what everybody expects it to do, you can then use it to test for a specific form of an empty array by writing a == array([]) a == array([[]]) and so on. To test for general emptyness, i.e. to test if an array has zero elements, you can write ravel(a) == array([]) > Another problem is that an empty array is not a printable object: I know, but there is no point in trying to fix this as long as many operations don't work properly on empty arrays. The expression maximum.reduce(data) for example should not raise an error if "data" is empty, but return the smallest representable number of the type of "data". ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Mar 25 12:18:17 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA01387 for matrix-sig-people; Mon, 25 Mar 1996 12:18:17 -0500 Received: from sol.cc.u-szeged.hu (sol.cc.u-szeged.hu [160.114.8.24]) by python.org (8.6.12/8.6.12) with SMTP id MAA01353 for ; Mon, 25 Mar 1996 12:16:34 -0500 Received: from szbk.u-szeged.hu (rosi.szbk.u-szeged.hu) by sol.cc.u-szeged.hu (5.0/SMI-SVR4) id AA27226; Mon, 25 Mar 1996 11:42:21 +0100 Received: from everx.szbk.u-szeged.hu by szbk.u-szeged.hu (5.0/SMI-SVR4) id AA02998; Fri, 22 Mar 1996 16:14:36 --100 Received: from SZBK_EVERX/MERCURY by everx.szbk.u-szeged.hu (Mercury 1.21); 22 Mar 96 16:15:42 MET DST Received: from MERCURY by SZBK_EVERX (Mercury 1.21); 22 Mar 96 16:11:08 MET DST From: "Groma Geza" Organization: Biological Research Center To: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Fri, 22 Mar 1996 16:11:05 MET Subject: [PYTHON MATRIX-SIG] empty array Cc: matrix-sig@python.org Priority: normal X-Mailer: Pegasus Mail for Windows (v2.01) Message-Id: <9EF360E2F63@everx.szbk.u-szeged.hu> Content-Length: 0 Sender: owner-matrix-sig@python.org Precedence: bulk Konrad Hinsen writes: > array() should accept empty lists in its arguments. Then array([]) > would be a rank-1 array of length zero, array([[],[]]) a rank-2 array > of shape (2,0) and so on. That is what I expected, too, but it does not work, at least on my system (built from v0.35). What is wrong in this code? ------------------------------------------------------------------- >>> from Numeric import * >>> array([]) Traceback (innermost last): File "", line 1, in ? TypeError: illegal argument type for built-in operation >>> ------------------------------------------------------------------- > A single "empty array" does not make sense, > since there can be many empty arrays that differ in shape. Matlab can > get away with that because its matrix concept is much more limited. Even if array([]) were correct, that would be useful only to create an empty array, and not to test it. In TestUtils.py I found a simple and natural way to check the equality of array a and b: a.notEqual(b).nonzero() == None In v0.35 this returns false. The equality can be tested by len(a.notEqual(b).nonzero()) == 0 but I personally prefer the previous style. None is a 'general empty object', something similar could be introduced for arrays, too. Another problem is that an empty array is not a printable object: -------------------------------------------------------------- >>> array([0]).nonzero() Traceback (innermost last): File "", line 1, in ? File "./PrettyPrinter.py", line 28, in arrayToString max_str_len = max(len(str(maximum.reduce(data))), ArrayError: can't take anything from an empty array >>> ----------------------------------------------------------------- Geza I. Groma Institute of Biophysics Biological Research Centre of Hungarian Academy of Sciences Szeged, Hungary ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Mar 25 13:45:13 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA02100 for matrix-sig-people; Mon, 25 Mar 1996 13:45:13 -0500 Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.6.12/8.6.12) with SMTP id NAA02095 for ; Mon, 25 Mar 1996 13:45:09 -0500 Received: by icf.llnl.gov (4.1/LLNL-1.19) id AA05868; Mon, 25 Mar 96 10:45:05 PST Date: Mon, 25 Mar 96 10:45:05 PST From: tbyang@icf.llnl.gov (Tser-Yuan (Brian) Yang) Message-Id: <9603251845.AA05868@icf.llnl.gov> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] PrettyPrinter problem Sender: owner-matrix-sig@python.org Precedence: bulk The following may have been posted or even corrected. If so, I apologize for sending junk mail. The PrettyPrinter in the numeric package seems to have trouble printing a float or double array with only one element, i.e., >>> a=zeros(1,'d') >>> a Traceback (innermost last): File "", line 1, in ? File "/dist/basis/Python/Python-1.3b/lib/python/numeric/PrettyPrinter.py", lin e 34, in arrayToString format, item_length = _floatFormat(data, precision) File "/dist/basis/Python/Python-1.3b/lib/python/numeric/PrettyPrinter.py", lin e 91, in _floatFormat precision = min(precision, apply(max, tuple(map(lambda x, p=precision, TypeError: min() or max() of non-sequence Brian Yang ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Mar 25 14:35:58 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA02552 for matrix-sig-people; Mon, 25 Mar 1996 14:35:58 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id OAA02548 for ; Mon, 25 Mar 1996 14:35:55 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA23372; Mon, 25 Mar 96 14:35:52 EST Date: Mon, 25 Mar 96 14:35:52 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9603251935.AA23372@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org In-Reply-To: <199603251329.IAA16483@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca) Subject: Re: [PYTHON MATRIX-SIG] Re: empty array Sender: owner-matrix-sig@python.org Precedence: bulk From: hinsenk@ere.umontreal.ca (Konrad HINSEN) > > > array() should accept empty lists in its arguments. Then array([]) > > would be a rank-1 array of length zero, array([[],[]]) a rank-2 array > > of shape (2,0) and so on. > > That is what I expected, too, but it does not work, at least on my > system (built from v0.35). What is wrong in this code? I know it doesn't work. When I wrote "it should work" I didn't mean "I think it works", but "I think it ought to work". Keep in mind that the array module is neither finished nor bug-free. I made the obvious patches to get this to work and noticed that array([]) creates an empty array of characters (This is the lowest type in the type hierarchy). This is probably not what people want in the general case. Any suggestions on how to deal with the type of an empty array? -Jim BTW - I'm making these patches in preparation for one a final 0.36 release containing a fairly large number of bug fixes. After that I plan to start work on v0.40 which will involve switching over to much of Konrad's proposed naming conventions. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Mar 25 15:15:13 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA02915 for matrix-sig-people; Mon, 25 Mar 1996 15:15:13 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id PAA02910 for ; Mon, 25 Mar 1996 15:15:02 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id PAA22760 (8.6.11/IDA-1.6); Mon, 25 Mar 1996 15:13:34 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id PAA02357; Mon, 25 Mar 1996 15:13:34 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id PAA26408; Mon, 25 Mar 1996 15:13:33 -0500 Date: Mon, 25 Mar 1996 15:13:33 -0500 Message-Id: <199603252013.PAA26408@cyclone.ERE.UMontreal.CA> To: tbyang@icf.llnl.gov CC: matrix-sig@python.org In-reply-to: <9603251845.AA05868@icf.llnl.gov> (tbyang@icf.llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] PrettyPrinter problem From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > The PrettyPrinter in the numeric package seems to have trouble printing > a float or double array with only one element, i.e., > > >>> a=zeros(1,'d') > >>> a That has been fixed a long time ago, but somehow the fixed version did not propagate far from me :-( Anyway, the updated version I sent to the list some time ago should not have that bug. If anyone didn't get it or lost it, please contact me. (For identification: I am talking about the version that can print general arrays.) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Mar 25 15:28:22 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id PAA03006 for matrix-sig-people; Mon, 25 Mar 1996 15:28:22 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id PAA03002 for ; Mon, 25 Mar 1996 15:28:19 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id PAA23533 (8.6.11/IDA-1.6); Mon, 25 Mar 1996 15:25:31 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id PAA05121; Mon, 25 Mar 1996 15:25:31 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id PAA28420; Mon, 25 Mar 1996 15:25:29 -0500 Date: Mon, 25 Mar 1996 15:25:29 -0500 Message-Id: <199603252025.PAA28420@cyclone.ERE.UMontreal.CA> To: jjh@Goldilocks.LCS.MIT.EDU CC: matrix-sig@python.org In-reply-to: <9603251935.AA23372@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU) Subject: Re: [PYTHON MATRIX-SIG] Re: empty array From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > I made the obvious patches to get this to work and noticed that > array([]) creates an empty array of characters (This is the lowest > type in the type hierarchy). This is probably not what people want in > the general case. Any suggestions on how to deal with the type of an > empty array? Of course explicit type specifications should work as expected. For the default type, I am surprised that "character" is in the hierarchy at all. Character arrays behave differently and are not coerced to number arrays. They should therefore be a separate category. The default type for empty arrays would then be "integer", which is fine. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Mar 27 14:57:48 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA02562 for matrix-sig-people; Wed, 27 Mar 1996 14:57:48 -0500 Received: from virginia.edu (mars.itc.Virginia.EDU [128.143.2.9]) by python.org (8.6.12/8.6.12) with SMTP id OAA02558 for ; Wed, 27 Mar 1996 14:57:44 -0500 From: tnb2d@viper.cs.virginia.edu Received: from archive.cs.virginia.edu by mail.virginia.edu id ab01800; 27 Mar 96 12:58 EST Received: from viper.cs.Virginia.EDU (viper-fo.cs.Virginia.EDU [128.143.136.16]) by archive.cs.Virginia.EDU (8.7.1/8.6.6) with SMTP id MAA20729 for ; Wed, 27 Mar 1996 12:58:28 -0500 (EST) Received: by viper.cs.Virginia.EDU (5.x/SMI-2.0) id AA25053; Wed, 27 Mar 1996 12:58:27 -0500 Date: Wed, 27 Mar 1996 12:58:27 -0500 Message-Id: <9603271758.AA25053@viper.cs.Virginia.EDU> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] FAQ? Reply-To: tommy@archive.cs.virginia.edu Sender: owner-matrix-sig@python.org Precedence: bulk Hi, I just hitched up on this list looking for info on the Matrix extension(s) to Python and was wondering if there's a FAQ before I start asking questions. Thanks, -------> Tommy. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Mar 28 09:13:23 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA10890 for matrix-sig-people; Thu, 28 Mar 1996 09:13:23 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id JAA10885 for ; Thu, 28 Mar 1996 09:13:03 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id JAA00579 (8.6.11/IDA-1.6 for ); Thu, 28 Mar 1996 09:11:36 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id JAA05860; Thu, 28 Mar 1996 09:11:35 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id JAA26035; Thu, 28 Mar 1996 09:11:34 -0500 Date: Thu, 28 Mar 1996 09:11:34 -0500 Message-Id: <199603281411.JAA26035@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Slices From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk The recently introduced slice notation (a:b etc.) has greatly improved readability of slice indexing, but is has also lead to the disappearance of the old Slice() objects (they are still there, but are not accepted as indices any more) and thereby important functionality: it is no longer possible to construct index expressions containing slices dynamically. For those who wonder why anyone might want to do this, I'll describe my application: I have written a class that represents a function defined on a grid, with linear interpolation for arguments between the grid points. Of course it should work for functions of any number of arguments. A crucial step in the interpolation is the extraction of the subarray of neighbouring grid points, i.e. a subarray of length 2 along any axis. It is no problem to calculate the first/last indices for each axis, but there is currently no way to construct an indexing expression with slices, because the : notation is legal only inside square brackets. There are basically three possible solutions: 1) Make the : notation legal everywhere. That would make slice objects first-class objects like anything else in Python. You could create them, assign them to variables, make them elements of lists, tuples, or arrays, etc. It would then make sense to make them real sequence objects that can be subscripted, used in for-loops, converted to lists or tuples etc. 2) Reintroduce the old Slice() notation for the new slice objects. The consequences would be the same as above, but with a less uniform syntax. I don't really see the point of doing this. 3) Keep slice objects restricted to index expressions, but create some way to make them act on a dynamically specified axis. Then the example given above could be handled by iteratively indexing along the axes. Less elegant, but better than nothing. My personal favourite is 1), and I can't see any disadvantage in this. But that makes me think about why slice objects were restricted to subscripts in the first place. Was there a good reason for that, or was it just the desire to make the smallest possible change to Python syntax? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Mar 28 09:20:10 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA10890 for matrix-sig-people; Thu, 28 Mar 1996 09:13:23 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id JAA10885 for ; Thu, 28 Mar 1996 09:13:03 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id JAA00579 (8.6.11/IDA-1.6 for ); Thu, 28 Mar 1996 09:11:36 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id JAA05860; Thu, 28 Mar 1996 09:11:35 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id JAA26035; Thu, 28 Mar 1996 09:11:34 -0500 Date: Thu, 28 Mar 1996 09:11:34 -0500 Message-Id: <199603281411.JAA26035@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Slices From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk The recently introduced slice notation (a:b etc.) has greatly improved readability of slice indexing, but is has also lead to the disappearance of the old Slice() objects (they are still there, but are not accepted as indices any more) and thereby important functionality: it is no longer possible to construct index expressions containing slices dynamically. For those who wonder why anyone might want to do this, I'll describe my application: I have written a class that represents a function defined on a grid, with linear interpolation for arguments between the grid points. Of course it should work for functions of any number of arguments. A crucial step in the interpolation is the extraction of the subarray of neighbouring grid points, i.e. a subarray of length 2 along any axis. It is no problem to calculate the first/last indices for each axis, but there is currently no way to construct an indexing expression with slices, because the : notation is legal only inside square brackets. There are basically three possible solutions: 1) Make the : notation legal everywhere. That would make slice objects first-class objects like anything else in Python. You could create them, assign them to variables, make them elements of lists, tuples, or arrays, etc. It would then make sense to make them real sequence objects that can be subscripted, used in for-loops, converted to lists or tuples etc. 2) Reintroduce the old Slice() notation for the new slice objects. The consequences would be the same as above, but with a less uniform syntax. I don't really see the point of doing this. 3) Keep slice objects restricted to index expressions, but create some way to make them act on a dynamically specified axis. Then the example given above could be handled by iteratively indexing along the axes. Less elegant, but better than nothing. My personal favourite is 1), and I can't see any disadvantage in this. But that makes me think about why slice objects were restricted to subscripts in the first place. Was there a good reason for that, or was it just the desire to make the smallest possible change to Python syntax? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Mar 28 11:45:51 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA12799 for matrix-sig-people; Thu, 28 Mar 1996 11:45:51 -0500 Received: from ncar.UCAR.EDU (ncar.ucar.edu [192.52.106.6]) by python.org (8.6.12/8.6.12) with ESMTP id LAA12795 for ; Thu, 28 Mar 1996 11:45:47 -0500 Received: from pocc.gpsmet.ucar.edu by ncar.ucar.EDU (NCAR Local/ NCAR Central Post Office 03/11/93) id JAA08700; Thu, 28 Mar 1996 09:45:29 -0700 (MST) Received: by pocc.gpsmet.ucar.edu (5.x/SMI-SVR4) id AA06838; Thu, 28 Mar 1996 09:40:25 -0700 Date: Thu, 28 Mar 1996 09:40:25 -0700 (MST) From: Doug Hunt To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Where is the Numerical Python extension? Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk Hi all: I looked at the short postscript paper on Numerical Python on www.python.org and it looked like just the thing I need for numerical programming project I've got coming up (plus I wanted to learn python). The trouble is that after having looked all over www.python.org, I can't find the bloody thing. In the paper there is reference to a beta version having been released. Can anyone point me in the right direction? Many thanks, Doug Hunt dhunt@ucar.edu GPS/MET project Tel. (303) 497-2611 ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Mar 29 11:18:57 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id LAA22002 for matrix-sig-people; Fri, 29 Mar 1996 11:18:57 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id LAA21998 for ; Fri, 29 Mar 1996 11:18:52 -0500 Received: from baribal (localhost.MIT.EDU) by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA29889; Fri, 29 Mar 96 11:18:18 EST Message-Id: <315C0D49.41C67EA6@mit.edu> Date: Fri, 29 Mar 1996 11:18:17 -0500 From: Jim Hugunin Organization: MIT - LCS - SLS X-Mailer: Mozilla 2.0 (X11; I; SunOS 4.1.3_U1 sun4m) Mime-Version: 1.0 To: Carlos Maltzahn Cc: jjh@mit.edu, matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Re: building problems on DEC Alphas References: <4jfiab$f18@csnews.cs.colorado.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk Both of these issues are related entirely to the Numerical extension and should really be discussed in the matrix-sig. I hate to sound like a net cop, but the main reason for this is that I pay a great deal of attention to bugs reported to the sig, while I skip many of the postings to the main newsgroup. Your problem lies in ofuncobject.c, and if you add the following line to the top of this file, this problem should go away. #define HAVE_FINITE (Interestingly enough, I just ran into this problem myself this morning as I was trying to get some code to run on our alcors). As to your qeustion about cluster analysis, I'd be interested to hear what answers you get. -Jim Carlos Maltzahn wrote: Does anybody know about cluster analysis software that could be easily integrated into Numerical Python? Or even better, is there already such a module out there? > > I tried to build Python on a DEC Alpha (DEC OSF/1 V3.2A (Rev. 17)). > During linking I get the following error message: > > ld: > Unresolved: > CHECK > *** Exit 1 > Stop. > *** Exit 1 > Stop. > > After doing a grep CHECK Modules/*.c I get many modules that define > CHECK as a macro. Does anybody has an idea what's wrong? > > Any comments are very appreciated! > Carlos > > -- > # Carlos Maltzahn University of Colorado at Boulder > # carlosm@cs.colorado.edu Department of Computer Science > # FAX: ++1-303-492-2844 Campus Box 430, Boulder, CO 80309-0430 > # phone: 8709 http://www.cs.colorado.edu/~carlosm/ ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Mar 29 12:01:22 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA22380 for matrix-sig-people; Fri, 29 Mar 1996 12:01:22 -0500 Received: from ncar.UCAR.EDU (ncar.ucar.edu [192.52.106.6]) by python.org (8.6.12/8.6.12) with ESMTP id MAA22376 for ; Fri, 29 Mar 1996 12:01:17 -0500 Received: from pocc.gpsmet.ucar.edu by ncar.ucar.EDU (NCAR Local/ NCAR Central Post Office 03/11/93) id KAA00968; Fri, 29 Mar 1996 10:00:42 -0700 (MST) Received: by pocc.gpsmet.ucar.edu (5.x/SMI-SVR4) id AA16030; Fri, 29 Mar 1996 09:55:38 -0700 Date: Fri, 29 Mar 1996 09:55:38 -0700 (MST) From: Doug Hunt To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Thanks Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk Hi all: Thanks for the quick replies regarding the location of Numeric Python. I downloaded it and was able to get it up and running (flawless build!) on my Linux machine at home. A few quick experiments lead me to believe that Numeric Python will do everything that I need. Keep up the good work! Doug Hunt dhunt@ucar.edu GPS/MET project Tel. (303) 497-2611 ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Mar 29 13:49:27 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA23206 for matrix-sig-people; Fri, 29 Mar 1996 13:49:27 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id NAA23202 for ; Fri, 29 Mar 1996 13:49:23 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA00977; Fri, 29 Mar 96 13:48:36 EST Date: Fri, 29 Mar 96 13:48:36 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9603291848.AA00977@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Floating point exceptions on DEC alpha Sender: owner-matrix-sig@python.org Precedence: bulk I've recently started running some fairly demanding code on our DEC alphas (using Numeric python of course) and I've run into a very annoying piece of behavior. If I do a division by zero, instead of returning an IEEE Infinity value, an exception is raised and I'm thrown out of python entirely. You'll have to take my word for the fact that these particular divide by zeros are not in fact a bug in my code, but something that is worth catching later. Here's a simple example: On a Sparc 10 (and most other machines I've worked with) >>> from Numeric import * >>> array(1.)/0 Infinity >>> On a DEC alpha >>> from Numeric import * >>> array(1.)/0 Floating exception (Dumped back to the shell) What I'd like to do is find a compiler flag, environment variable, or whatever that would make the alpha behave like the Sparc 10 in this and similar cases. Does anybody have any hints? -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Mar 29 16:59:47 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id QAA24516 for matrix-sig-people; Fri, 29 Mar 1996 16:59:47 -0500 Received: from anchor.cs.colorado.edu (carlosm@anchor.cs.colorado.edu [128.138.242.1]) by python.org (8.6.12/8.6.12) with ESMTP id QAA24512 for ; Fri, 29 Mar 1996 16:59:43 -0500 Received: (from carlosm@localhost) by anchor.cs.colorado.edu (8.7.5/8.7.3) id OAA12556 for matrix-sig@python.org; Fri, 29 Mar 1996 14:59:03 -0700 (MST) From: Carlos Maltzahn Message-Id: <199603292159.OAA12556@anchor.cs.colorado.edu> Subject: [PYTHON MATRIX-SIG] Are the matrix-sig articles archived? To: matrix-sig@python.org Date: Fri, 29 Mar 1996 14:59:03 -0700 (MST) X-Mailer: ELM [version 2.4 PL23] Content-Type: text Sender: owner-matrix-sig@python.org Precedence: bulk If so, could somebody tell me where the archive is? Thanks, Carlos ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Mar 30 21:31:10 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id VAA31818 for matrix-sig-people; Sat, 30 Mar 1996 21:31:10 -0500 Received: from anchor.cs.colorado.edu (carlosm@anchor.cs.colorado.edu [128.138.242.1]) by python.org (8.6.12/8.6.12) with ESMTP id VAA31814 for ; Sat, 30 Mar 1996 21:31:07 -0500 Received: (from carlosm@localhost) by anchor.cs.colorado.edu (8.7.5/8.7.3) id TAA25619 for matrix-sig@python.org; Sat, 30 Mar 1996 19:30:12 -0700 (MST) From: Carlos Maltzahn Message-Id: <199603310230.TAA25619@anchor.cs.colorado.edu> Subject: [PYTHON MATRIX-SIG] max_element array method To: matrix-sig@python.org Date: Sat, 30 Mar 1996 19:30:11 -0700 (MST) X-Mailer: ELM [version 2.4 PL23] Content-Type: text Sender: owner-matrix-sig@python.org Precedence: bulk I don't know whether this has been discussed before because I have yet to find the matrix-sig archive. I'm currently implementing some clustering algorithms and came across two features that would be very helpful to me: 1. A space-efficient implementation of a symmetric matrix object, and 2. a max_elem() and min_elem() method that would return the tuple (max_elem, index). For example: > from Numeric import * > m == zeros(3, 3, 2) > m[2, 0, 1] = 12.04 > m.max_elem() > (12.04, (2, 0, 1)) > m.min_elem() > (0.0, (2, 2, 1)) > m[2, 0].max_elem() > (12.04, (1,)) Currently, I'm using a Python class that does just that. But a C-level implementation would be probably faster and more space-efficient. Comments? Carlos ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Mar 31 09:52:59 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id JAA01669 for matrix-sig-people; Sun, 31 Mar 1996 09:52:59 -0500 Received: from harfang.CC.UMontreal.CA ([132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id JAA01665 for ; Sun, 31 Mar 1996 09:52:55 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id JAA09169 (8.6.11/IDA-1.6); Sun, 31 Mar 1996 09:50:43 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id JAA18573; Sun, 31 Mar 1996 09:50:42 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id JAA27508; Sun, 31 Mar 1996 09:50:40 -0500 Date: Sun, 31 Mar 1996 09:50:40 -0500 Message-Id: <199603311450.JAA27508@cyclone.ERE.UMontreal.CA> To: carlosm@anchor.cs.colorado.edu CC: matrix-sig@python.org In-reply-to: <199603310230.TAA25619@anchor.cs.colorado.edu> (message from Carlos Maltzahn on Sat, 30 Mar 1996 19:30:11 -0700 (MST)) Subject: Re: [PYTHON MATRIX-SIG] max_element array method From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > I don't know whether this has been discussed before because I have yet > to find the matrix-sig archive. I'm currently implementing some clustering > algorithms and came across two features that would be very helpful to > me: > > 1. A space-efficient implementation of a symmetric matrix object, and That is not so difficult. The most straightforward way is to define symmetric matrices as a Python class and use a one-dimensional array object as the data representation. The most reasonable arrangement is the packed format used by LAPACK, and of course by using it you have immediate access to all the linear algebra functions from that library. All the elementwise operations can be mapped directly on the internal representation. Indexing is straightforward. Matrix multiplication has to be rewritten, and is a bit tricky, since the product of two symmetric matrices is in general not a symmetric matrix. But if you want full compatibility with ordinary arrays, the implementation gets complicated. Slices, for example, are messy. So are reduction operations, for example. But these operations don't make much sense for symmetric matrices anyway. So the best solution is not to pretend that symmetric matrices are a special case of general arrays and implement only a meaningful subset of operations. > 2. a max_elem() and min_elem() method that would return the tuple > (max_elem, index). For example: Right, there is very little support now for efficient determination of specific indices. I had a similar problem recently with my class for function interpolation: finding the index of the largest element smaller than a given number in a sorted array. > Currently, I'm using a Python class that does just that. But a C-level > implementation would be probably faster and more space-efficient. I don't know how you do it in Python, but if you are doing the whole search in Python, consider the following alternative: def max_index(a): max = maximum.reduce(a) index = compress(equal(a, max), arange(a.shape[0])) return max, index This version is only for rank-1 arrays; a general version is possible, but messy. It is still inefficient compared to C, but there is no loop in Python, which probably makes it faster for long arrays. And if the maximum occurs more than once, you even get all the indices. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Mar 31 12:29:17 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA02437 for matrix-sig-people; Sun, 31 Mar 1996 12:29:17 -0500 Received: from clark.lcs.mit.edu (clark.lcs.mit.edu [18.30.0.101]) by python.org (8.6.12/8.6.12) with SMTP id MAA02429 for ; Sun, 31 Mar 1996 12:29:13 -0500 Received: from john-muir by clark.lcs.mit.edu (NX5.67d/NX3.0M-PIA-MT1.0M) id AA13359; Sun, 31 Mar 96 12:28:19 -0500 Message-Id: <9603311728.AA13359@clark.lcs.mit.edu> Received: by john-muir.lcs.mit.edu (NX5.67e/NX3.0X-PIA-MT1.0X) id AA11073; Sun, 31 Mar 96 12:28:18 -0500 Content-Type: text/plain Mime-Version: 1.0 (NeXT Mail 3.3 v118.2) Received: by NeXT.Mailer (1.118.2) From: James Hugunin Date: Sun, 31 Mar 96 12:28:16 -0500 To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] max_element array method Sender: owner-matrix-sig@python.org Precedence: bulk Try argmax and argmin. (And since I had similar problems as Konrad, the next release will add argsort). ie. >>> argmax([1,2,3,4,3,2,1]) (4, 3) This will only return the index of the first maxima in the list (which is usually what I want anyway). This will work on arbitrarily high dimensional arrays, always acting along the last axis. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Mar 31 14:34:20 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA03023 for matrix-sig-people; Sun, 31 Mar 1996 14:34:20 -0500 Received: from catch.ivab.se (root@catch.ivab.se [193.180.173.22]) by python.org (8.6.12/8.6.12) with ESMTP id OAA03019 for ; Sun, 31 Mar 1996 14:34:15 -0500 Received: from arnold.image.ivab.se (arnold.image.ivab.se [193.180.173.119]) by catch.ivab.se (8.6.12/8.6.12) with SMTP id VAA32444 for ; Sun, 31 Mar 1996 21:33:24 +0200 Received: by arnold.image.ivab.se; (5.65v3.2/1.1.8.2/16Nov95-0104AM) id AA06008; Sun, 31 Mar 1996 21:33:19 +0200 Date: Sun, 31 Mar 1996 21:33:19 +0200 Message-Id: <9603311933.AA06008@arnold.image.ivab.se> From: Fredrik Lundh To: matrix-sig@python.org In-Reply-To: <9603311728.AA13359@clark.lcs.mit.edu> (message from James Hugunin on Sun, 31 Mar 96 12:28:16 -0500) Subject: [PYTHON MATRIX-SIG] Sorry to have to ask this question again... Reply-To: fredrik_lundh@ivab.se Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk but how do I get my hands at the most recent distribution? cannot seem to get through to sls-ftp.lcs.mit.edu... /F -------------------------------------------------------------------- PS! The first version of the Python Imaging Library is now released! Join the image-sig for more info! ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Mar 31 14:47:00 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA03089 for matrix-sig-people; Sun, 31 Mar 1996 14:47:00 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id OAA03085 for ; Sun, 31 Mar 1996 14:46:57 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA05474; Sun, 31 Mar 96 14:45:59 EST Date: Sun, 31 Mar 96 14:45:59 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9603311945.AA05474@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: fredrik_lundh@ivab.se Subject: Re: [PYTHON MATRIX-SIG] Sorry to have to ask this question again... Cc: matrix-sig@python.org Sender: owner-matrix-sig@python.org Precedence: bulk Our group file server is down for some serious maintenence this weekend. It should be back up by Monday morning. (Good news about the imaging lib, I'll have to give it a try.) -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Apr 3 04:07:39 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id EAA03998 for matrix-sig-people; Wed, 3 Apr 1996 04:07:39 -0500 Received: from catch.ivab.se (root@catch.ivab.se [193.180.173.22]) by python.org (8.6.12/8.6.12) with ESMTP id EAA03991 for ; Wed, 3 Apr 1996 04:07:34 -0500 Received: from arnold.image.ivab.se (arnold.image.ivab.se [193.180.173.119]) by catch.ivab.se (8.6.12/8.6.12) with SMTP id LAA05278 for ; Wed, 3 Apr 1996 11:07:29 +0200 Received: by arnold.image.ivab.se; (5.65v3.2/1.1.8.2/16Nov95-0104AM) id AA05551; Wed, 3 Apr 1996 11:07:26 +0200 Date: Wed, 3 Apr 1996 11:07:26 +0200 Message-Id: <9604030907.AA05551@arnold.image.ivab.se> From: Fredrik Lundh To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Errors when importing the Numeric module Reply-To: fredrik_lundh@ivab.se Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk Just built Python 1.3 with NumericPython 0.31. Can anyone explain the following... (ok, the first two lines are obviously test stuff left in UserArray.py, but the exception)? Python 1.3 (Apr 2 1996) [C] Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> import Numeric UserArray([1,2,3], 'l') UserArray([0.841470956802,0.909297406673,0.141120001674], 'f') Traceback (innermost last): File "", line 1, in ? File "/usr/local/lib/python/numeric/Numeric.py", line 9, in ? import string, types, math File "/usr/local/lib/python/types.py", line 12, in ? if vars(__builtins__).has_key('complex'): TypeError: vars() argument must have __dict__ attribute When importing it again, everything works fine. BTW, it seems as if a Matrix<->PIL interface consists of ~2 lines of code for each direction. Watch this space for info. /F ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Apr 3 05:23:57 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id FAA04443 for matrix-sig-people; Wed, 3 Apr 1996 05:23:57 -0500 Received: from stork.EMBL-Heidelberg.DE (stork.EMBL-Heidelberg.DE [192.54.41.54]) by python.org (8.6.12/8.6.12) with ESMTP id FAA04436 for ; Wed, 3 Apr 1996 05:23:53 -0500 Received: from nu.EMBL-Heidelberg.DE (nu [192.54.41.120]) by stork.EMBL-Heidelberg.DE (8.7.1/8.7.1) with SMTP id MAA23837 for ; Wed, 3 Apr 1996 12:23:40 +0200 (MDT) Received: by nu.EMBL-Heidelberg.DE (8.6.11) id MAA02513; Wed, 3 Apr 1996 12:23:39 +0200 Date: Wed, 3 Apr 1996 12:23:39 +0200 Message-Id: <199604031023.MAA02513@nu.EMBL-Heidelberg.DE> From: Rob Hooft To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Comparisons Sender: owner-matrix-sig@python.org Precedence: bulk Hi, I just subscribed to this list this week, and have spend a few evenings reading the 1.8MB archive.... I haven't even compiled the software yet, and still I'd like to mention something that surprised me in the discussions so far. Repeatedly there has been a discussion about "<" and ">" comparisons for Array objects. Why can that not be implemented in the "dictionary" way? It just needs to be made recursive. Like: a; Wed, 3 Apr 1996 06:00:58 -0500 Received: from arnold.image.ivab.se (arnold.image.ivab.se [193.180.173.119]) by catch.ivab.se (8.6.12/8.6.12) with SMTP id NAA05573 for ; Wed, 3 Apr 1996 13:00:43 +0200 Received: by arnold.image.ivab.se; (5.65v3.2/1.1.8.2/16Nov95-0104AM) id AA22601; Wed, 3 Apr 1996 13:00:40 +0200 Date: Wed, 3 Apr 1996 13:00:40 +0200 Message-Id: <9604031100.AA22601@arnold.image.ivab.se> From: Fredrik Lundh To: matrix-sig@python.org In-Reply-To: <199604031023.MAA02513@nu.EMBL-Heidelberg.DE> (message from Rob Hooft on Wed, 3 Apr 1996 12:23:39 +0200) Subject: [PYTHON MATRIX-SIG] Regarding my problems with 0.31... Reply-To: fredrik_lundh@ivab.se Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk Rob wrote: > I'll be trying to compile 0.35 soon. And I'll better do the same. Please ignore my previous bug report! (unless the problem is still there with the 0.35 release...). /F ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Apr 3 06:25:36 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id GAA04769 for matrix-sig-people; Wed, 3 Apr 1996 06:25:36 -0500 Received: from catch.ivab.se (root@catch.ivab.se [193.180.173.22]) by python.org (8.6.12/8.6.12) with ESMTP id GAA04762; Wed, 3 Apr 1996 06:25:32 -0500 Received: from arnold.image.ivab.se (arnold.image.ivab.se [193.180.173.119]) by catch.ivab.se (8.6.12/8.6.12) with SMTP id NAA05666; Wed, 3 Apr 1996 13:25:22 +0200 Received: by arnold.image.ivab.se; (5.65v3.2/1.1.8.2/16Nov95-0104AM) id AA20167; Wed, 3 Apr 1996 13:25:19 +0200 Date: Wed, 3 Apr 1996 13:25:19 +0200 Message-Id: <9604031125.AA20167@arnold.image.ivab.se> From: Fredrik Lundh To: matrix-sig@python.org, image-sig@python.org Subject: [PYTHON MATRIX-SIG] ANNOUNCE: A minimal Matrix<->PIL interface Reply-To: fredrik_lundh@ivab.se Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk If someone would like to fool around with the Python Imaging Library and the Numerical extension at the same time, here's a really minimal interface between these two libraries. A little more checking should of course be added, to make sure that the image isn't multiband, and that the array really contains unsigned 8-bit integers, but at least you'll get the idea :-) /F -------------------------------------------------------------------- import Image, Numeric def ImageToArray(i): a = Numeric.array(i._tostring(), "b") a.shape = i.size[1], i.size[0] return a def ArrayToImage(a): i = Image.new("L", (a.shape[1], a.shape[0])) i._fromstring(a.toString()) return i # "If you cannot do it in 8 lines of Python, it is probably # not worth doing." -------------------------------------------------------------------- The current PIL distribution can be found at: http://www.python.org/sigs/image-sig/Imaging-0.0.tar.gz http://www.python.org/sigs/image-sig/Imaging-0.1-handbook-draft.ps.gz Release 0.1 of the library will be available real soon now (it would be nice if at least someone had any comments on the current release, though :-) ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Apr 3 06:42:40 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id GAA04866 for matrix-sig-people; Wed, 3 Apr 1996 06:42:40 -0500 Received: from catch.ivab.se (root@catch.ivab.se [193.180.173.22]) by python.org (8.6.12/8.6.12) with ESMTP id GAA04859 for ; Wed, 3 Apr 1996 06:42:35 -0500 Received: from arnold.image.ivab.se (arnold.image.ivab.se [193.180.173.119]) by catch.ivab.se (8.6.12/8.6.12) with SMTP id NAA05740 for ; Wed, 3 Apr 1996 13:42:30 +0200 Received: by arnold.image.ivab.se; (5.65v3.2/1.1.8.2/16Nov95-0104AM) id AA10297; Wed, 3 Apr 1996 13:42:26 +0200 Date: Wed, 3 Apr 1996 13:42:26 +0200 Message-Id: <9604031142.AA10297@arnold.image.ivab.se> From: Fredrik Lundh To: matrix-sig@python.org In-Reply-To: <9604031125.AA20167@arnold.image.ivab.se> (message from Fredrik Lundh on Wed, 3 Apr 1996 13:25:19 +0200) Subject: [PYTHON MATRIX-SIG] Re: [PYTHON IMAGE-SIG] ANNOUNCE: A minimal Matrix<->PIL interface Reply-To: fredrik_lundh@ivab.se Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk BTW, have anyone written a 2D FFT module for the Matrix library? /F ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Apr 3 07:11:04 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id HAA05037 for matrix-sig-people; Wed, 3 Apr 1996 07:11:04 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id HAA05030 for ; Wed, 3 Apr 1996 07:11:00 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id HAA05516 (8.6.11/IDA-1.6); Wed, 3 Apr 1996 07:08:18 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id HAA01587; Wed, 3 Apr 1996 07:08:18 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id HAA28780; Wed, 3 Apr 1996 07:08:17 -0500 Date: Wed, 3 Apr 1996 07:08:17 -0500 Message-Id: <199604031208.HAA28780@cyclone.ERE.UMontreal.CA> To: Rob.Hooft@EMBL-Heidelberg.de CC: matrix-sig@python.org In-reply-to: <199604031023.MAA02513@nu.EMBL-Heidelberg.DE> (message from Rob Hooft on Wed, 3 Apr 1996 12:23:39 +0200) Subject: Re: [PYTHON MATRIX-SIG] Comparisons From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > > Repeatedly there has been a discussion about "<" and ">" comparisons > for Array objects. Why can that not be implemented in the "dictionary" > way? It just needs to be made recursive. Like: > > a (a[0] (a[0]==b[0] and (a[1] .... It probably can be implemented in such a way, but why? For what application would this definition be useful? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Apr 3 08:01:48 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id IAA05361 for matrix-sig-people; Wed, 3 Apr 1996 08:01:48 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id IAA05354 for ; Wed, 3 Apr 1996 08:01:45 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id IAA06180 (8.6.11/IDA-1.6 for ); Wed, 3 Apr 1996 08:00:19 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id IAA06992; Wed, 3 Apr 1996 08:00:18 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id IAA00757; Wed, 3 Apr 1996 08:00:17 -0500 Date: Wed, 3 Apr 1996 08:00:17 -0500 Message-Id: <199604031300.IAA00757@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org In-reply-to: <9604031142.AA10297@arnold.image.ivab.se> (message from Fredrik Lundh on Wed, 3 Apr 1996 13:42:26 +0200) Subject: Re: [PYTHON MATRIX-SIG] Re: [PYTHON IMAGE-SIG] ANNOUNCE: A minimal Matrix<->PIL interface From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > > BTW, have anyone written a 2D FFT module for the Matrix library? Or 1D, or 3D? Or, better yet, anyD? ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Apr 3 08:05:13 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id IAA05385 for matrix-sig-people; Wed, 3 Apr 1996 08:05:13 -0500 Received: from stork.EMBL-Heidelberg.DE (stork.EMBL-Heidelberg.DE [192.54.41.54]) by python.org (8.6.12/8.6.12) with ESMTP id IAA05377 for ; Wed, 3 Apr 1996 08:05:07 -0500 Received: from nu.EMBL-Heidelberg.DE (nu [192.54.41.120]) by stork.EMBL-Heidelberg.DE (8.7.1/8.7.1) with SMTP id PAA26271; Wed, 3 Apr 1996 15:04:54 +0200 (MDT) Received: by nu.EMBL-Heidelberg.DE (8.6.11) id PAA24794; Wed, 3 Apr 1996 15:04:53 +0200 Date: Wed, 3 Apr 1996 15:04:53 +0200 Message-Id: <199604031304.PAA24794@nu.EMBL-Heidelberg.DE> From: Rob Hooft To: hinsenk@ere.umontreal.ca (Konrad HINSEN) Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Comparisons In-Reply-To: <199604031208.HAA28780@cyclone.ERE.UMontreal.CA> References: <199604031023.MAA02513@nu.EMBL-Heidelberg.DE> <199604031208.HAA28780@cyclone.ERE.UMontreal.CA> Sender: owner-matrix-sig@python.org Precedence: bulk >>>>> "KH" == Konrad HINSEN writes: >> Repeatedly there has been a discussion about "<" and ">" >> comparisons for Array objects. Why can that not be implemented in >> the "dictionary" way? It just needs to be made recursive. Like: >> >> a> .... KH> It probably can be implemented in such a way, but why? For what KH> application would this definition be useful? It would allow sorting an Array of transformation matrices, which then allows some sort of binary search to see whether a certain transformation is in there. Of course, a hash value could be used as well in this case (is that implemented? Arrays are not hasheable, as they are mutable, isn't it?), but there might be other uses for comparisons. As I said I haven't even compiled the software yet, so risking that I'm quoting outdated information: I think Guido has said that there is no provision for comparisons to generate exceptions yet. The implementation I proposed might be a reasonable one to convert (a; Wed, 3 Apr 1996 09:34:15 -0500 Message-Id: <199604031434.JAA05935@python.org> Received: by bartel.jhuapl.edu (1.38.193.4/16.2) id AA01353; Wed, 3 Apr 1996 09:41:26 -0500 Date: Wed, 3 Apr 1996 09:41:26 -0500 From: Chris Chase S1A To: hinsenk@ere.umontreal.ca (Konrad HINSEN) Cc: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Slices In-Reply-To: <199603281411.JAA26035@cyclone.ERE.UMontreal.CA> References: <199603281411.JAA26035@cyclone.ERE.UMontreal.CA> Reply-To: chris.chase@jhuapl.edu Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk >>>>> "KH" == Konrad HINSEN writes: KH> The recently introduced slice notation (a:b etc.) has greatly improved KH> readability of slice indexing, but is has also lead to the disappearance KH> of the old Slice() objects (they are still there, but are not accepted KH> as indices any more) and thereby important functionality: it is no KH> longer possible to construct index expressions containing slices KH> dynamically. KH> There are basically three possible solutions: KH> 1) Make the : notation legal everywhere. That would make slice objects KH> first-class objects like anything else in Python. You could create KH> them, assign them to variables, make them elements of lists, tuples, KH> or arrays, etc. It would then make sense to make them real sequence KH> objects that can be subscripted, used in for-loops, converted to KH> lists or tuples etc. KH> 2) Reintroduce the old Slice() notation for the new slice objects. KH> The consequences would be the same as above, but with a less KH> uniform syntax. I don't really see the point of doing this. KH> 3) Keep slice objects restricted to index expressions, but create KH> some way to make them act on a dynamically specified axis. Then KH> the example given above could be handled by iteratively indexing KH> along the axes. Less elegant, but better than nothing. KH> My personal favourite is 1), and I can't see any disadvantage in KH> this. But that makes me think about why slice objects were restricted KH> to subscripts in the first place. Was there a good reason for that, or KH> was it just the desire to make the smallest possible change to Python KH> syntax? There are problems with this as I found when initially considering such a possibility. If ":" notation is allowed wherever a "test" expression can occur then there can be ambiguity with statements where ":" is used to introduce a compound statement, i.e. in "for" or "if" statements. I think that we will have to go with option (2). However, the interface to Slice() should change to duplicate ":" rather than have the old interface. Chris ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Apr 3 10:38:11 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id KAA06396 for matrix-sig-people; Wed, 3 Apr 1996 10:38:11 -0500 Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.6.12/8.6.12) with SMTP id KAA06389 for ; Wed, 3 Apr 1996 10:38:07 -0500 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA14920; Wed, 3 Apr 96 10:33:28 -0500 Received: by maigret (950215.SGI.8.6.10/940406.SGI) id KAA04418; Wed, 3 Apr 1996 10:33:29 -0500 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199604031533.KAA04418@maigret> Subject: Re: [PYTHON MATRIX-SIG] Re: [PYTHON IMAGE-SIG] ANNOUNCE: A minimal Matrix<->PIL interface To: hinsenk@ere.umontreal.ca (Konrad HINSEN) Date: Wed, 3 Apr 1996 10:33:28 -0500 (EST) Cc: matrix-sig@python.org In-Reply-To: <199604031300.IAA00757@cyclone.ERE.UMontreal.CA> from "Konrad HINSEN" at Apr 3, 96 08:00:17 am X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 1050 Sender: owner-matrix-sig@python.org Precedence: bulk > > > > BTW, have anyone written a 2D FFT module for the Matrix library? > > Or 1D, or 3D? Or, better yet, anyD? I have a module which is an interface to the fft routines used by TELA (they are basically a C translation of FFTPACK). I haven't released it because I hadn't finished porting the AnyD to 1D TELA code into Python. Since it looks like I won't have the time to do much related to Python in the next three weeks, I'm willing to let it out, with the caveat that it hasn't been tested. Does anyone know who to contact about getting permission to release a file from the tela library? I'll assume it's ok for now, but I'd like to get an official ok sometime. http://maigret.cog.brown.edu/python/fft has the files you need: fftpackmodule.c, and the fftpack files from the TELA distribution. If someone wants to finish the fft.ct -> fft.py conversion, please do, and let me know about it. I've included my first stab at the translation on the web page. --david PS: Let me know about bugs and bug fixes, of course. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Apr 3 12:32:16 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id MAA07335 for matrix-sig-people; Wed, 3 Apr 1996 12:32:16 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id MAA07328 for ; Wed, 3 Apr 1996 12:32:09 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id MAA17256 (8.6.11/IDA-1.6); Wed, 3 Apr 1996 12:30:27 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id MAA25176; Wed, 3 Apr 1996 12:30:27 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id MAA23822; Wed, 3 Apr 1996 12:30:25 -0500 Date: Wed, 3 Apr 1996 12:30:25 -0500 Message-Id: <199604031730.MAA23822@cyclone.ERE.UMontreal.CA> To: chris.chase@jhuapl.edu CC: matrix-sig@python.org In-reply-to: <199604031432.JAA07404@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Wed, 3 Apr 1996 09:41:26 -0500) Subject: Re: [PYTHON MATRIX-SIG] Slices From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > There are problems with this as I found when initially considering > such a possibility. If ":" notation is allowed wherever a "test" > expression can occur then there can be ambiguity with statements where > ":" is used to introduce a compound statement, i.e. in "for" or "if" > statements. I see. That's of course a good argument. > I think that we will have to go with option (2). However, the > interface to Slice() should change to duplicate ":" rather than have > the old interface. But that is rather difficult with standard function-call syntax. Another idea: introduce an object "index_expression" that can be indexed to get an equivalent expression. Then i = index_expression[2:4,...] a[i] would be equivalent to a[2:4,...] There would be no new syntax to learn; although users might have to know that index expressions are really tuples if they want to do something that is not possible with standard indexing. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Apr 3 13:28:53 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA07744 for matrix-sig-people; Wed, 3 Apr 1996 13:28:53 -0500 Received: from CNRI.Reston.VA.US (cnri.reston.va.us [132.151.1.1]) by python.org (8.6.12/8.6.12) with SMTP id NAA07737 for ; Wed, 3 Apr 1996 13:28:50 -0500 Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa12145; 3 Apr 96 13:27 EST Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id NAA09584; Wed, 3 Apr 1996 13:27:01 -0500 Message-Id: <199604031827.NAA09584@monty> To: Konrad HINSEN cc: chris.chase@jhuapl.edu, matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Slices In-reply-to: Your message of "Wed, 03 Apr 1996 12:30:25 EST." <199604031730.MAA23822@cyclone.ERE.UMontreal.CA> References: <199604031730.MAA23822@cyclone.ERE.UMontreal.CA> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Wed, 03 Apr 1996 13:27:01 -0500 Sender: owner-matrix-sig@python.org Precedence: bulk > Another idea: introduce an object "index_expression" that can be > indexed to get an equivalent expression. Then > i = index_expression[2:4,...] > a[i] > would be equivalent to > a[2:4,...] Very clever. The index_expression object would simply package all its arguments and return them as a Slice() object. Way to go! --Guido van Rossum URL: ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Apr 3 13:44:51 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA07920 for matrix-sig-people; Wed, 3 Apr 1996 13:44:51 -0500 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU (baribal.lcs.mit.edu [18.27.0.172]) by python.org (8.6.12/8.6.12) with SMTP id NAA07913 for ; Wed, 3 Apr 1996 13:44:47 -0500 Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA15163; Wed, 3 Apr 96 13:44:35 EST Date: Wed, 3 Apr 96 13:44:35 EST From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9604031844.AA15163@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Slices Sender: owner-matrix-sig@python.org Precedence: bulk From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Another idea: introduce an object "index_expression" that can be indexed to get an equivalent expression. Then i = index_expression[2:4,...] a[i] would be equivalent to a[2:4,...] There would be no new syntax to learn; although users might have to know that index expressions are really tuples if they want to do something that is not possible with standard indexing. Anybody who's interested in this idea can do the following under v0.35. import Numeric index_expression = Numeric.__slice_generating_class() -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Apr 3 13:58:45 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id NAA08077 for matrix-sig-people; Wed, 3 Apr 1996 13:58:45 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id NAA08070 for ; Wed, 3 Apr 1996 13:58:42 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id NAA20196 (8.6.11/IDA-1.6); Wed, 3 Apr 1996 13:57:10 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id NAA07263; Wed, 3 Apr 1996 13:57:09 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id NAA00933; Wed, 3 Apr 1996 13:57:08 -0500 Date: Wed, 3 Apr 1996 13:57:08 -0500 Message-Id: <199604031857.NAA00933@cyclone.ERE.UMontreal.CA> To: guido@CNRI.Reston.VA.US CC: chris.chase@jhuapl.edu, matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Slices From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk While putting my recent suggestion into an existing application, I noticed that a somewhat better definition is: class _index_expression_class: def __getitem__(self, item): if type(item) != type(()): return (item,) else: return item index_expression = _index_expression_class() This version always returns a tuple, even if the index is a single number. That makes it easier to write code that strings together index expressions. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Apr 3 14:44:59 1996 Received: (from majordom@localhost) by python.org (8.6.12/8.6.12) id OAA08536 for matrix-sig-people; Wed, 3 Apr 1996 14:44:59 -0500 Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.6.12/8.6.12) with ESMTP id OAA08528 for ; Wed, 3 Apr 1996 14:44:52 -0500 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id NAA19872 (8.6.11/IDA-1.6); Wed, 3 Apr 1996 13:49:53 -0500 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id NAA06056; Wed, 3 Apr 1996 13:49:53 -0500 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id NAA00321; Wed, 3 Apr 1996 13:49:51 -0500 Date: Wed, 3 Apr 1996 13:49:51 -0500 Message-Id: <199604031849.NAA00321@cyclone.ERE.UMontreal.CA> To: guido@CNRI.Reston.VA.US CC: chris.chase@jhuapl.edu, matrix-sig@python.org In-reply-to: <199604031827.NAA09584@monty> (message from Guido van Rossum on Wed, 03 Apr 1996 13:27:01 -0500) Subject: Re: [PYTHON MATRIX-SIG] Slices From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > > Another idea: introduce an object "index_expression" that can be > > indexed to get an equivalent expression. Then > > i = index_expression[2:4,...] > > a[i] > > would be equivalent to > > a[2:4,...] > > Very clever. The index_expression object would simply package all its > arguments and return them as a Slice() object. Way to go! I actually realized that the implementation is completely trivial: class _index_expression_class: def __getitem__(self, item): return item index_expression = _index_expression_class() Now let's try it: from Numeric import * a = array([[1,2,3],[6,5,4]]) i = index_expression[..., 1:3] print i print a[i] Output: (..., Slice(1, 3, 1)) 2 3 5 4 It works! Let's just put that into Numeric and be happy... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Apr 9 22:14:52 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id WAA07756; Tue, 9 Apr 1996 22:07:28 -0400 Received: from csc-sun.math.utah.edu by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id WAA07746; Tue, 9 Apr 1996 22:07:19 -0400 Received: from blab01.math.utah.edu (hohn@blab01.math.utah.edu [155.99.144.28]) by csc-sun.math.utah.edu (8.7.4/8.7.3) with ESMTP id UAA26597 for ; Tue, 9 Apr 1996 20:05:32 -0600 (MDT) Received: (from hohn@localhost) by blab01.math.utah.edu (8.7.4/8.7.3) id UAA28261; Tue, 9 Apr 1996 20:05:30 -0600 (MDT) Date: Tue, 9 Apr 1996 20:05:30 -0600 (MDT) Message-Id: <199604100205.UAA28261@blab01.math.utah.edu> From: Michael Hohn To: matrix-sig@larch.python.org Sender: owner-matrix-sig Precedence: bulk content-length: 183 help archive ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Apr 20 18:58:28 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id SAA02736; Sat, 20 Apr 1996 18:56:19 -0400 Received: from monet.math.chalmers.se by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id SAA02729; Sat, 20 Apr 1996 18:56:07 -0400 Received: (from hansbo@localhost) by monet.math.chalmers.se (8.7.5/8.7.3) id AAA29610 for matrix-sig@python.org; Sun, 21 Apr 1996 00:54:32 +0200 (MET DST) Date: Sun, 21 Apr 1996 00:54:32 +0200 (MET DST) From: Peter Hansbo Message-Id: <199604202254.AAA29610@monet.math.chalmers.se> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Whereabouts of numerical python Sender: owner-matrix-sig Precedence: bulk content-length: 896 Hi there! Allow me to introduce myself: I am a finite element researcher at a Swedish university. One of the projects I have been involved in is a FE GUI for teaching purposes, with automatic mesh generation and different adaptive solvers, see http://www.math.chalmers.se/Research/Femlab/femlab.html Now I have found this thing called Python, and it really makes me want to get out of the F77/F90/C/C++ straightjackets. However, speed issues force me to consider C extensions. Then I notice the work of this SIG, with promises of just 10 percent overhead in floating point arithmetic, which I can live with. My question now is just this: can I put my hands on a beta version of the numerical python? Regards Peter Hansbo ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sat Apr 20 19:39:59 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id TAA02937; Sat, 20 Apr 1996 19:37:39 -0400 Received: from atlantis.actrix.gen.nz by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id TAA02930; Sat, 20 Apr 1996 19:37:11 -0400 Received: from sputnik3.pac.gen.nz (sputnik3.actrix.gen.nz [192.100.53.70]) by atlantis.actrix.gen.nz (8.6.11/8.6.9) with SMTP id LAA26124 for ; Sun, 21 Apr 1996 11:35:32 +1200 Date: Sun, 21 Apr 1996 11:35:30 +1200 (NZST) From: Michael Hamilton X-Sender: michael@sputnik3.pac.gen.nz To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] NetCDF and matrix extensions? Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig Precedence: bulk content-length: 1098 I was wondering if anyone has a NetCDF module that uses the matrix extension. I made some tentative enquiries in this direction earlier, in comp.lang.python, and was pleased to see that some things were pending. Unfortunately I accidentally blew away some email when I upgraded my Linux box, so I've lost the info. So... Is anything available? I have a project that could use this stuff right now. My objective is to demonstrate a small application written using NetCDF in the next week or two? I'm quite happy to collect and build beta code - so long as it mostly works, and so long as the future changes to the matrix support won't require me to totally rewrite my code (I don't mind minor rewrites). My other alternatives are perl and Tcl, but I'd like an environment that fosters readable/maintainable scripts. Thanks. --- Michael Hamilton (Happy Linux user since 0.11) http://www.actrix.gen.nz/users/michael ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Apr 21 04:22:16 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id EAA04029; Sun, 21 Apr 1996 04:22:03 -0400 Received: from gate.artinet.de by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id EAA04022; Sun, 21 Apr 1996 04:21:30 -0400 Received: from samba.artinet.de by gate.artinet.de with smtp (Linux Smail3.1.29.1 #2) id m0uAuMG-0001ZRC; Sun, 21 Apr 96 10:19 MET DST Received: by samba.artinet.de (Smail3.1.29.1 #3) id m0uAuME-0007EEC; Sun, 21 Apr 96 10:18 MET DST Date: Sun, 21 Apr 1996 10:18:58 +0200 (MET DST) From: Tom Schwaller To: Michael Hamilton cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] NetCDF and matrix extensions? In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig Precedence: bulk content-length: 622 Some time ago I contacted a person which made some remarks about it in the newsgroup and he told me to make it public as soon as possible. Unfortunately I do not remember his name. I suggest to post your message in the newsgroup, I'm sure we will get some answers. As far as I can see he wrote the wrapper code for the NetCFD API, nothing more. Tom (do not use et@appl-math.tu-muenchen.de anymore, I'm working for the German Linux Journal now! :-) ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Sun Apr 21 04:47:07 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id EAA04059; Sun, 21 Apr 1996 04:33:26 -0400 Received: from gate.artinet.de by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id EAA04052; Sun, 21 Apr 1996 04:33:18 -0400 Received: from samba.artinet.de by gate.artinet.de with smtp (Linux Smail3.1.29.1 #2) id m0uAuW0-0001ZRC; Sun, 21 Apr 96 10:29 MET DST Received: by samba.artinet.de (Smail3.1.29.1 #3) id m0uAuVy-0007EEC; Sun, 21 Apr 96 10:29 MET DST Date: Sun, 21 Apr 1996 10:29:02 +0200 (MET DST) From: Tom Schwaller To: Peter Hansbo cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Whereabouts of numerical python In-Reply-To: <199604202254.AAA29610@monet.math.chalmers.se> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig Precedence: bulk content-length: 674 Concernig FEM stuff with Python, you should check ftp://ftp.appl-math.tu-muenchen.de/pub/et/ delaunaymodule-0.33.c.gz trimodule.tgz I once considered to pythonify femlab, but now that you are with us, you should give it a try yourself, would be very nice to have a FEM package with Python. Which needs much more design issues to be solved compared to the matrixmodule. BTW My Mail of the last 3 week (also the one from the matrix and other sigs) is lost, did I miss something? Tom German Linux Journal ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Apr 22 08:29:10 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id IAA06799; Mon, 22 Apr 1996 08:25:36 -0400 Received: from monet.math.chalmers.se by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id IAA06792; Mon, 22 Apr 1996 08:25:32 -0400 Received: (from hansbo@localhost) by monet.math.chalmers.se (8.7.5/8.7.3) id OAA18644 for matrix-sig@python.org; Mon, 22 Apr 1996 14:23:47 +0200 (MET DST) Date: Mon, 22 Apr 1996 14:23:47 +0200 (MET DST) From: Peter Hansbo Message-Id: <199604221223.OAA18644@monet.math.chalmers.se> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Numerical P. in standard dist.? Sender: owner-matrix-sig Precedence: bulk content-length: 597 One of the important things with Python is the excellent portability. Now, Numerical python at sls-ftp.lcs.mit.edu is a Unix-variant, which is natural, seeing as mot numerical work is done on workstations. Unfortunately I do a lot of developing on a Mac, which raises the following question: has it been decided when and if Numerical Python will be a part of the standard distribution? What timespan is to be expected? P. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Mon Apr 22 12:21:55 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id MAA07984; Mon, 22 Apr 1996 12:17:19 -0400 Received: from netcom8.netcom.com by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id MAA07974; Mon, 22 Apr 1996 12:17:05 -0400 Received: from [10.0.2.15] (demars@localhost.netcom.com [127.0.0.1]) by netcom8.netcom.com (8.6.13/Netcom) id JAA02466; Mon, 22 Apr 1996 09:15:25 -0700 X-Sender: demars@10.0.2.2 Message-Id: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Mon, 22 Apr 1996 09:15:33 -0700 To: matrix-sig@python.org From: demars@netcom.com (Dennis C. De Mars) Subject: Re: [PYTHON MATRIX-SIG] Numerical P. in standard dist.? Sender: owner-matrix-sig Precedence: bulk content-length: 1383 >One of the important things with Python >is the excellent portability. Now, Numerical python >at sls-ftp.lcs.mit.edu is a Unix-variant, which >is natural, seeing as mot numerical work is done >on workstations. Unfortunately I do a lot of developing >on a Mac, which raises the following question: >has it been decided when and if Numerical Python will be >a part of the standard distribution? What timespan is >to be expected? I would guess that the folks who are doing this work are not going to put it into the standard distribuition until it is ready for release; right now Numerical Python is still a work in progress. (However, if there is a target date for putting this stuff into the standard distribution, I'd like to hear it too!) However, there is nothing to stop you from taking the standard distribution source (which include Mac build files) and the source for Numerical Python and creating your own build. That's what I did. BTW, the Mac version itself is in a state of flux due to changes to support Tk and Mac native system calls. However, I expect this to settle down soon and it should be possible to create a "grand unified" Mac build that supports Numerical Python _and_ Tk. - Dennis D. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Tue Apr 23 17:46:16 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id RAA12024; Tue, 23 Apr 1996 17:41:02 -0400 Received: from harfang.CC.UMontreal.CA by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id RAA12016; Tue, 23 Apr 1996 17:40:45 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id RAA14219 (8.6.11/IDA-1.6 for ); Tue, 23 Apr 1996 17:37:57 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id RAA17601; Tue, 23 Apr 1996 17:37:56 -0400 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id RAA21300; Tue, 23 Apr 1996 17:37:55 -0400 Date: Tue, 23 Apr 1996 17:37:55 -0400 Message-Id: <199604232137.RAA21300@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Plotting with Mathematica From: hinsenk@ERE.UMontreal.CA (Konrad HINSEN) Sender: owner-matrix-sig Precedence: bulk content-length: 3826 While we are all waiting for a good plotting package for Python, I offer a simplistic, but very convenient temporary solution to those who have Mathematica and run Unix. The following module is hardly documented, not carefully designed, and completely unsupported, but it works for me. # Interface to Mathematica for plotting # # Written by Konrad Hinsen # last revision: 1996-4-23 import os, regsub, tempfile # Class representing a Mathematica process class Mathematica: def __init__(self, progname = 'math'): self.progname = progname self.script = '' self.font = "Courier" self.textsize = 10 def clear(self): self.script = '' def execute(self): scriptfile = tempfile.mktemp() file = open(scriptfile, 'w') file.write(self.script) file.close() os.system(self.progname + ' -record "" <' + scriptfile + ' >/dev/null') os.unlink(scriptfile) def command(self, command): self.script = self.script + command def backup(self, n): self.script = self.script[:-n] def defineVariable(self, name, array): self.command(name + ' = ') self.command(formatValue(array)) self.command('\n') def setFont(self, font, size): self.font = font self.textsize = size def displayOptions(self): s = 'DefaultFont->{"' + self.font + '",' + \ formatValue(self.textsize) + '}' return s def plot(self, data): s = 'Show[' for dataset in data: s = s + 'ListPlot[' + formatValue(dataset) + \ ', PlotJoined->True, DisplayFunction->Identity, ' + \ self.displayOptions() + '], ' s = s + 'DisplayFunction->$DisplayFunction]\n' self.command(s) def contourPlot(self, xaxis, yaxis, data, contours=None): s = 'Show[ContourGraphics[' + formatValue(data.transpose()) \ + ', MeshRange->' \ + formatValue([[xaxis[0], xaxis[-1]], [yaxis[0], yaxis[-1]]]) \ + ', ContourShading->False' if contours: s = s + ', Contours->' + formatValue(contours) s = s + ', ' + self.displayOptions() + ']]\n' self.command(s) # Format scalars, arrays, and nested lists for Mathematica def formatValue(x): is_sequence = 1 try: x[0] except: is_sequence = 0 if is_sequence: s = '{' for e in x: s = s + formatValue(e) + ', ' s = s[:-2] + '}' else: if type(x) == type(''): s = '"' + x + '"' elif type(x) == type(0): s = `x` elif type(x) == type(0.): s = regsub.sub('e','*10^', `x`) elif type(x) == type(0.j): s = '(' + regsub.sub('e','*10^', `x.real`) + \ '+' + regsub.sub('e','*10^', `x.imag`) + 'I)' else: raise TypeError, 'Unknown type ' + `type(x)` return s # Simple plot functions def plot(*data): m = Mathematica() m.plot(data) m.execute() def contourPlot(xaxis, yaxis, data, contours=None): m = Mathematica() m.contourPlot(xaxis, yaxis, data, contours) m.execute() # These examples are all the documentation you will get! if __name__ == '__main__': from Numeric import * from umath import * plot([4,6,5,3]) plot([(3,6.8),(4,4.2),(5,0.5)]) x = arange(10) y = arange(15) data = x[:, NewAxis]*sin(y/2.) contourPlot(x, y, data, arange(0.,10.,0.1)) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Apr 24 11:32:06 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id LAA14478; Wed, 24 Apr 1996 11:27:53 -0400 Received: from estel.uindy.edu by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id LAA14471; Wed, 24 Apr 1996 11:27:50 -0400 Received: by estel.uindy.edu (NX5.67d/NX3.0M) id AA00917; Wed, 24 Apr 96 10:30:23 -0500 Date: Wed, 24 Apr 96 10:30:23 -0500 From: Steve Spicklemire Message-Id: <9604241530.AA00917@estel.uindy.edu> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] URNG module.... Sender: owner-matrix-sig Precedence: bulk content-length: 573 Hi folkx, I was trying to use the MLab.py stuff and I realized I needed the URNG module for part of it... (well I suppose I could cut that out.. but I thought I'd try here first..). Upon trying to build URNG on my NeXT 3.2 system I get: ld: Undefined symbols: _drand48 _seed48 _lcong48 *** Exit 1 Stop. *** Exit 1 Stop. Where can I find these.. is there more source I need to get? thanks, -steve ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Apr 24 18:00:07 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id RAA16310; Wed, 24 Apr 1996 17:57:14 -0400 Received: from lassi.ece.uiuc.edu by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id RAA16303; Wed, 24 Apr 1996 17:57:08 -0400 From: Message-Id: <199604242157.RAA16303@python.org> Received: by lassi.ece.uiuc.edu (1.38.193.4/16.2) id AA26561; Wed, 24 Apr 1996 16:54:13 -0500 Date: Wed, 24 Apr 1996 16:54:13 -0500 To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] strange slice behaviour Sender: owner-matrix-sig Precedence: bulk content-length: 1677 I just recently subscribed to the matrix-sig, so I appologize if this has gone over allready (I've tried to look over the archive, but its rather large.) Thanks to everyone for putting Numerical Python together - it looks great. I have a 2D array and I slice it, it works fine for non-zero length slices, but behaves strangely (IMHO) for zero length slices. It works fine for 1D arrays though... >>> from Numeric import * >>> a = array([[1,2,3],[4,5,6],[7,8,9]]) >>> a 1 2 3 4 5 6 7 8 9 >>> a[0:2,1] # OK 2 5 >>> a[0:1,1] # OK 2 >>>>>> a[0:0,1] # This is strange !!! 2 5 8 >>> b = array([1,2,3]) >>> b 1 2 3 >>> b[0:0] # Oops this breaks the PrettyPrinter Traceback (innermost last): File "", line 1, in ? File "/usr/local/lib/python/numeric/PrettyPrinter.py", line 28, in arrayToString max_str_len = max(len(str(maximum.reduce(data))), ArrayError: can't take anything from an empty array >>> >>> b[0:0].shape # This gives what I'd expect. (0,) Is this the expected behaviour? If it is, does anyone know of a nice workaround to get the behaviour I'd expect (i.e., a zero size matrix)? Thanks, -- -tim +--------------------------------------------------------------------+ | Tim Hochberg Ultrahigh Speed Digital Electronics Lab | | tim@lassi.ece.uiuc.edu University of Illinois | | http://dogbert.ece.uiuc.edu/~tim (217) 333-6014 | +--------------------------------------------------------------------+ ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Apr 24 18:41:56 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id SAA16503; Wed, 24 Apr 1996 18:36:22 -0400 Received: from harfang.CC.UMontreal.CA by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id SAA16496; Wed, 24 Apr 1996 18:36:09 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id RAA17548 (8.6.11/IDA-1.6 for ); Wed, 24 Apr 1996 17:27:57 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id RAA18641; Wed, 24 Apr 1996 17:27:56 -0400 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id RAA20328; Wed, 24 Apr 1996 17:27:56 -0400 Date: Wed, 24 Apr 1996 17:27:56 -0400 Message-Id: <199604242127.RAA20328@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] FFT From: hinsenk@ERE.UMontreal.CA (Konrad HINSEN) Sender: owner-matrix-sig Precedence: bulk content-length: 832 Does anyone have a working 1-d FFT package? I have to calculate the autocorrelation of long time series, and the "straightforward" approach is much too slow for me. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Apr 24 19:04:24 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id SAA16674; Wed, 24 Apr 1996 18:59:47 -0400 Received: from meter.eng.uci.edu by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id SAA16667; Wed, 24 Apr 1996 18:59:27 -0400 Received: from trout.ece.uci.edu by meter.eng.uci.edu (8.7.4) id PAA23405; Wed, 24 Apr 1996 15:57:48 -0700 (PDT) Received: by trout.ece.uci.edu (5.x) id AA06017; Wed, 24 Apr 1996 15:57:42 -0700 Date: Wed, 24 Apr 1996 15:57:39 -0700 (PDT) From: Prashanth Mundkur To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] strange slice behaviour In-Reply-To: <199604242157.RAA16303@python.org> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig Precedence: bulk content-length: 368 I just subscribed to this list yesterday, but I haven't been yet been able to locate the Numeric module. Where do I find it, along with the install instructs? Thanks for any help. --prashanth ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Apr 24 19:03:46 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id TAA16782; Wed, 24 Apr 1996 19:00:29 -0400 Received: from harfang.CC.UMontreal.CA by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id SAA16676; Wed, 24 Apr 1996 18:59:56 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id SAA20250 (8.6.11/IDA-1.6); Wed, 24 Apr 1996 18:57:00 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id SAA28303; Wed, 24 Apr 1996 18:56:59 -0400 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id SAA27372; Wed, 24 Apr 1996 18:56:58 -0400 Date: Wed, 24 Apr 1996 18:56:58 -0400 Message-Id: <199604242256.SAA27372@cyclone.ERE.UMontreal.CA> To: tim@lassi.ece.uiuc.edu CC: matrix-sig@python.org In-reply-to: <199604242157.RAA16303@python.org> (tim@lassi.ece.uiuc.edu) Subject: Re: [PYTHON MATRIX-SIG] strange slice behaviour From: hinsenk@ERE.UMontreal.CA (Konrad HINSEN) Sender: owner-matrix-sig Precedence: bulk content-length: 939 > I have a 2D array and I slice it, it works fine for non-zero length > slices, but behaves strangely (IMHO) for zero length slices. It works > fine for 1D arrays though... There are many operations that don't work correctly with empty arrays at the moment. Just wait... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed Apr 24 19:41:27 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id TAA17131; Wed, 24 Apr 1996 19:38:12 -0400 Received: from csc-sun.math.utah.edu by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id TAA17124; Wed, 24 Apr 1996 19:38:08 -0400 Received: from blab02.math.utah.edu (hohn@blab02.math.utah.edu [128.110.198.250]) by csc-sun.math.utah.edu (8.7.4/8.7.3) with ESMTP id RAA23723 for ; Wed, 24 Apr 1996 17:36:31 -0600 (MDT) Received: (from hohn@localhost) by blab02.math.utah.edu (8.7.4/8.7.3) id RAA21348; Wed, 24 Apr 1996 17:36:29 -0600 (MDT) Date: Wed, 24 Apr 1996 17:36:29 -0600 (MDT) Message-Id: <199604242336.RAA21348@blab02.math.utah.edu> From: Michael Hohn To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] matlab loading/saving Sender: owner-matrix-sig Precedence: bulk content-length: 359 After being unable to find a MATLAB binary file load routine, I have started writing one to handle dense matrices. If someone has already done this, please let me know :) Thanks. Mike ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Apr 25 16:45:04 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id QAA20434; Thu, 25 Apr 1996 16:41:25 -0400 Received: from goldilocks.LCS.MIT.EDU by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id QAA20427; Thu, 25 Apr 1996 16:41:22 -0400 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU by goldilocks.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA06062; Thu, 25 Apr 96 16:39:44 EDT Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA14995; Thu, 25 Apr 96 16:39:39 EDT Date: Thu, 25 Apr 96 16:39:39 EDT From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9604252039.AA14995@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Release 0.36 is available! Sender: owner-matrix-sig Precedence: bulk content-length: 739 Release 0.36 of the numerical extension to python is available at: ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericalPython-0.36.tar.gz There is no patch available for this release for several minor reasons. My apologies to overseas users for the size. If somebody would like to make this available in Europe/Asia, that would be welcome. New in this release: Numerous small bug fixes. This should now compile on Windows and Macintosh platforms. Added functions argsort, and position. Surely you don't want documentation? The TestSuite works again! Enjoy - Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu Apr 25 17:11:54 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id RAA20592; Thu, 25 Apr 1996 17:08:35 -0400 Received: from goldilocks.LCS.MIT.EDU by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id RAA20585; Thu, 25 Apr 1996 17:08:32 -0400 Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU by goldilocks.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA06353; Thu, 25 Apr 96 17:06:55 EDT Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA15086; Thu, 25 Apr 96 17:06:50 EDT Date: Thu, 25 Apr 96 17:06:50 EDT From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin) Message-Id: <9604252106.AA15086@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] FAQ Sender: owner-matrix-sig Precedence: bulk content-length: 2838 Here's my first step at creating a FAQ for this project. If anybody else wants to take it up, please feel free. This is going to be very free form, off the top of my head and unedited. Still, I anticipate that it will be useful. Occaionally I will steal text entirely from other people on the SIG without any attribution. I'll try to fix this later. Written by Jim Hugunin (hugunin@mit.edu) on April 25, 1996. 1) What's Numerical Python? Here should really go Paul's humorous depiction of Monty's more serious brother, but I don't have that right now, so the high level stuff will have to wait. 2) Where do I get it? ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericalPython-0.36.tar.gz This is the latest version. New versions will be made available at this location. 3) Is there any documentation? There's incomplete online documentation written by David Ascher for a course he taught using Numerical Python at: http://maigret.cog.brown.edu/python/arraytut.html There's a paper soon to be published in Computers and Physics written by Dubois, Hinsen and Hugunin at: ftp://ftp-icf.llnl.gov/pub/basis/numerical_python.ps There's my talk from the 3rd python workshop at: http://www.python.org/workshops/1995-12/papers/hugunin.html 4) What modules are available that use Numerical python? (Note: I know that this needs pointers, I just don't have the time to put them together today) Tom Schwaller's delaunaymodule and trimodule Tom's and my opengl, glu, and glut modules Paul DuBois' URNGmodule Doug Heisterkamp's interface to the LAPACK libraries at: ftp://ftp.cs.unl.edu/pub/drh/python/pylapack.0.02.tar.gz 5) What are the future plans? First goal is a general release to the python community. Before this happens, I want a somewhat finalized API (which Konrad seems to have provided) implemented (for which I just need some free time). I don't want to have too many users developing code with the system before the API is at least closer to its final form (to minimize the changes they need to make). On the other hand, the C API is essentially final. I'm perfectly willing to guarantee that I won't introduce any major incompatibilities in subsequent release of the system. This means that people have no excuse not to develop modules to interface to all that great C/FORTRAN numerical code out there. Obviously the ultimate goal is to have this a part of the base python distribution. I'll start thinking more about this once I make the general release to the community at large. 6) Hints for Windows Users You must use binary mode files for pickling and unpickling matrices in the windows world. Blame Bill for the silliness. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Apr 26 05:19:44 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id FAA22211; Fri, 26 Apr 1996 05:16:20 -0400 Received: from stork.EMBL-Heidelberg.DE by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id FAA22204; Fri, 26 Apr 1996 05:15:32 -0400 Received: from nu.EMBL-Heidelberg.DE (nu [192.54.41.120]) by stork.EMBL-Heidelberg.DE (8.7.1/8.7.1) with SMTP id LAA10445; Fri, 26 Apr 1996 11:11:35 +0200 (MDT) Received: by nu.EMBL-Heidelberg.DE (8.6.11) id LAA17540; Fri, 26 Apr 1996 11:11:25 +0200 Date: Fri, 26 Apr 1996 11:11:25 +0200 Message-Id: <199604260911.LAA17540@nu.EMBL-Heidelberg.DE> From: Rob Hooft To: James Hugunin Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Release 0.36 is available! In-Reply-To: <9604252039.AA14995@baribal.LCS.MIT.EDU.LCS.MIT.EDU> References: <9604252039.AA14995@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Mime-Version: 1.0 (generated by tm-edit 7.52) Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig Precedence: bulk content-length: 1050 >>>>> "JH" == James Hugunin writes: JH> ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericalPython-0.36.tar.gz JH> If somebody would like to make this available in Europe/Asia, JH> that would be welcome. I copied the whole directory as soon as I could get through (even from New York at 6 in the morning that was not an easy task). Now as a special offer to European members of the matrix-sig list, it is available from: ftp://swift.embl-heidelberg.de/pub/Numeric/ Please note that this is NOT an automatic mirror! Regards, -- === Rob.Hooft@EMBL-Heidelberg.DE http://www.Sander.EMBL-Heidelberg.DE/rob/ == ==== In need of protein modeling? http://www.Sander.EMBL-Heidelberg.DE/whatif/ Validation of protein structures? http://biotech.EMBL-Heidelberg.DE:8400/ ==== ================ Long Live Linux! Free Software Rules The World! ============= ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Apr 26 11:59:35 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id LAA22938; Fri, 26 Apr 1996 11:56:01 -0400 Received: from rnet.rose.rsoc.rockwell.com by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id LAA22931; Fri, 26 Apr 1996 11:55:55 -0400 Received: by rnet.rose.rsoc.rockwell.com (4.1/SMI-4.1) id AA04982; Fri, 26 Apr 96 10:54:15 CDT Received: from sunrise(161.40.85.21) by rnet via smap (V1.3) id sma004979; Fri Apr 26 10:54:06 1996 Received: from darwin.rsoc.rockwell.com by rose.rsoc.rockwell.com (SMI-8.6/SMI-SVR4) id KAA07594; Fri, 26 Apr 1996 10:53:59 -0500 Received: by darwin.rsoc.rockwell.com (SMI-8.6/SMI-SVR4) id KAA18756; Fri, 26 Apr 1996 10:53:53 -0500 Date: Fri, 26 Apr 1996 10:53:53 -0500 From: friedric@rose.rsoc.rockwell.com (Robin Friedrich) Message-Id: <199604261553.KAA18756@darwin.rsoc.rockwell.com> To: matrix-sig@python.org, jjh@goldilocks.lcs.mit.edu Subject: Re: [PYTHON MATRIX-SIG] FAQ X-Sun-Charset: US-ASCII Sender: owner-matrix-sig Precedence: bulk content-length: 1337 Hi Jim, Whipped up this little script to htmlize your FAQ. Thought you might like it. #!/bin/env python # Script to htmlize Jim's FAQ. Might be generally useful though. import regsub, regex, sys try: file = sys.argv[1] except IndexError: print 'Usage: '+sys.argv[0]+' filename' print 'The processed file will have the same name with a .html suffix added.' outname = file+'.html' out = open(outname, 'w') httppat = regex.compile('\(http:[^ \n]+\)') ftppat = regex.compile('\(ftp:[^ \n]+\)') mailpat = regex.compile('\(\w+@[^ ()\n]+\)') headpat = regex.compile('\(^[0-9]+).*$\)') trailpat = regex.compile('\([^ ]$\)') text = open(file).read() out.write('Numerical Python FAQ\n') #change this to taste out.write('\n') text = regsub.gsub(httppat, '\\1', text) text = regsub.gsub(ftppat, '\\1', text) text = regsub.gsub(mailpat, '\\1', text) text = regsub.gsub(headpat, '

\\1

', text) text = regsub.gsub('\n\n', '\n

', text) text = regsub.gsub(trailpat, '\\1 ', text) out.write(text) out.write('\n\n') print 'Done. Output may be found in '+outname ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Fri Apr 26 17:02:02 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id QAA23857; Fri, 26 Apr 1996 16:58:28 -0400 Received: from harfang.CC.UMontreal.CA by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id QAA23850; Fri, 26 Apr 1996 16:58:24 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id KAA09479 (8.6.11/IDA-1.6); Fri, 26 Apr 1996 10:44:26 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id KAA15809; Fri, 26 Apr 1996 10:43:47 -0400 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id KAA09879; Fri, 26 Apr 1996 10:43:46 -0400 Date: Fri, 26 Apr 1996 10:43:46 -0400 Message-Id: <199604261443.KAA09879@cyclone.ERE.UMontreal.CA> To: jjh@goldilocks.lcs.mit.edu CC: matrix-sig@python.org In-reply-to: <9604252106.AA15086@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (message from James Hugunin on Thu, 25 Apr 96 17:06:50 EDT) Subject: Re: [PYTHON MATRIX-SIG] FAQ From: hinsenk@ERE.UMontreal.CA (Konrad HINSEN) Sender: owner-matrix-sig Precedence: bulk content-length: 2432 > 4) What modules are available that use Numerical python? > > (Note: I know that this needs pointers, I just don't have the time to > put them together today) ... I have a couple of Python modules using arrays/umath, and I bet I am not the only one. Maybe it would be a good idea to start a collection somewhere. Even if the API is not final yet, this would be useful to 1) Have useful code right now 2) Prevent duplicated development effort 3) Provide examples to newcomers to this SIG In the first public release this code could then appear as examples. My current collection of modules of general interest is: Derivatives.py Automatic derivatives; uses umath, but no arrays (yet). Interpolation.py Represents functions defined by numerical values on a grid. Provides linear interpolation, differentiation, and integration. LeastSquares.py Levenberg-Marquardt algorithm to find a least-squares solution for nonlinear problems. PhysicalQuantities.py Represents physical quantities with units; handles mixed-unit arithmetic and unit conversion. Uses umath, but not arrays. Vector.py 3D geometrical vectors, uses umath. A version based on arrays exists, but makes sense only when arrays of general objects work reliably (I have to test that with the new release). Special-interest code: G90Output.py Extracts information from output files of Gaussian 90. Potential.py Automatic calculation of first and second derivatives for potential functions. wham.py Calculates a potential of mean force from umbrella sampling data; works for arbitrary number of variables. All of these have been used for real-life applications and can be considered working, although as always there is absolutely no warranty... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu May 2 14:21:16 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id OAA09440; Thu, 2 May 1996 14:19:04 -0400 Received: from studbolt.mit.edu by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id OAA09433; Thu, 2 May 1996 14:19:01 -0400 Received: (from templon@localhost) by studbolt.mit.edu (8.6.13/8.6.9) id OAA00861; Thu, 2 May 1996 14:17:19 -0400 Date: Thu, 2 May 1996 14:17:19 -0400 From: Jeff Templon Message-Id: <199605021817.OAA00861@studbolt.mit.edu> To: matrix-sig@python.org CC: Alex Cannon , templon@studbolt.mit.edu Subject: [PYTHON MATRIX-SIG] any matlab-like python extensions??? Sender: owner-matrix-sig Precedence: bulk content-length: 1756 Hi, I had posted the following on comp.lang.python, but someone pointed out that it was better posted here. Since my newsgroup posting, I've heard from at least one other person who is interested in the results. Below is the content of my comp.lang.python post. Please respond to me via email, as I am not on the matrix-sig list (perhaps yet.) Also, please cc your message to Alex Cannon who is also very interested in what you have to say. Jeff Templon wrote: > > Hi, > > I'm looking to do some numerical programming, and would like to do it > in Python. I know about the new Matrix-SIG activities, but I am also > interested in a graphical display of the results. I've looked at SciLab, > which is a sort of PD version of Matlab, and this is exactly what > I'm looking for except it's not Python (although a lot of the syntax > is close!) > > The thing in SciLab that Python (as far as I know) is missing, is a > package of plotting primitives. In Scilab, you can say something like > "plot (vector1, vector2)" where vector1 is some array of x values and > vector2 is an array of y values; you can also give three vectors and > get a surface or contour plot (or even both, with the contour plot > overlaid on the surface!) Is there anything like this for Python? > I'd guess it'd be interfaced to Tk or SUIT or somesuch. > > I know I could do a poor-man's interface using a pipe to Gnuplot. > I'm considering that as a first step. Any info on products or > things in the works is welcomed. > > Jeff Templon ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu May 2 16:40:10 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id QAA10029; Thu, 2 May 1996 16:34:48 -0400 Received: from gate.artinet.de by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id QAA10022; Thu, 2 May 1996 16:34:41 -0400 Received: from samba.artinet.de by gate.artinet.de with smtp (Linux Smail3.1.29.1 #2) id m0uF53Q-0001d5C; Thu, 2 May 96 22:32 MET DST Received: by samba.artinet.de (Smail3.1.29.1 #3) id m0uF53N-0007EEC; Thu, 2 May 96 22:32 MET DST Date: Thu, 2 May 1996 22:32:45 +0200 (MET DST) From: Tom Schwaller To: Jeff Templon cc: matrix-sig@python.org, Alex Cannon , templon@studbolt.mit.edu Subject: Re: [PYTHON MATRIX-SIG] any matlab-like python extensions??? In-Reply-To: <199605021817.OAA00861@studbolt.mit.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig Precedence: bulk content-length: 1030 Hi Jeff, well try the plplotmodule I've written (interfaces the plplot library). It is not pefect, but has a Tk widget with it, which no other package has. I one worked on a Python module interfacing the Visual toolkit++, which is a very tool cool, but I'm a little bit short in time for it. Otherwise you can also try the OpenGL module By myself and Jim H.) Join this group, that's the right place for you. Here you will also learn about where to get all that stuff and ther are many people , which will help you I guess. BTW. Have some of you tried out pgplot (Fortran library, no Tk widget, but very nice output, Interfacing it to Python is as easy as plplot, somebody going to do it? There's already a Tcl and Perl Module (I hear you SWIG! :-) We should at least do a Python module. Look at the URL http://astro.caltech.edu/~tjp/pgplot/ Comments Tom ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Thu May 2 17:12:44 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id RAA10377; Thu, 2 May 1996 17:08:03 -0400 Received: from drew.cog.brown.edu by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id RAA10370; Thu, 2 May 1996 17:07:51 -0400 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA20324; Thu, 2 May 96 17:00:48 -0400 Received: by maigret (950215.SGI.8.6.10/940406.SGI) id RAA09570; Thu, 2 May 1996 17:01:53 -0400 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199605022101.RAA09570@maigret> Subject: Re: [PYTHON MATRIX-SIG] any matlab-like python extensions??? To: tschwal@artinet.de (Tom Schwaller) Date: Thu, 2 May 1996 17:01:52 -0400 (EDT) Cc: templon@studbolt.mit.edu, matrix-sig@python.org, acannon@geog.ubc.ca In-Reply-To: from "Tom Schwaller" at May 2, 96 10:32:45 pm X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig Precedence: bulk content-length: 650 > BTW. Have some of you tried out pgplot (Fortran library, no Tk widget, > but very nice output, Interfacing it to Python is as easy as plplot, > somebody going to do it? > There's already a Tcl and Perl Module (I hear you SWIG! :-) > We should at least do a Python module. > Look at the URL > http://astro.caltech.edu/~tjp/pgplot/ I'll look at it. My favorite output (sober but very professional) comes out of GLE, which alas is nothing one can interface to easily. --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed May 8 15:53:05 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id PAA16162; Wed, 8 May 1996 15:46:12 -0400 Received: from drew.cog.brown.edu by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id PAA16155; Wed, 8 May 1996 15:45:58 -0400 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA15551; Wed, 8 May 96 14:41:00 -0400 Received: by maigret (950215.SGI.8.6.10/940406.SGI) for matrix-sig@python.org id OAA22407; Wed, 8 May 1996 14:41:47 -0400 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199605081841.OAA22407@maigret> Subject: [PYTHON MATRIX-SIG] Tela/Python conversion help needed To: matrix-sig@larch.python.org Date: Wed, 8 May 1996 14:41:46 -0400 (EDT) X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig Precedence: bulk content-length: 2758 Hi folks. I've ported the fftpack routines which are part of Tela into a python module, which is fine. Now I need to port a little piece of Tela code which does the n-dimensional fft's out of the 1-d fft routines. I've given it a try, but my nonexistent knowledge of tela and my lack of experience with multidimensional fft's makes it a hard thing for me to debug. Does anyone out there feel willing to help me with this conversion? Let me know. --david PS: Tela seems to have a nice set of routines which I'd like to see in Python. What do you think? From the "Tela Feature List": ----------------------------------------------------------------------- Linear algebra: matrix inversion, matrix product, eigenvalues, eigenvectors, determinant, LU, Cholesky, singular value decompositions and linear system solution. Full set of fast Fourier transform routines. In addition to ordinary complex and real FFTs there are sine, cosine, plus quarter-wave sine and cosine transforms. All routines can work along any dimension in an N-dimensional real or complex array, making it possible e.g. to solve any constant-coefficient elliptic boundary value problem in 2D or 3D with any boundary conditions (Dirichlet, Neumann or periodic type; each boundary may have its own type of boundary condition). An example how this is done is included in the distribution (testpoisson.t). Unix commands can be run under Tela control. Standard input and output can be mapped to Tela strings. C stdio like functions for manipulating files. Can work with HDF, netCDF (vers.>=1.23), plain ASCII, MATLAB binary files (MAT-files) and PBM (PPM, PGM) image files directly. Graphics can be saved as PostScript and GIF from the PlotMTV program. Can interface to XV. Graphics: 2D and 3D line and curve plots, contour plots, density plots, vector field plots and surface plots. Cartesian intersection surfaces in 3D space. Bar charts and histograms. Possibility to redraw existing windows or create new window for each new plot. Possibilities for overlaying and stacking plots and saving them in PostScript or GIF files. Possibility to annotate plots with text strings and simple geometric primitives. Palette can be changed and nonuniform grids can be used (vers>=1.23). Numerical analysis: linear interpolation (intpol), integration (integrate), function minimization (fmin), root finding (roots, fsolve), linear regression (fitline, fitlinear), nonlinear fitting (fitnl). Preliminary version of special function package (specfun.t), which contains a relatively complete set of special functions. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed May 8 16:07:06 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id PAA16194; Wed, 8 May 1996 15:57:04 -0400 Received: from drew.cog.brown.edu by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id PAA16187; Wed, 8 May 1996 15:56:38 -0400 Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA26693; Wed, 8 May 96 15:50:10 -0400 Received: by bekas (951211.SGI.8.6.12.PATCH1042/951211.SGI) for matrix-sig@python.org id PAA07821; Wed, 8 May 1996 15:50:04 -0400 From: da@bekas.cog.brown.edu (David Ascher) Message-Id: <199605081950.PAA07821@bekas> Subject: [PYTHON MATRIX-SIG] remind me To: matrix-sig@larch.python.org Date: Wed, 8 May 1996 15:50:02 -0400 (EDT) X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig Precedence: bulk content-length: 260 Is there a shorthand for: x2 = ravel(x) x2[n] = v x2.shape = x.shape x = x2 ? --da ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig Wed May 8 18:12:32 1996 Return-Path: Received: by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id RAA03169; Wed, 8 May 1996 17:57:00 -0400 Received: from harfang.CC.UMontreal.CA by python.org (SMI-8.6/SMI-SVR4 python.org klm d: sendmail.cf) id RAA02813; Wed, 8 May 1996 17:56:37 -0400 Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id QAA16659 (8.6.11/IDA-1.6); Wed, 8 May 1996 16:37:22 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id QAA09207; Wed, 8 May 1996 16:37:21 -0400 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id QAA28375; Wed, 8 May 1996 16:37:20 -0400 Date: Wed, 8 May 1996 16:37:20 -0400 Message-Id: <199605082037.QAA28375@cyclone.ERE.UMontreal.CA> To: da@bekas.cog.brown.edu CC: matrix-sig@larch.python.org In-reply-to: <199605081950.PAA07821@bekas> (message from David Ascher on Wed, 8 May 1996 15:50:02 -0400 (EDT)) Subject: Re: [PYTHON MATRIX-SIG] remind me From: hinsenk@ERE.UMontreal.CA (Konrad HINSEN) Sender: owner-matrix-sig Precedence: bulk content-length: 1002 > Is there a shorthand for: > > x2 = ravel(x) > x2[n] = v > x2.shape = x.shape > x = x2 The way ravel() is implemented now, the first two lines should be enough, i.e. they should change x. My API proposal contains a feature that would allow this to be written as x[......][n] = v I don't think you can get it much shorter... ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Tue May 14 20:15:30 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id UAA10130 for matrix-sig-people; Tue, 14 May 1996 20:07:23 -0400 (EDT) Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.7.5/8.7.3) with SMTPid UAA10086 for ; Tue, 14 May 1996 20:06:54 -0400 (EDT) Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA14942; Tue, 14 May 96 20:02:10 -0400 Received: by maigret (950215.SGI.8.6.10/940406.SGI) for matrix-sig@python.org id UAA12840; Tue, 14 May 1996 20:03:05 -0400 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199605150003.UAA12840@maigret> Subject: [PYTHON MATRIX-SIG] arange w/ fixed # of elements? To: matrix-sig@python.org Date: Tue, 14 May 1996 20:03:04 -0400 (EDT) X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk What is the recommended way to get an arange of a given # of elements between a min & and max? If I do: step = (max - min) / float(number) r = arange(min, max, step) then I am sometimes getting arrays w/ number + 1 elements, due to the odd case when max and min are exactly (or close enough) divisible by the stepsize. So, the following: from Numeric import * def test(s): x = 1.2*pi / s n = 300 st = x / 300 if len(arange(0,x,st)) == 301: print s for x in arange(-5,5,.05): if x != 0: test(x) shows that test(-0.8) and test(-1.6) on my machine yield arrays of size 301, while all the others yield arrays of size 300. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Tue May 14 21:12:42 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id VAA17135 for matrix-sig-people; Tue, 14 May 1996 21:09:22 -0400 (EDT) Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.7.5/8.7.3) with ESMTPid VAA17079 for ; Tue, 14 May 1996 21:08:55 -0400 (EDT) Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id VAA13021 (8.6.11/IDA-1.6); Tue, 14 May 1996 21:07:17 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id VAA15647; Tue, 14 May 1996 21:07:16 -0400 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id VAA29247; Tue, 14 May 1996 21:07:15 -0400 Date: Tue, 14 May 1996 21:07:15 -0400 Message-Id: <199605150107.VAA29247@cyclone.ERE.UMontreal.CA> To: da@maigret.cog.brown.edu CC: matrix-sig@python.org In-reply-to: <199605150003.UAA12840@maigret> (da@maigret.cog.brown.edu) Subject: Re: [PYTHON MATRIX-SIG] arange w/ fixed # of elements? From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > What is the recommended way to get an arange of a given # of elements > between a min & and max? > > If I do: > step = (max - min) / float(number) > r = arange(min, max, step) > > then I am sometimes getting arrays w/ number + 1 elements, due to > the odd case when max and min are exactly (or close enough) divisible > by the stepsize. Try: min+(max-min)*arange(number)/number There are no problems with roundoff error in this case. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Wed May 15 14:16:44 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id OAA29025 for matrix-sig-people; Wed, 15 May 1996 14:06:25 -0400 (EDT) Received: from simsg1.mdc.com (SIMSG1.MDC.COM [129.200.225.33]) by python.org (8.7.5/8.7.3) with SMTPid OAA28988 for ; Wed, 15 May 1996 14:06:12 -0400 (EDT) Received: by simsg1.mdc.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) id LAA22699; Wed, 15 May 1996 11:05:43 -0700 Date: Wed, 15 May 1996 11:05:42 -0700 (PDT) From: Brien Barton To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Where is source? Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk Where can I get the numeric extension for Python? I looked all over the python.org web site and can't find it. =============================================================================== Brien Barton McDonnell Douglas Aerospace, Huntington Beach, CA email: barton@simvx1.mdc.com, voice: (714)896-2249, fax:(714)896-5939 ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Wed May 15 14:27:18 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id OAA00460 for matrix-sig-people; Wed, 15 May 1996 14:23:01 -0400 (EDT) Received: from goldilocks.LCS.MIT.EDU (goldilocks.lcs.mit.edu [18.27.0.167]) by python.org (8.7.5/8.7.3) with SMTPid OAA00453 for ; Wed, 15 May 1996 14:22:51 -0400 (EDT) Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU by goldilocks.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA07783; Wed, 15 May 96 14:22:50 EDT Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA16460; Wed, 15 May 96 14:22:46 EDT Date: Wed, 15 May 96 14:22:46 EDT From: jjh@goldilocks.lcs.mit.edu (James Hugunin) Message-Id: <9605151822.AA16460@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: barton@simsg1.mdc.com Cc: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Reposting of the FAQ Sender: owner-matrix-sig@python.org Precedence: bulk This FAQ needs to be updated, and should be put on the web somewhere so that I can just put a pointer here occasionally. But, until the semester is over here that's not going to happen, so here's my brief FAQ again for newbies to the list. (To the rest of you I apologize). -Jim Written by Jim Hugunin (hugunin@mit.edu) on April 25, 1996. 1) What's Numerical Python? Here should really go Paul's humorous depiction of Monty's more serious brother, but I don't have that right now, so the high level stuff will have to wait. 2) Where do I get it? ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericalPython-0.36.tar.gz This is the latest version. New versions will be made available at this location. 3) Is there any documentation? There's incomplete online documentation written by David Ascher for a course he taught using Numerical Python at: http://maigret.cog.brown.edu/python/arraytut.html There's a paper soon to be published in Computers and Physics written by Dubois, Hinsen and Hugunin at: ftp://ftp-icf.llnl.gov/pub/basis/numerical_python.ps There's my talk from the 3rd python workshop at: http://www.python.org/workshops/1995-12/papers/hugunin.html 4) What modules are available that use Numerical python? (Note: I know that this needs pointers, I just don't have the time to put them together today) Tom Schwaller's delaunaymodule and trimodule Tom's and my opengl, glu, and glut modules Paul DuBois' URNGmodule Doug Heisterkamp's interface to the LAPACK libraries at: ftp://ftp.cs.unl.edu/pub/drh/python/pylapack.0.02.tar.gz 5) What are the future plans? First goal is a general release to the python community. Before this happens, I want a somewhat finalized API (which Konrad seems to have provided) implemented (for which I just need some free time). I don't want to have too many users developing code with the system before the API is at least closer to its final form (to minimize the changes they need to make). On the other hand, the C API is essentially final. I'm perfectly willing to guarantee that I won't introduce any major incompatibilities in subsequent release of the system. This means that people have no excuse not to develop modules to interface to all that great C/FORTRAN numerical code out there. Obviously the ultimate goal is to have this a part of the base python distribution. I'll start thinking more about this once I make the general release to the community at large. 6) Hints for Windows Users You must use binary mode files for pickling and unpickling matrices in the windows world. Blame Bill for the silliness. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Wed May 15 15:06:44 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id OAA02607 for matrix-sig-people; Wed, 15 May 1996 14:51:36 -0400 (EDT) Received: from CNRI.Reston.Va.US (glyph.CNRI.Reston.Va.US [132.151.1.13]) by python.org (8.7.5/8.7.3) with ESMTPid OAA02593 for ; Wed, 15 May 1996 14:51:23 -0400 (EDT) Received: (from klm@localhost) by CNRI.Reston.Va.US (8.7.5/8.7.3)id OAA28310; Wed, 15 May 1996 14:51:03 -0400 (EDT) Date: Wed, 15 May 1996 14:51:02 -0400 (EDT) From: Ken Manheimer Reply-To: klm@CNRI.Reston.Va.US To: James Hugunin cc: barton@simsg1.mdc.com, matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Reposting of the FAQ In-Reply-To: <9605151822.AA16460@baribal.LCS.MIT.EDU.LCS.MIT.EDU> Message-ID: X-Organization: Corporation for National Research Initiatives MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk I've added the FAQ and a blurb about it to the matrix-sig index page: Jim - how about creating a symlink to the latest version of the distribution, so the links in the FAQ, etc, can refer to it, and not need to be changed when the version changes? Eg: ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericalPython.tar.gz rather than ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericalPython-0.36.tar.gz (Soon SIG managers will be able to upload and arrange their python.org SIG dirs via anonymous ftp, incidentally, making this kind of info dissemination much more fluid, i hope.) Ken Manheimer klm@cnri.reston.va.us 703 620-8990 x268 (orporation for National Research |nitiatives 1895 Preston White Drive, Suite 100 Reston, VA 22091 ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Thu May 16 21:23:12 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id VAA19970 for matrix-sig-people; Thu, 16 May 1996 21:14:44 -0400 (EDT) Received: from flamingo.Stanford.EDU (flamingo.Stanford.EDU [171.64.68.20]) by python.org (8.7.5/8.7.3) with ESMTPid VAA19963 for ; Thu, 16 May 1996 21:14:29 -0400 (EDT) Received: from scottie.Stanford.EDU (scottie.Stanford.EDU [171.64.68.11]) by flamingo.Stanford.EDU (8.7.1/8.7.1) with ESMTP id SAA15380 for ; Thu, 16 May 1996 18:14:26 -0700 Received: (from marko@localhost) by scottie.Stanford.EDU (8.7.1/8.6.9) id SAA26194 for matrix-sig@python.org; Thu, 16 May 1996 18:14:24 -0700 Date: Thu, 16 May 96 18:14:21 PDT From: Marko Balabanovic Reply-To: Marko Balabanovic To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Compact Representation? Message-ID: Sender: owner-matrix-sig@python.org Precedence: bulk I'm kind of an amateur at linear algebra, so this may be a stupid question: Is there already, or are there any plans to include, a compact representation for sparse or symmetric matrices in Numeric Python (possibly with interfaces to the LAPACK/CLAPACK routines which work with a packed representation for symmetric matrices). I have an approx. 10,000 x 10,000 symmetric matrix, all values are 1 or 0, and I need to get the first couple of eigenvectors. If anyone has any pointers to code that would help do this without needing to construct the whole thing in memory at once I would be very grateful. Marko .Marko Balabanovic.......Department of Computer Science.................... .Stanford University Email: marko@cs.stanford.edu Office: Gates 132 . .Gates Building 1A Phone: 415 725 8783 Fax: 415 725 1449 . .Stanford CA 94305-9010 Url: http://robotics.stanford.edu/people/marko . .USA....................................................................... ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Fri May 17 10:02:26 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id JAA21333 for matrix-sig-people; Fri, 17 May 1996 09:50:15 -0400 (EDT) Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.7.5/8.7.3) with ESMTPid JAA21317 for ; Fri, 17 May 1996 09:49:51 -0400 (EDT) Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id JAA28753 (8.6.11/IDA-1.6); Fri, 17 May 1996 09:48:01 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id JAA20102; Fri, 17 May 1996 09:48:00 -0400 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id JAA16768; Fri, 17 May 1996 09:47:59 -0400 Date: Fri, 17 May 1996 09:47:59 -0400 Message-Id: <199605171347.JAA16768@cyclone.ERE.UMontreal.CA> To: marko@cs.stanford.edu CC: matrix-sig@python.org In-reply-to: (message from Marko Balabanovic on Thu, 16 May 96 18:14:21 PDT) Subject: Re: [PYTHON MATRIX-SIG] Compact Representation? From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > Is there already, or are there any plans to include, a compact > representation for sparse or symmetric matrices in Numeric Python > (possibly with interfaces to the LAPACK/CLAPACK routines which work > with a packed representation for symmetric matrices). Support for symmetric packed matrices is on my to-do-list for the high-level LAPACK interface, but I won't make any promises about when this is going to happen. I am not even quite sure about how to handle such matrices. I certainly don't want to take the Fortran-like approach of considering packed matrices like 1D-arrays and leaving indexing etc. to the user. So there will be a class representing symmetric matrices, and the LAPACK-interface will select the suitable LAPACK-routine. But how this will work in detail also depends on what mechanisms will be available in the final array package for defining array-like Python classes and making array functions available on them. > I have an approx. 10,000 x 10,000 symmetric matrix, all values are 1 > or 0, and I need to get the first couple of eigenvectors. If anyone > has any pointers to code that would help do this without needing to > construct the whole thing in memory at once I would be very grateful. In that case LAPACK won't help you. You should look at the iterative eigenvalue algorithms that only require a routine to to matrix-vector multiplication. In writing this routine you could even take advantage of having only two different elements in the matrix. But this will end up in very specialized code, probably best done in a small C extension module interfacing to the array module. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Tue May 21 19:25:37 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id TAA02879 for matrix-sig-people; Tue, 21 May 1996 19:19:35 -0400 (EDT) Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.7.5/8.7.3) with SMTPid TAA02872 for ; Tue, 21 May 1996 19:19:26 -0400 (EDT) Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA12110; Tue, 21 May 96 18:14:23 -0400 Received: by maigret (950215.SGI.8.6.10/940406.SGI) for matrix-sig@python.org id SAA14621; Tue, 21 May 1996 18:15:20 -0400 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199605212215.SAA14621@maigret> Subject: [PYTHON MATRIX-SIG] bug, I claim To: matrix-sig@python.org Date: Tue, 21 May 1996 18:15:19 -0400 (EDT) X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk That the following happens is, I believe, unfortunate: from Numeric import * from Ranf import * a = random_sample(10) b = (a.greater(0.5).choose(0,0.1)) print b 0 0 0 0 0 0 0 0 0 0 c = (a.greater(0.5).choose(0.0,0.1)) print c 0.10000000 0.10000000 0.10000000 0.00000000 0.00000000 0.00000000 0.00000000 0.00000000 0.10000000 0.00000000 Why should the type of the first argument to choose() be valued over the most generic type? ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Wed May 22 10:55:35 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id KAA04681 for matrix-sig-people; Wed, 22 May 1996 10:43:58 -0400 (EDT) Received: from lems.brown.edu (lems.brown.edu [128.148.128.4]) by python.org (8.7.5/8.7.3) with SMTPid KAA04674 for ; Wed, 22 May 1996 10:43:52 -0400 (EDT) Received: from lems33.lems.brown.edu by lems.brown.edu (5.x/SMI-SVR4) id AA04471; Wed, 22 May 1996 10:43:44 -0400 Received: by lems33.lems.brown.edu (5.x/SMI-SVR4) id AA28291; Wed, 22 May 1996 10:43:43 -0400 Date: Wed, 22 May 1996 10:43:42 -0400 (EDT) From: "Perry A. Stoll" X-Sender: pas@lems33 To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] bug and fix for Numeric.where Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk While were looking at bugs... I finally understand the beauty of the Numeric.where function (and the "choose" function, but that's a separate story). But I think the function in Numeric is wrong. Here's a replacement. The function is pretty trivial, but I like the way it reads in code and vote for keeping it. def where(condition, x, y): """where(condition,x,y) is shaped like condition and has elements of x and y where condition is respectively true or false """ return condition.choose(y, x) # removed condition as first argument As a test demonstrating the problem and fix: Select elements from either a or b, which ever is greater - exactly what the "where" function is for. >>> import Numeric >>> a = Numeric.arange(10) >>> b = Numeric.zeros(10,'l') + 4 >>> Numeric.where(a.greater(b),a,b) 0 0 0 0 0 4 4 4 4 4 # huh?? >>> where(a.greater(b),a,b) 4 4 4 4 4 5 6 7 8 9 # ahhh, that's better. -Perry ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Wed May 22 11:09:47 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id LAA04753 for matrix-sig-people; Wed, 22 May 1996 11:05:44 -0400 (EDT) Received: from goldilocks.LCS.MIT.EDU (goldilocks.lcs.mit.edu [18.27.0.167]) by python.org (8.7.5/8.7.3) with SMTPid LAA04746 for ; Wed, 22 May 1996 11:05:40 -0400 (EDT) Received: from baribal.LCS.MIT.EDU.LCS.MIT.EDU by goldilocks.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA05518; Wed, 22 May 96 11:05:35 EDT Received: by baribal.LCS.MIT.EDU.LCS.MIT.EDU (4.1/SMI-4.1) id AA00341; Wed, 22 May 96 11:05:34 EDT Date: Wed, 22 May 96 11:05:34 EDT From: jjh@goldilocks.lcs.mit.edu (James Hugunin) Message-Id: <9605221505.AA00341@baribal.LCS.MIT.EDU.LCS.MIT.EDU> To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] bugs Sender: owner-matrix-sig@python.org Precedence: bulk Perry Stoll suggests a small bug in the where function, and he's of course correct. The change has been made. def where(condition, x, y): """where(condition,x,y) is shaped like condition and has elements of x and y where condition is respectively true or false """ return condition.choose(y, x) # removed condition as first argument This and similar functions will be retained in the great renaming that is beginning even as we speak. They might, however, be moved to a shortcuts module that wouldn't always be automatically loaded. -- David Ascher's bug is a little more troublesome, but it too is being fixed. > Why should the type of the first argument to choose() be valued over the > most generic type? In short, it shouldn't, and it won't in the next release. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Thu May 23 05:46:48 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id FAA07984 for matrix-sig-people; Thu, 23 May 1996 05:29:44 -0400 (EDT) Received: from nz11.rz.uni-karlsruhe.de (nz11.rz.uni-karlsruhe.de [129.13.64.7]) by python.org (8.7.5/8.7.3) with ESMTPid FAA07977 for ; Thu, 23 May 1996 05:29:40 -0400 (EDT) Received: from ipc2ibm1.chemie.uni-karlsruhe.de by nz11.rz.uni-karlsruhe.de with SMTP (PP); Thu, 23 May 1996 10:18:50 +0200 Received: by ipc2ibm1.chemie.uni-karlsruhe.de (AIX 3.2/UCB 5.64/4.03) id AA26109; Thu, 23 May 1996 10:18:39 +0200 From: martin@ipc2ibm1.chemie.uni-karlsruhe.de (MARTIN GEGENHEIMER AK kappes) Message-Id: <9605230818.AA26109@ipc2ibm1.chemie.uni-karlsruhe.de> Subject: [PYTHON MATRIX-SIG] [Q] Array->String conversion, how? To: matrix-sig@python.org Date: Thu, 23 May 1996 10:18:39 +0200 (MSZ) X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk Hello, I think I need som help: I'd like to convert a numeric array to a string like (to pass it to Tcl) { 1.0 2.0 .... } I tried : str(), repr(), but that gives me : "array([0,1,2,3,4,5,6,7,8,9], 'l')" a.toList() is much more what I want (its also fast for larger arrays) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] but I don't want ',' or braces. a.toString() does no do what I want. so, how ? maybe make a modified toList() ? -- Martin Gegenheimer Institut fuer Physikalische Chemie II Kaiserstr.12 76128 Karlsruhe Germany home-page: http://www.chemie.uni-karlsruhe.de/~martin e-mail: martin@ipc2ibm1.chemie.uni-karlsruhe.de voice: ++49 721 608-3255 fax : ++49 721 608-3310 ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Thu May 23 10:16:42 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id HAA08316 for matrix-sig-people; Thu, 23 May 1996 07:55:33 -0400 (EDT) Received: from nz11.rz.uni-karlsruhe.de (nz11.rz.uni-karlsruhe.de [129.13.64.7]) by python.org (8.7.5/8.7.3) with ESMTPid HAA08309 for ; Thu, 23 May 1996 07:55:10 -0400 (EDT) Received: from ipc2ibm1.chemie.uni-karlsruhe.de by nz11.rz.uni-karlsruhe.de with SMTP (PP); Thu, 23 May 1996 13:52:02 +0200 Received: by ipc2ibm1.chemie.uni-karlsruhe.de (AIX 3.2/UCB 5.64/4.03) id AA05001; Thu, 23 May 1996 13:51:45 +0200 From: martin@ipc2ibm1.chemie.uni-karlsruhe.de (MARTIN GEGENHEIMER AK kappes) Message-Id: <9605231151.AA05001@ipc2ibm1.chemie.uni-karlsruhe.de> Subject: [PYTHON MATRIX-SIG] [Q] Array->String conversion, how? To: matrix-sig@python.org Date: Thu, 23 May 1996 13:51:45 +0200 (MSZ) X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk Forwarded message: > From root Thu May 23 13:42:41 1996 > Date: Thu, 23 May 96 07:37:58 EDT > Message-Id: <9605231137.AA02767@acdc.eeel.nist.gov> > From: Michael McLay > To: martin@tchibm3.chemie.uni-karlsruhe.de (MARTIN GEGENHEIMER AK kappes) > Subject: [PYTHON MATRIX-SIG] [Q] Array->String conversion, how? > In-Reply-To: <9605230818.AA26109@ipc2ibm1.chemie.uni-karlsruhe.de> > References: <9605230818.AA26109@ipc2ibm1.chemie.uni-karlsruhe.de> > > MARTIN GEGENHEIMER AK writes: > > Hello, > > > > I think I need som help: > > > > I'd like to convert a numeric array to a string like > > (to pass it to Tcl) > > > > { 1.0 2.0 .... } > > > > I tried : > > > > str(), repr(), but that gives me : > > "array([0,1,2,3,4,5,6,7,8,9], 'l')" > > > > a.toList() is much more what I want (its also fast for larger arrays) > > > > [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] > > but I don't want ',' or braces. > > > > a.toString() > > does no do what I want. > > > > > > so, how ? > > > > maybe make a modified toList() ? > > > How about using a.toList() to produce > [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] > > and then do a gsub to replace ", " with " ". > I wanted to do that when everything else fails. :-( - Martin -- Martin Gegenheimer Institut fuer Physikalische Chemie II Kaiserstr.12 76128 Karlsruhe Germany home-page: http://www.chemie.uni-karlsruhe.de/~martin e-mail: martin@ipc2ibm1.chemie.uni-karlsruhe.de voice: ++49 721 608-3255 fax : ++49 721 608-3310 ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Thu May 23 10:26:34 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id KAA08541 for matrix-sig-people; Thu, 23 May 1996 10:16:49 -0400 (EDT) Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.7.5/8.7.3) with SMTPid JAA08401 for ; Thu, 23 May 1996 09:18:05 -0400 (EDT) Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA16052; Thu, 23 May 96 09:11:58 -0400 Received: by maigret (950215.SGI.8.6.10/940406.SGI) id JAA18508; Thu, 23 May 1996 09:12:53 -0400 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199605231312.JAA18508@maigret> Subject: Re: [PYTHON MATRIX-SIG] [Q] Array->String conversion, how? To: martin@ipc2ibm1.chemie.uni-karlsruhe.de (MARTIN GEGENHEIMER AK kappes) Date: Thu, 23 May 1996 09:12:52 -0400 (EDT) Cc: matrix-sig@python.org In-Reply-To: <9605230818.AA26109@ipc2ibm1.chemie.uni-karlsruhe.de> from "MARTIN GEGENHEIMER AK kappes" at May 23, 96 10:18:39 am X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk Well, this might be a place to start: a = arange(10) string.join(map(str, a)) '0 1 2 3 4 5 6 7 8 9' a = arange(0.0,10.0) string.join(map(str, a)) '0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0' Speaking of which, isn't it odd that: >>> arange(10) 0 1 2 3 4 5 6 7 8 9 >>> arange(10.0) 0 1 2 3 4 5 6 7 8 9 >>> arange(0.0,10.0) 0.00000 1.00000 2.00000 3.00000 4.00000 5.00000 6.00000 7.00000 8.00000 9.00000 ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Thu May 23 11:09:19 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id KAA08677 for matrix-sig-people; Thu, 23 May 1996 10:58:34 -0400 (EDT) Received: from CNRI.Reston.VA.US (CNRI.Reston.Va.US [132.151.1.1]) by python.org (8.7.5/8.7.3) with SMTPid KAA08669 for ; Thu, 23 May 1996 10:58:14 -0400 (EDT) Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa09081; 23 May 96 10:54 EDT Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id KAA25393; Thu, 23 May 1996 10:53:54 -0400 Message-Id: <199605231453.KAA25393@monty> To: MARTIN GEGENHEIMER AK kappes cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] [Q] Array->String conversion, how? In-reply-to: Your message of "Thu, 23 May 1996 13:51:45 +0200." <9605231151.AA05001@ipc2ibm1.chemie.uni-karlsruhe.de> References: <9605231151.AA05001@ipc2ibm1.chemie.uni-karlsruhe.de> From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 23 May 1996 10:53:53 -0400 Sender: owner-matrix-sig@python.org Precedence: bulk > > How about using a.toList() to produce > > [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] > > > > and then do a gsub to replace ", " with " ". Or this: "{%s}" % string.join(map(str, a)) --Guido van Rossum (home page: http://www.python.org/~guido/) ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Fri May 24 12:20:15 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id MAA12234 for matrix-sig-people; Fri, 24 May 1996 12:10:34 -0400 (EDT) Received: from lems.brown.edu (lems.brown.edu [128.148.128.4]) by python.org (8.7.5/8.7.3) with SMTPid MAA12227 for ; Fri, 24 May 1996 12:10:26 -0400 (EDT) Received: from lems61.lems.brown.edu by lems.brown.edu (5.x/SMI-SVR4) id AA08260; Fri, 24 May 1996 12:10:16 -0400 Received: by lems61.lems.brown.edu (SMI-8.6/SMI-SVR4) id MAA05351; Fri, 24 May 1996 12:10:15 -0400 Date: Fri, 24 May 1996 12:10:14 -0400 (EDT) From: "Perry A. Stoll" X-Sender: pas@lems61 To: David Ascher Cc: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] having problems (in arange)? In-Reply-To: <199605231312.JAA18508@maigret> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk Did everyone remember to remove the old .pyc files for Numeric? I did that only when I couldn't find 'index_expression' when I imported Numeric and all sorts of problems went away. I get the following: >>> arange(10) 0 1 2 3 4 5 6 7 8 9 >>> arange(10.0) 0. 1. 2. 3. 4. 5. 6. 7. 8. 9. -Perry On Thu, 23 May 1996, David Ascher wrote: > Speaking of which, isn't it odd that: > > >>> arange(10) > 0 1 2 3 4 5 6 7 8 9 > >>> arange(10.0) > 0 1 2 3 4 5 6 7 8 9 > >>> arange(0.0,10.0) > 0.00000 1.00000 2.00000 3.00000 4.00000 5.00000 6.00000 7.00000 8.00000 > ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Fri May 24 17:16:23 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id RAA12960 for matrix-sig-people; Fri, 24 May 1996 17:00:55 -0400 (EDT) Received: from CNRI.Reston.VA.US (CNRI.Reston.Va.US [132.151.1.1]) by python.org (8.7.5/8.7.3) with SMTPid RAA12936 for ; Fri, 24 May 1996 17:00:10 -0400 (EDT) Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa13931; 24 May 96 16:59 EDT Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) for id QAA07559; Fri, 24 May 1996 16:59:13 -0400 Message-Id: <199605242059.QAA07559@monty> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] what to do with public symbols c_* From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Fri, 24 May 1996 16:59:13 -0400 Sender: owner-matrix-sig@python.org Precedence: bulk I'm trying to make Python 1.4 "clean" with respect to exporting symbols: all exported symbols except for init will have a Py or _Py prefix. There are a bunch of symbols defined in complexobject.c and cmathmodule.c whose name begin with "c_", for instance c_sum() and c_sin(). If I rename the ones defined in complexoject.h (c_sum, c_diff, c_neg, c_prod, c_quot and c_pow) and make the ones in cmathmodule.c static, would this break any code in the matrix extensions? --Guido van Rossum (home page: http://www.python.org/~guido/) ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Fri May 24 17:57:23 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id RAA13041 for matrix-sig-people; Fri, 24 May 1996 17:45:42 -0400 (EDT) Received: from goldilocks.LCS.MIT.EDU (goldilocks.lcs.mit.edu [18.27.0.167]) by python.org (8.7.5/8.7.3) with SMTPid RAA13034 for ; Fri, 24 May 1996 17:45:21 -0400 (EDT) Received: from ling-ling.zoo (ling-ling.lcs.mit.edu) by goldilocks.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA27556; Fri, 24 May 96 17:45:14 EDT Received: by ling-ling.zoo (SMI-8.6/SMI-SVR4) id RAA00282; Fri, 24 May 1996 17:45:20 -0400 Date: Fri, 24 May 1996 17:45:20 -0400 Message-Id: <199605242145.RAA00282@ling-ling.zoo> From: James Hugunin To: guido@CNRI.Reston.Va.US Cc: matrix-sig@python.org In-Reply-To: <199605242059.QAA07559@monty> (message from Guido van Rossum on Fri, 24 May 1996 16:59:13 -0400) Subject: Re: [PYTHON MATRIX-SIG] what to do with public symbols c_* Sender: owner-matrix-sig@python.org Precedence: bulk From: Guido van Rossum I'm trying to make Python 1.4 "clean" with respect to exporting symbols: all exported symbols except for init will have a Py or _Py prefix. There are a bunch of symbols defined in complexobject.c and cmathmodule.c whose name begin with "c_", for instance c_sum() and c_sin(). If I rename the ones defined in complexoject.h (c_sum, c_diff, c_neg, c_prod, c_quot and c_pow) and make the ones in cmathmodule.c static, would this break any code in the matrix extensions? --Guido van Rossum (home page: http://www.python.org/~guido/) This won't cause any problems with the matrix extension. These functions are defined statically in the array math code (twice actually, but that's too embarrassing to explain here). This is a hold over from when it wasn't guaranteed that complex numbers would be part of the python core. After the release of 1.4, I'll probably remove the static definitions and use the Py prefixed versions instead. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Fri May 24 18:00:22 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id RAA13041 for matrix-sig-people; Fri, 24 May 1996 17:45:42 -0400 (EDT) Received: from goldilocks.LCS.MIT.EDU (goldilocks.lcs.mit.edu [18.27.0.167]) by python.org (8.7.5/8.7.3) with SMTPid RAA13034 for ; Fri, 24 May 1996 17:45:21 -0400 (EDT) Received: from ling-ling.zoo (ling-ling.lcs.mit.edu) by goldilocks.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA27556; Fri, 24 May 96 17:45:14 EDT Received: by ling-ling.zoo (SMI-8.6/SMI-SVR4) id RAA00282; Fri, 24 May 1996 17:45:20 -0400 Date: Fri, 24 May 1996 17:45:20 -0400 Message-Id: <199605242145.RAA00282@ling-ling.zoo> From: James Hugunin To: guido@CNRI.Reston.Va.US Cc: matrix-sig@python.org In-Reply-To: <199605242059.QAA07559@monty> (message from Guido van Rossum on Fri, 24 May 1996 16:59:13 -0400) Subject: Re: [PYTHON MATRIX-SIG] what to do with public symbols c_* Sender: owner-matrix-sig@python.org Precedence: bulk From: Guido van Rossum I'm trying to make Python 1.4 "clean" with respect to exporting symbols: all exported symbols except for init will have a Py or _Py prefix. There are a bunch of symbols defined in complexobject.c and cmathmodule.c whose name begin with "c_", for instance c_sum() and c_sin(). If I rename the ones defined in complexoject.h (c_sum, c_diff, c_neg, c_prod, c_quot and c_pow) and make the ones in cmathmodule.c static, would this break any code in the matrix extensions? --Guido van Rossum (home page: http://www.python.org/~guido/) This won't cause any problems with the matrix extension. These functions are defined statically in the array math code (twice actually, but that's too embarrassing to explain here). This is a hold over from when it wasn't guaranteed that complex numbers would be part of the python core. After the release of 1.4, I'll probably remove the static definitions and use the Py prefixed versions instead. -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Mon May 27 20:06:25 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id TAA20307 for matrix-sig-people; Mon, 27 May 1996 19:55:14 -0400 (EDT) Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.7.5/8.7.3) with ESMTPid TAA20298 for ; Mon, 27 May 1996 19:53:19 -0400 (EDT) Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id TAA03233 (8.6.11/IDA-1.6); Mon, 27 May 1996 19:51:38 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id TAA27880; Mon, 27 May 1996 19:51:38 -0400 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id TAA22272; Mon, 27 May 1996 19:51:37 -0400 Date: Mon, 27 May 1996 19:51:37 -0400 Message-Id: <199605272351.TAA22272@cyclone.ERE.UMontreal.CA> To: da@maigret.cog.brown.edu CC: matrix-sig@python.org In-reply-to: <199605231312.JAA18508@maigret> (da@maigret.cog.brown.edu) Subject: Re: [PYTHON MATRIX-SIG] [Q] Array->String conversion, how? From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > Speaking of which, isn't it odd that: > > >>> arange(10) > 0 1 2 3 4 5 6 7 8 9 > >>> arange(10.0) > 0 1 2 3 4 5 6 7 8 9 > >>> arange(0.0,10.0) > 0.00000 1.00000 2.00000 3.00000 4.00000 5.00000 6.00000 7.00000 8.00000 9.00000 On my machine the last two give the same answer, as expected. If they don't for you, I suspect rounding problems. Check if arange(0.0,10.0)-arange(10.0) really gives zeroes. If so, my printing code behaves funnily on your machine. If not, arange() behaves funnily ;-) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Mon May 27 20:06:34 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id TAA20314 for matrix-sig-people; Mon, 27 May 1996 19:58:05 -0400 (EDT) Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.7.5/8.7.3) with ESMTPid TAA20298 for ; Mon, 27 May 1996 19:53:19 -0400 (EDT) Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id TAA03233 (8.6.11/IDA-1.6); Mon, 27 May 1996 19:51:38 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id TAA27880; Mon, 27 May 1996 19:51:38 -0400 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id TAA22272; Mon, 27 May 1996 19:51:37 -0400 Date: Mon, 27 May 1996 19:51:37 -0400 Message-Id: <199605272351.TAA22272@cyclone.ERE.UMontreal.CA> To: da@maigret.cog.brown.edu CC: matrix-sig@python.org In-reply-to: <199605231312.JAA18508@maigret> (da@maigret.cog.brown.edu) Subject: Re: [PYTHON MATRIX-SIG] [Q] Array->String conversion, how? From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > Speaking of which, isn't it odd that: > > >>> arange(10) > 0 1 2 3 4 5 6 7 8 9 > >>> arange(10.0) > 0 1 2 3 4 5 6 7 8 9 > >>> arange(0.0,10.0) > 0.00000 1.00000 2.00000 3.00000 4.00000 5.00000 6.00000 7.00000 8.00000 9.00000 On my machine the last two give the same answer, as expected. If they don't for you, I suspect rounding problems. Check if arange(0.0,10.0)-arange(10.0) really gives zeroes. If so, my printing code behaves funnily on your machine. If not, arange() behaves funnily ;-) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Tue May 28 12:56:25 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id MAA22562 for matrix-sig-people; Tue, 28 May 1996 12:46:58 -0400 (EDT) Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.7.5/8.7.3) with SMTPid MAA22555 for ; Tue, 28 May 1996 12:46:51 -0400 (EDT) Received: by icf.llnl.gov (4.1/LLNL-1.19) id AA29432; Tue, 28 May 96 09:46:40 PDT Date: Tue, 28 May 96 09:46:40 PDT From: busby@icf.llnl.gov (L. Busby) Message-Id: <9605281646.AA29432@icf.llnl.gov> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] ANNOUNCE: Gist Scientific graphics module for Python Sender: owner-matrix-sig@python.org Precedence: bulk I'm releasing the first version of my Gist module for Python. You can pick up a copy at ftp-icf.llnl.gov:/pub/python/busby/pygist-1.0.tgz This module is dependent on the Numeric module due to Hugunin and others. So until that module reaches public release, the Gist module should also stay within the matrix-sig. Following is the README.gist file from the release: ============================================================================== This is the README file for the Python Gist Scientific Graphics Module, version 1.0, written by Lee Busby of Lawrence Livermore National Laboratory. Copyright (c) 1996. The Regents of the University of California. All rights reserved. ============================================================================== DESCRIPTION "Gist" is a scientific graphics library written by David H. Munro of Lawrence Livermore National Laboratory. It features support for three common graphics output devices: X-Windows, (Color) PostScript, and ANSI/ISO Standard Computer Graphics Metafiles (CGM). The library is small (written directly to Xlib), portable, efficient, and full-featured. It produces x-vs-y plots with "good" tick marks and tick labels, 2-D quadrilateral mesh plots with contours, vector fields, or pseudocolor maps on such meshes, with 3-D plots on the way. The Python Gist module utilizes the new ``Numeric'' module due to J. Hugunin and others. It is therefore fast and able to handle large datasets. The Gist module includes an X-windows event dispatcher which can be dynamically added (e.g., via importing a dynamically loaded module) to the Python interpreter after a simple two-line modification to the Python core. This makes fast mouse-controlled zoom, pan, and other graphic operations available to the researcher while maintaining the usual Python command-line interface. ============================================================================== AVAILABILITY ftp-icf.llnl.gov:/pub/python/busby/pygist-1.0.tgz ============================================================================== CONTENTS OF THE PACKAGE ./LEGAL.gist : Copyright and Disclaimer ./Lib/numeric/gist.help : Major documentation for gist module ./Lib/numeric/gist.py : Python code for gist module ./Lib/numeric/gistdemo.py : Demonstration program ./Lib/numeric/help.help : Documentation for help module ./Lib/numeric/help.py : The help module ./Modules/Setup.forgist : Typical compilation lines for gist ./Modules/gistCmodule.c : C code for gist module ./Parser/myreadline.c : Replacement for python file ./README.gist : This file ============================================================================== INSTALLATION Installation of the Python gist module is complicated by its dependence on another not-yet standard Python module, the "Numeric" module by Hugunin et al, and the Gist graphics library itself. The Gist module has been tested only with Python 1.3. The current version of the Numeric module as of 24May96 can be obtained as ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericalPython-0.36.tar.gz It includes instructions sufficient for its installation. The Gist library is included with the distribution of David Munro's Yorick interpreter. The current version is 1.2. Yorick can be obtained at ftp-icf.llnl.gov:/pub/Yorick/yorick-1.2.tar.gz wuarchive.wustl.edu: /languages/yorick/yorick-1.2.tar.gz sunsite.unc.edu: /pub/languages/yorick/yorick-1.2.tar.gz sunsite.unc.edu: /pub/Linux/apps/math/matrix/yorick-1.2.tar.gz netlib.att.com: /netlib/env/yorick-1.2.tar.gz netlib2.cs.utk.edu: /env/yorick-1.2.tar.gz Yorick also includes ample instructions for its installation. It is easiest to simply install the entire Yorick distribution. This requires only about 5MB of disk space. Building Yorick requires 10-15MB of additional temporary space. If you are interested in Python Gist, chances are you may also be interested in Yorick itself. However, should you desire to remove the files not strictly necessary for compilation and installation of Python Gist, here is a list of the *required* files and directories: (${prefix} defaults to /usr/local.) ${prefix}/Yorick/gist # Style and color palette files ${prefix}/bin/gist # Standalone CGM browser program ${prefix}/lib/libgist.a # The Gist library ${prefix}/yorhome/{dispas.h, dispat.h, gist.h, hlevel.h} # header files You may also want to save or print out various documentation files included with Yorick. After you have installed Yorick and unpacked the Numeric module into your toplevel Python working directory, you are ready to install Gist: CONCISE INSTRUCTIONS 1) Install Yorick. 2) Untar the Numeric module into your toplevel Python working directory, and follow its instructions up to the point where you would begin compilation of python. (Untar the additional patches and make necessary additions and modifications to Modules/Setup.) 3) Copy pygist-1.0.tgz to the top of your python distribution. 4) cd Python-1.3; zcat pygist-1.0.tgz | tar xf - This adds files at the top level, and in subdirectories Lib and Modules. It overwrites the file Parser/myreadline.c with a slightly modified version. Save a backup copy if you want to compare the change. 5) Modify Modules/Setup by adding appropriate lines from Modules/Setup.forgist. Setup.forgist assumes that you installed Yorick in its default location. Change YPREFIX as necessary if your Yorick is installed somewhere else. The gist module can be dynamically loaded on most platforms, if you prefer. 6) Configure and compile Python. ============================================================================= RUNNING GIST After you have successfully compiled Python with Gist, you can test it by running >>> import gistdemo >>> gistdemo.run() and you can get started with the online help using >>> from gist import * >>> help("help.") >>> help("gist.") ============================================================================= AUTHOR'S ADDRESS Lee Busby, mailstop L-472 Lawrence Livermore National Laboratory 7000 East Avenue Livermore, CA, USA 94550 E-mail: busby1@llnl.gov ============================================================================= ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Tue May 28 12:58:11 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id MAA22562 for matrix-sig-people; Tue, 28 May 1996 12:46:58 -0400 (EDT) Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.7.5/8.7.3) with SMTPid MAA22555 for ; Tue, 28 May 1996 12:46:51 -0400 (EDT) Received: by icf.llnl.gov (4.1/LLNL-1.19) id AA29432; Tue, 28 May 96 09:46:40 PDT Date: Tue, 28 May 96 09:46:40 PDT From: busby@icf.llnl.gov (L. Busby) Message-Id: <9605281646.AA29432@icf.llnl.gov> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] ANNOUNCE: Gist Scientific graphics module for Python Sender: owner-matrix-sig@python.org Precedence: bulk I'm releasing the first version of my Gist module for Python. You can pick up a copy at ftp-icf.llnl.gov:/pub/python/busby/pygist-1.0.tgz This module is dependent on the Numeric module due to Hugunin and others. So until that module reaches public release, the Gist module should also stay within the matrix-sig. Following is the README.gist file from the release: ============================================================================== This is the README file for the Python Gist Scientific Graphics Module, version 1.0, written by Lee Busby of Lawrence Livermore National Laboratory. Copyright (c) 1996. The Regents of the University of California. All rights reserved. ============================================================================== DESCRIPTION "Gist" is a scientific graphics library written by David H. Munro of Lawrence Livermore National Laboratory. It features support for three common graphics output devices: X-Windows, (Color) PostScript, and ANSI/ISO Standard Computer Graphics Metafiles (CGM). The library is small (written directly to Xlib), portable, efficient, and full-featured. It produces x-vs-y plots with "good" tick marks and tick labels, 2-D quadrilateral mesh plots with contours, vector fields, or pseudocolor maps on such meshes, with 3-D plots on the way. The Python Gist module utilizes the new ``Numeric'' module due to J. Hugunin and others. It is therefore fast and able to handle large datasets. The Gist module includes an X-windows event dispatcher which can be dynamically added (e.g., via importing a dynamically loaded module) to the Python interpreter after a simple two-line modification to the Python core. This makes fast mouse-controlled zoom, pan, and other graphic operations available to the researcher while maintaining the usual Python command-line interface. ============================================================================== AVAILABILITY ftp-icf.llnl.gov:/pub/python/busby/pygist-1.0.tgz ============================================================================== CONTENTS OF THE PACKAGE ./LEGAL.gist : Copyright and Disclaimer ./Lib/numeric/gist.help : Major documentation for gist module ./Lib/numeric/gist.py : Python code for gist module ./Lib/numeric/gistdemo.py : Demonstration program ./Lib/numeric/help.help : Documentation for help module ./Lib/numeric/help.py : The help module ./Modules/Setup.forgist : Typical compilation lines for gist ./Modules/gistCmodule.c : C code for gist module ./Parser/myreadline.c : Replacement for python file ./README.gist : This file ============================================================================== INSTALLATION Installation of the Python gist module is complicated by its dependence on another not-yet standard Python module, the "Numeric" module by Hugunin et al, and the Gist graphics library itself. The Gist module has been tested only with Python 1.3. The current version of the Numeric module as of 24May96 can be obtained as ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericalPython-0.36.tar.gz It includes instructions sufficient for its installation. The Gist library is included with the distribution of David Munro's Yorick interpreter. The current version is 1.2. Yorick can be obtained at ftp-icf.llnl.gov:/pub/Yorick/yorick-1.2.tar.gz wuarchive.wustl.edu: /languages/yorick/yorick-1.2.tar.gz sunsite.unc.edu: /pub/languages/yorick/yorick-1.2.tar.gz sunsite.unc.edu: /pub/Linux/apps/math/matrix/yorick-1.2.tar.gz netlib.att.com: /netlib/env/yorick-1.2.tar.gz netlib2.cs.utk.edu: /env/yorick-1.2.tar.gz Yorick also includes ample instructions for its installation. It is easiest to simply install the entire Yorick distribution. This requires only about 5MB of disk space. Building Yorick requires 10-15MB of additional temporary space. If you are interested in Python Gist, chances are you may also be interested in Yorick itself. However, should you desire to remove the files not strictly necessary for compilation and installation of Python Gist, here is a list of the *required* files and directories: (${prefix} defaults to /usr/local.) ${prefix}/Yorick/gist # Style and color palette files ${prefix}/bin/gist # Standalone CGM browser program ${prefix}/lib/libgist.a # The Gist library ${prefix}/yorhome/{dispas.h, dispat.h, gist.h, hlevel.h} # header files You may also want to save or print out various documentation files included with Yorick. After you have installed Yorick and unpacked the Numeric module into your toplevel Python working directory, you are ready to install Gist: CONCISE INSTRUCTIONS 1) Install Yorick. 2) Untar the Numeric module into your toplevel Python working directory, and follow its instructions up to the point where you would begin compilation of python. (Untar the additional patches and make necessary additions and modifications to Modules/Setup.) 3) Copy pygist-1.0.tgz to the top of your python distribution. 4) cd Python-1.3; zcat pygist-1.0.tgz | tar xf - This adds files at the top level, and in subdirectories Lib and Modules. It overwrites the file Parser/myreadline.c with a slightly modified version. Save a backup copy if you want to compare the change. 5) Modify Modules/Setup by adding appropriate lines from Modules/Setup.forgist. Setup.forgist assumes that you installed Yorick in its default location. Change YPREFIX as necessary if your Yorick is installed somewhere else. The gist module can be dynamically loaded on most platforms, if you prefer. 6) Configure and compile Python. ============================================================================= RUNNING GIST After you have successfully compiled Python with Gist, you can test it by running >>> import gistdemo >>> gistdemo.run() and you can get started with the online help using >>> from gist import * >>> help("help.") >>> help("gist.") ============================================================================= AUTHOR'S ADDRESS Lee Busby, mailstop L-472 Lawrence Livermore National Laboratory 7000 East Avenue Livermore, CA, USA 94550 E-mail: busby1@llnl.gov ============================================================================= ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Wed May 29 13:46:34 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id NAA26475 for matrix-sig-people; Wed, 29 May 1996 13:41:01 -0400 (EDT) Received: from ibp.ibp.fr (ibp.ibp.fr [132.227.60.30]) by python.org (8.7.5/8.7.3) with ESMTPid NAA26465 for ; Wed, 29 May 1996 13:40:48 -0400 (EDT) Received: from laforia.ibp.fr (laforia.ibp.fr [132.227.60.10]) by ibp.ibp.fr (8.6.12/jtpda-5.0) with ESMTP id TAA03200 ; Wed, 29 May 1996 19:40:38 +0200 Received: from lpia2.ibp.fr (singha.ibp.fr [132.227.201.18]) by laforia.ibp.fr (8.6.10/jtpda-5.0) with ESMTP id TAA01220 ; Wed, 29 May 1996 19:40:37 +0200 From: Emmanuel.Viennet@laforia.ibp.fr (VIENNET Emmanuel 48.06.27.97 Equipe Gallinari) Received: from (viennet@localhost) by lpia2.ibp.fr (8.6.10/jtpda-5.0) id TAA26065 ; Wed, 29 May 1996 19:40:31 +0200 Date: Wed, 29 May 1996 19:40:31 +0200 Message-Id: <199605291740.TAA26065@lpia2.ibp.fr> To: matrix-sig@python.org CC: viennet@ura1507.univ-paris13.fr Subject: [PYTHON MATRIX-SIG] Two questions Sender: owner-matrix-sig@python.org Precedence: bulk Hi all, I'm using Numerical Python 0.36 since a few weeks and I'm really impressed... Two questions, however: 1- I'd like to compute with floats, instead of doubles, to save storage (for now, I frequently run out of my 100Mo swap space). 1.1 : this is probably a bug: >>> from Numeric import * >>> a = ones(10,Float(32)) >>> a.typecode() 'd' 1.2 : the above bug comes from an implicit conversion : >>> a = zeros(10,Float(32)) >>> a.typecode() 'f' >>> b = a + 1 >>> b.typecode() 'd' The consequence is that's currently impossible to compute with floats... ? I don't exactly remember the conclusions from the previous discussions about conversion rules, but the current behaviour is uncomfortable, especially with *big* arrays. Did I miss something ? 2- The second question : The following code builds and returns an array from C: PyArrayObject arr = (PyArrayObject *)PyArray_FromDims( nbdims,dims, PyArray_FLOAT ); if (!arr) { ... } do something with arr->data return Py_BuildValue("O", arr ); Is this right ? (This seems to leak). Thanks for your attention Emmanuel -- Emmanuel Viennet: LIPN - Institut Galilee - Universite Paris-Nord 93430 Villetaneuse - France ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Wed May 29 14:02:15 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id OAA26508 for matrix-sig-people; Wed, 29 May 1996 14:00:44 -0400 (EDT) Received: from icf.llnl.gov (icf.llnl.gov [128.115.45.2]) by python.org (8.7.5/8.7.3) with SMTPid OAA26501 for ; Wed, 29 May 1996 14:00:40 -0400 (EDT) Received: from nixon.llnl.gov (amy2.llnl.gov) by icf.llnl.gov (4.1/LLNL-1.19) id AA15205; Wed, 29 May 96 11:00:21 PDT Date: Wed, 29 May 96 11:00:21 PDT Message-Id: <9605291800.AA15205@icf.llnl.gov> X-Sender: dubois@icf.llnl.gov Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" To: Emmanuel.Viennet@laforia.ibp.fr (VIENNET Emmanuel 48.06.27.97 Equipe Gallinari) From: dubois1@llnl.gov (Paul F. Dubois) Subject: Re: [PYTHON MATRIX-SIG] Two questions Cc: matrix-sig@python.org X-Mailer: Sender: owner-matrix-sig@python.org Precedence: bulk We decided to make it harder to compute with floats rather than hard to behave normally. (:->. Seriously, the design tradeoff was to either force the person who wanted to be sure they stayed float to be more careful or to screw up automatic type conversion. Since double does not cost a great deal more CPU than single, the issue is just those arrays for which space is a major issue. In your example you need to make the scalar you are adding to 'a' to also be a float array of size 1. Yes, your example leaks. Just return the new array without Py_BuildValue, since the latter adds a reference. You already have one from the creation call. > > Hi all, > >I'm using Numerical Python 0.36 since a few weeks and I'm really >impressed... >Two questions, however: > > 1- I'd like to compute with floats, instead of doubles, to save > storage (for now, I frequently run out of my 100Mo swap space). > 1.1 : this is probably a bug: > >>> from Numeric import * > >>> a = ones(10,Float(32)) > >>> a.typecode() > 'd' > 1.2 : the above bug comes from an implicit conversion : > >>> a = zeros(10,Float(32)) > >>> a.typecode() > 'f' > >>> b = a + 1 > >>> b.typecode() > 'd' >The consequence is that's currently impossible to compute with >floats... ? I don't exactly remember the conclusions from the >previous discussions about conversion rules, but the current behaviour >is uncomfortable, especially with *big* arrays. Did I miss something ? > > > 2- The second question : >The following code builds and returns an array from C: > > PyArrayObject arr = (PyArrayObject *)PyArray_FromDims( > nbdims,dims, PyArray_FLOAT ); > if (!arr) { > ... > } > do something with arr->data > > return Py_BuildValue("O", arr ); > >Is this right ? (This seems to leak). > > >Thanks for your attention >Emmanuel > >-- >Emmanuel Viennet: >LIPN - Institut Galilee - Universite Paris-Nord >93430 Villetaneuse - France > >================= >MATRIX-SIG - SIG on Matrix Math for Python > >send messages to: matrix-sig@python.org >administrivia to: matrix-sig-request@python.org >================= > > Paul F. Dubois Lawrence Livermore National Laboratory ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Wed May 29 14:21:18 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id OAA26578 for matrix-sig-people; Wed, 29 May 1996 14:19:27 -0400 (EDT) Received: from lems.brown.edu (lems.brown.edu [128.148.128.4]) by python.org (8.7.5/8.7.3) with SMTPid OAA26549 for ; Wed, 29 May 1996 14:17:08 -0400 (EDT) Received: from lems23.lems.brown.edu by lems.brown.edu (5.x/SMI-SVR4) id AA14646; Wed, 29 May 1996 14:16:48 -0400 Received: by lems23.lems.brown.edu (5.x/SMI-SVR4) id AA26599; Wed, 29 May 1996 14:16:47 -0400 Date: Wed, 29 May 1996 14:16:46 -0400 (EDT) From: "Perry A. Stoll" X-Sender: pas@lems23 To: "VIENNET Emmanuel 48.06.27.97 Equipe Gallinari" Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Two questions In-Reply-To: <199605291740.TAA26065@lpia2.ibp.fr> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk On Wed, 29 May 1996, VIENNET Emmanuel 48.06.27.97 Equipe Gallinari wrote: > > Hi all, > > I'm using Numerical Python 0.36 since a few weeks and I'm really great! > > 1- I'd like to compute with floats, instead of doubles, to save Nasty matte, that. I'll wait for more knowledgable folks to answer that... > 2- The second question : > The following code builds and returns an array from C: > > PyArrayObject arr = (PyArrayObject *)PyArray_FromDims( > nbdims,dims, PyArray_FLOAT ); [snip]> > return Py_BuildValue("O", arr ); > Is this right ? (This seems to leak). That's a leak. In this case all you need to do: return (PyObject *)arr; When you create an object, in this case with the function PyArra_FromDims, you almost always own a reference to it. All you need to do is pass "your" reference back to the interpreter. Using Py_BuildValue, you're creating an extra reference to "arr" which never gets deleted and leaks memory. --Perry ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Wed May 29 14:21:20 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id OAA26556 for matrix-sig-people; Wed, 29 May 1996 14:17:21 -0400 (EDT) Received: from lems.brown.edu (lems.brown.edu [128.148.128.4]) by python.org (8.7.5/8.7.3) with SMTPid OAA26549 for ; Wed, 29 May 1996 14:17:08 -0400 (EDT) Received: from lems23.lems.brown.edu by lems.brown.edu (5.x/SMI-SVR4) id AA14646; Wed, 29 May 1996 14:16:48 -0400 Received: by lems23.lems.brown.edu (5.x/SMI-SVR4) id AA26599; Wed, 29 May 1996 14:16:47 -0400 Date: Wed, 29 May 1996 14:16:46 -0400 (EDT) From: "Perry A. Stoll" X-Sender: pas@lems23 To: "VIENNET Emmanuel 48.06.27.97 Equipe Gallinari" Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Two questions In-Reply-To: <199605291740.TAA26065@lpia2.ibp.fr> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk On Wed, 29 May 1996, VIENNET Emmanuel 48.06.27.97 Equipe Gallinari wrote: > > Hi all, > > I'm using Numerical Python 0.36 since a few weeks and I'm really great! > > 1- I'd like to compute with floats, instead of doubles, to save Nasty matte, that. I'll wait for more knowledgable folks to answer that... > 2- The second question : > The following code builds and returns an array from C: > > PyArrayObject arr = (PyArrayObject *)PyArray_FromDims( > nbdims,dims, PyArray_FLOAT ); [snip]> > return Py_BuildValue("O", arr ); > Is this right ? (This seems to leak). That's a leak. In this case all you need to do: return (PyObject *)arr; When you create an object, in this case with the function PyArra_FromDims, you almost always own a reference to it. All you need to do is pass "your" reference back to the interpreter. Using Py_BuildValue, you're creating an extra reference to "arr" which never gets deleted and leaks memory. --Perry ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Wed May 29 18:46:38 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id SAA27538 for matrix-sig-people; Wed, 29 May 1996 18:41:19 -0400 (EDT) Received: from simsg1.mdc.com (SIMSG1.MDC.COM [129.200.225.33]) by python.org (8.7.5/8.7.3) with SMTPid SAA27527 for ; Wed, 29 May 1996 18:40:49 -0400 (EDT) Received: by simsg1.mdc.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) id PAA23739; Wed, 29 May 1996 15:40:08 -0700 Date: Wed, 29 May 1996 15:40:05 -0700 (PDT) From: Brien Barton To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Numerical exception handling Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk My apologies if this has already been covered. The following is a demonstration of an error that is not handled gracefully when using array objects: >>> b=array([1,2,3]) >>> b.x=10 Fatal Python error: print_error called but no exception Abort (core dumped) Other objects, such as an integer variable, don't cause a core dump: >>> a=5 >>> a.x=10 Traceback (innermost last): File "", line 1, in ? TypeError: attribute-less object (assign or del) =============================================================================== Brien Barton McDonnell Douglas Aerospace, Huntington Beach, CA email: barton@simsg1.mdc.com, voice: (714)896-2249, fax:(714)896-5939 ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Sun Jun 2 08:33:36 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id IAA07225 for matrix-sig-people; Sun, 2 Jun 1996 08:12:13 -0400 (EDT) Received: from relay2.eunet.fr (relay2.EUnet.fr [192.134.192.149]) by python.org (8.7.5/8.7.3) with SMTPid IAA07218 for ; Sun, 2 Jun 1996 08:11:28 -0400 (EDT) Received: from d.univ-paris13.fr by relay2.eunet.fr (5.65c8d/96.05.03) via EUnet-France id AA17888; Sun, 2 Jun 1996 14:10:45 +0200 (MET) Received: from univ-paris13.fr (ura1507.univ-paris13.fr) by d.univ-paris13.fr; Sun, 2 Jun 96 14:04:27 +0200 (MET) Received: by univ-paris13.fr (SMI-8.6/SMI-SVR4) id OAA05452; Sun, 2 Jun 1996 14:08:52 +0100 Date: Sun, 2 Jun 1996 14:08:52 +0100 From: viennet@ura1507.univ-paris13.fr (Emmanuel Viennet) Message-Id: <199606021308.OAA05452@univ-paris13.fr> To: busby@icf.llnl.gov (L. Busby) Cc: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Gist and GNU readline In-Reply-To: <9605281646.AA29432@icf.llnl.gov> References: <9605281646.AA29432@icf.llnl.gov> Sender: owner-matrix-sig@python.org Precedence: bulk Hi, GIST seems to be a cool plotting package. For those like me who can't live without the line editing facilities from GNU readline, here is a useful patch. I did not make extensive tests, but it seems to work. Apply the following diff to the file gistCmodule.c from ftp-icf.llnl.gov:/pub/python/busby/pygist-1.0.tgz Then configure python with readline enabled and recompile. 13a14,22 > #ifdef WITH_READLINE > # error Sorry: The Gist module cannot presently co-exist with Readline. > #endif > > /* (Actually, Gist probably *can* co-exist with readline, by using the > * trick of replacing read() with a gist_read(), but I haven't had a chance > * to review the readline source yet.) > */ > 82,87d90 < /* GNU readline callback (E.viennet) */ < #ifdef WITH_READLINE < static int readline_event_hook(void); < extern int (*rl_event_hook)(); /* from readline.h */ < #endif < 409d411 < #ifndef WITH_READLINE 420,422d421 < #else /* GNU READLINE */ < rl_event_hook = readline_event_hook; < #endif 521d519 < #ifndef WITH_READLINE 524d521 < #endif 1429,1436d1425 < < #ifdef WITH_READLINE < static int readline_event_hook(void) { < do YGDispatch (); < while (!yPendingIn); < return 1; < } < #endif -- Emmanuel Viennet: LIPN - Institut Galilee - Universite Paris-Nord 93430 Villetaneuse - France ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Sun Jun 2 17:35:00 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id RAA07909 for matrix-sig-people; Sun, 2 Jun 1996 17:13:40 -0400 (EDT) Received: from goldilocks.LCS.MIT.EDU (goldilocks.lcs.mit.edu [18.27.0.167]) by python.org (8.7.5/8.7.3) with SMTPid RAA07902 for ; Sun, 2 Jun 1996 17:12:54 -0400 (EDT) Received: from ling-ling.zoo (ling-ling.lcs.mit.edu) by goldilocks.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA17174; Sun, 2 Jun 96 17:12:42 EDT Received: by ling-ling.zoo (SMI-8.6/SMI-SVR4) id RAA00600; Sun, 2 Jun 1996 17:13:03 -0400 Date: Sun, 2 Jun 1996 17:13:03 -0400 Message-Id: <199606022113.RAA00600@ling-ling.zoo> From: James Hugunin To: barton@simsg1.mdc.com Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Numerical exception handling Sender: owner-matrix-sig@python.org Precedence: bulk Interestingly enough, I'd received private mail about this bug hours before this was posted to the list. From: Brien Barton My apologies if this has already been covered. The following is a demonstration of an error that is not handled gracefully when using array objects: >>> b=array([1,2,3]) >>> b.x=10 Fatal Python error: print_error called but no exception Abort (core dumped) This will be fixed in the next release (which should be coming out shortly after the python workshop). -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Tue Jun 11 16:53:37 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id QAA05679 for matrix-sig-people; Tue, 11 Jun 1996 16:40:16 -0400 (EDT) Received: from harfang.CC.UMontreal.CA (harfang.CC.UMontreal.CA [132.204.2.102]) by python.org (8.7.5/8.7.3) with ESMTPid QAA05672 for ; Tue, 11 Jun 1996 16:40:11 -0400 (EDT) Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by harfang.CC.UMontreal.CA with ESMTP id QAA13152 (8.6.11/IDA-1.6 for ); Tue, 11 Jun 1996 16:38:05 -0400 Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id QAA13085; Tue, 11 Jun 1996 16:38:04 -0400 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id QAA18825; Tue, 11 Jun 1996 16:38:03 -0400 Date: Tue, 11 Jun 1996 16:38:03 -0400 Message-Id: <199606112038.QAA18825@cyclone.ERE.UMontreal.CA> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Vacation and move From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk This weekend I will leave Montreal for an extended vacation (all across the country to the West coast, then down to California, and finally to Texas), after which I will move to France, where I will be working for the next two years. I will stay on the SIG (hoping not to be inundated by large messages) and check my e-mail whenever I find an opportunity, but don't expect quick reactions from me during that time. I hope that you will all be working hard, and that I will find a much improved array module with lots of interesting applications when I return to work... My research project in France will require lots of Python programming. Essentially I will write a molecular modelling toolkit in Python, using C modules where necessary for speed. Although this toolkit will be specialized to my personal needs, I'll try to keep it as general as possible, and if I can find interested collaborators, it might eventually become a general-purpose tool, which would certainly be the most modern one in existence (which is not much of an achievement in view of the "state of the art" in this field). I expect to produce many general-purpose numerical modules as a side product, which I will of course make available and announce to the public. Finally, I'd like to announce my latest Python package, which is a Gaussian random number generator using Paul Dubois' URNG module. Anyone who wants a copy please write to me. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Fri Jun 14 02:47:28 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id CAA11550 for matrix-sig-people; Fri, 14 Jun 1996 02:32:45 -0400 (EDT) Received: from marsu.univ-lyon1.fr (root@marsu.univ-lyon1.fr [134.214.4.74]) by python.org (8.7.5/8.7.3) with SMTPid CAA11543 for ; Fri, 14 Jun 1996 02:31:54 -0400 (EDT) Received: from marsu (jlv@localhost [127.0.0.1]) by marsu.univ-lyon1.fr (8.6.12/8.6.9) with SMTP id IAA12155 for ; Fri, 14 Jun 1996 08:33:16 +0300 Message-ID: <31C0F99B.23AC23D5@zorglub.univ-lyon1.fr> Date: Fri, 14 Jun 1996 08:33:15 +0300 From: Jean-Louis Villecroze Organization: Observatoire de Lyon X-Mailer: Mozilla 3.0b4Gold (X11; I; Linux 1.3.94 i586) MIME-Version: 1.0 To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Create and fill an array from Numeric module in C function ?? Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk Hello, I'm currently working in the integration of a library in Python, and i would like to create and fill a Numeric array from my Data but directly in a C function, which be used in Python ... How make that's ... Tanks for any support .... Jean-Louis -- Jean-Louis Villecroze C.R.A.L. Tel (33) 78.86.83.81 Observatoire de Lyon Fax (33) 78.86.83.86 9, Avenue Charles Andre 69561 St Genis-Laval Cedex E-mail: jlv@zorglub.univ-lyon1.fr France URL: http://www-obs.univ-lyon1.fr/~jlv/jl.html ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Fri Jun 14 12:23:26 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id MAA12548 for matrix-sig-people; Fri, 14 Jun 1996 12:17:08 -0400 (EDT) Received: from ibp.ibp.fr (ibp.ibp.fr [132.227.60.30]) by python.org (8.7.5/8.7.3) with ESMTPid MAA12541 for ; Fri, 14 Jun 1996 12:16:30 -0400 (EDT) Received: from laforia.ibp.fr (laforia.ibp.fr [132.227.60.10]) by ibp.ibp.fr (8.6.12/jtpda-5.0) with ESMTP id SAA11655 ; Fri, 14 Jun 1996 18:15:59 +0200 Received: from lpia2.ibp.fr (singha.ibp.fr [132.227.201.18]) by laforia.ibp.fr (8.6.10/jtpda-5.0) with ESMTP id SAA03838 ; Fri, 14 Jun 1996 18:15:58 +0200 From: Emmanuel.Viennet@laforia.ibp.fr (VIENNET Emmanuel 48.06.27.97 Equipe Gallinari) Received: from (viennet@localhost) by lpia2.ibp.fr (8.6.10/jtpda-5.0) id SAA00386 ; Fri, 14 Jun 1996 18:15:57 +0200 Date: Fri, 14 Jun 1996 18:15:57 +0200 Message-Id: <199606141615.SAA00386@lpia2.ibp.fr> To: Jean-Louis Villecroze Cc: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Create and fill an array from Numeric module in C function ?? In-Reply-To: <31C0F99B.23AC23D5@zorglub.univ-lyon1.fr> References: <31C0F99B.23AC23D5@zorglub.univ-lyon1.fr> Sender: owner-matrix-sig@python.org Precedence: bulk Jean-Louis Villecroze writes: > Hello, > > I'm currently working in the integration of a library in Python, and i > would like to create and fill a Numeric array from my Data but directly > in a C function, which be used in Python ... > Quite easy :-) write a python-C glue function: - to create a new array and return it: int dims[ nbdims ] = { ... }; arr = (PyArrayObject *)PyArray_FromDims( nbdims, dims, PyArray_FLOAT ); use the memory pointed by arr->data When you're done, return a python object: return (PyObject *)arr; For more infos, look at Include/arrayobject.h, where you will find usefull comments. Have fun Emmanuel -- Emmanuel Viennet: LIPN - Institut Galilee - Universite Paris-Nord 93430 Villetaneuse - France ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Wed Jun 19 13:24:39 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id NAA04179 for matrix-sig-people; Wed, 19 Jun 1996 13:14:34 -0400 (EDT) Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.7.5/8.7.3) with SMTPid NAA04172 for ; Wed, 19 Jun 1996 13:13:14 -0400 (EDT) Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA22559; Wed, 19 Jun 96 13:08:00 -0400 Received: by maigret (950215.SGI.8.6.10/940406.SGI) for matrix-sig@python.org id NAA11503; Wed, 19 Jun 1996 13:09:35 -0400 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199606191709.NAA11503@maigret> Subject: [PYTHON MATRIX-SIG] FFT's To: matrix-sig@python.org Date: Wed, 19 Jun 1996 13:09:34 -0400 (EDT) X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk I never got time to test it, so I'm letting people give it a go as is -- my module to interface to fftpack is up on my website http://maigret.cog.brown.edu/Python/Extensions/FFT/ This is based on the fftpack.c which is part of TELA. I suspect my translation of TELA's fft.ct frontend into my FFT.py is buggy, since I don't really read TELA code. I'm especially suspicious of the real FFT's, but the complex ones seem to work ok. Anyway, if anyone gives this a try, let me know when you find bugs. --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Wed Jun 19 14:43:47 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id OAA04536 for matrix-sig-people; Wed, 19 Jun 1996 14:40:12 -0400 (EDT) Received: from pp2.shef.ac.uk (pp2.shef.ac.uk [143.167.1.32]) by python.org (8.7.5/8.7.3) with ESMTPid OAA04531 for ; Wed, 19 Jun 1996 14:40:06 -0400 (EDT) Received: from acse.sheffield.ac.uk (actually host osprey.shef.ac.uk) by pp2.shef.ac.uk with local SMTP (PP); Wed, 19 Jun 1996 19:39:32 +0100 Received: from serin.acse by acse.sheffield.ac.uk; Wed, 19 Jun 96 19:39:27 BST Received: by serin.acse (SMI-8.6/SMI-SVR4) id TAA00611; Wed, 19 Jun 1996 19:38:40 +0100 Date: Wed, 19 Jun 1996 19:38:40 +0100 From: fonseca@acse.shef.ac.uk (carlos fonseca) Message-Id: <199606191838.TAA00611@serin.acse> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Matrix multiplication X-Sun-Charset: US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk Hello, I may have missed any previous discussion on the implementation of matrix multiplication in Numerical Python. Anyway, I understand that the current implementation of .matrixMultiply() requires arrays to be 2-dimensional (has this changed?). While trying to port some matlab code to Numerical Python and make it more general (eliminate for loops, add functionality, etc), I realised that it would be nice to be able to: 1) treat (e.g.) the last 2 dimensions of an array as matrices, i.e., having arrays of matrices. For example, A[m-by-n-by-p] * B[p-by-q] = A[m-by-n-by-p] * B[1-p-by-q] = C[m-by-n-by-q] In this case, each of the m n-by-p matrices in A is multiplied by B. Another example would be A[m-by-n-by-p] * D[m-p-by-q] = E[m-by-n-by-q] Now, each of the m n-by-p matrices in A is multiplied by the corresponding p-by-q matrix in B 2) be able to do the sort of thing Yorick does where, for example, A[n-by-m-by-p] * B[p-by-q-by-r] = C[n-by-m-by-q-by-r] This looks more like (2-dimensional) matrix mutiplication. A simple way of achieving both would be to define a .dot(B,n) method such that A.dot(B,n) would compute the dot product of the rows of A by the corresponding rows of B (or any other dimension n, for that matter), reducing the number of dimensions by one like add.reduce does for a single array. The arrays should be conformable as for addition and multiplication. Essentially, A.dot(B,n) = add.reduce(A*B,n), but memory for the intermediate result A*B would not need to be allocated (a saving when casting of dimensions occurs, as in the example below). For two-dimensional arrays, A.matrixMultiply(B) could be written something like A[...,NewAxis].dot(B,-2). The automatic casting of the dimension introduced into A to match that of B would produce the desired results. Note that add.reduce(A[...,NewAxis]*B) would imply storing a 3-dimensional array. This would generalize well for any number of dimensions, I believe. For example, case 1) above could be written C = A[...,NewAxis].dot(B,-2) E = A[...,NewAxis].dot(D[...,NewAxis,:,:],-2) and case 2) could be written A[...,NewAxis,NewAxis].dot(B,-3) How do others feel about this issue? Carlos Fonseca P.S.: Why doesn't A[RubberIndex,NewAxis,NewAxis] work? >>> a 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 >>> a[RubberIndex,NewAxis] Traceback (innermost last): File "", line 1, in ? ArrayError: each subindex must be either a slice, an integer, RubberIndex, or None ^^^^^^^^^^^ !?? ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Wed Jun 19 14:44:15 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id OAA04536 for matrix-sig-people; Wed, 19 Jun 1996 14:40:12 -0400 (EDT) Received: from pp2.shef.ac.uk (pp2.shef.ac.uk [143.167.1.32]) by python.org (8.7.5/8.7.3) with ESMTPid OAA04531 for ; Wed, 19 Jun 1996 14:40:06 -0400 (EDT) Received: from acse.sheffield.ac.uk (actually host osprey.shef.ac.uk) by pp2.shef.ac.uk with local SMTP (PP); Wed, 19 Jun 1996 19:39:32 +0100 Received: from serin.acse by acse.sheffield.ac.uk; Wed, 19 Jun 96 19:39:27 BST Received: by serin.acse (SMI-8.6/SMI-SVR4) id TAA00611; Wed, 19 Jun 1996 19:38:40 +0100 Date: Wed, 19 Jun 1996 19:38:40 +0100 From: fonseca@acse.shef.ac.uk (carlos fonseca) Message-Id: <199606191838.TAA00611@serin.acse> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Matrix multiplication X-Sun-Charset: US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk Hello, I may have missed any previous discussion on the implementation of matrix multiplication in Numerical Python. Anyway, I understand that the current implementation of .matrixMultiply() requires arrays to be 2-dimensional (has this changed?). While trying to port some matlab code to Numerical Python and make it more general (eliminate for loops, add functionality, etc), I realised that it would be nice to be able to: 1) treat (e.g.) the last 2 dimensions of an array as matrices, i.e., having arrays of matrices. For example, A[m-by-n-by-p] * B[p-by-q] = A[m-by-n-by-p] * B[1-p-by-q] = C[m-by-n-by-q] In this case, each of the m n-by-p matrices in A is multiplied by B. Another example would be A[m-by-n-by-p] * D[m-p-by-q] = E[m-by-n-by-q] Now, each of the m n-by-p matrices in A is multiplied by the corresponding p-by-q matrix in B 2) be able to do the sort of thing Yorick does where, for example, A[n-by-m-by-p] * B[p-by-q-by-r] = C[n-by-m-by-q-by-r] This looks more like (2-dimensional) matrix mutiplication. A simple way of achieving both would be to define a .dot(B,n) method such that A.dot(B,n) would compute the dot product of the rows of A by the corresponding rows of B (or any other dimension n, for that matter), reducing the number of dimensions by one like add.reduce does for a single array. The arrays should be conformable as for addition and multiplication. Essentially, A.dot(B,n) = add.reduce(A*B,n), but memory for the intermediate result A*B would not need to be allocated (a saving when casting of dimensions occurs, as in the example below). For two-dimensional arrays, A.matrixMultiply(B) could be written something like A[...,NewAxis].dot(B,-2). The automatic casting of the dimension introduced into A to match that of B would produce the desired results. Note that add.reduce(A[...,NewAxis]*B) would imply storing a 3-dimensional array. This would generalize well for any number of dimensions, I believe. For example, case 1) above could be written C = A[...,NewAxis].dot(B,-2) E = A[...,NewAxis].dot(D[...,NewAxis,:,:],-2) and case 2) could be written A[...,NewAxis,NewAxis].dot(B,-3) How do others feel about this issue? Carlos Fonseca P.S.: Why doesn't A[RubberIndex,NewAxis,NewAxis] work? >>> a 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 >>> a[RubberIndex,NewAxis] Traceback (innermost last): File "", line 1, in ? ArrayError: each subindex must be either a slice, an integer, RubberIndex, or None ^^^^^^^^^^^ !?? ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Wed Jun 19 14:45:47 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id OAA04554 for matrix-sig-people; Wed, 19 Jun 1996 14:44:02 -0400 (EDT) Received: from pp2.shef.ac.uk (pp2.shef.ac.uk [143.167.1.32]) by python.org (8.7.5/8.7.3) with ESMTPid OAA04549 for ; Wed, 19 Jun 1996 14:43:58 -0400 (EDT) Received: from acse.sheffield.ac.uk (actually host osprey.shef.ac.uk) by pp2.shef.ac.uk with local SMTP (PP); Wed, 19 Jun 1996 19:43:30 +0100 Received: from serin.acse by acse.sheffield.ac.uk; Wed, 19 Jun 96 19:43:24 BST Received: by serin.acse (SMI-8.6/SMI-SVR4) id TAA00617; Wed, 19 Jun 1996 19:42:40 +0100 Date: Wed, 19 Jun 1996 19:42:40 +0100 From: fonseca@acse.shef.ac.uk (carlos fonseca) Message-Id: <199606191842.TAA00617@serin.acse> To: matrix-sig@PYTHON.ORG Subject: [PYTHON MATRIX-SIG] a.choose() segfaults X-Sun-Charset: US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk Apologies if this bug is already known: >>> a=arange(10) >>> a 0 1 2 3 4 5 6 7 8 9 >>> a.choose >>> a.choose() Segmentation Fault (core dumped) This happens both under Linux-i386, and under Solaris 2 on a Sun. Carlos ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Thu Jun 20 14:02:46 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id NAA07550 for matrix-sig-people; Thu, 20 Jun 1996 13:56:43 -0400 (EDT) Received: from gate.artinet.de (gate.artinet.de [193.96.148.1]) by python.org (8.7.5/8.7.3) with SMTPid NAA07544 for ; Thu, 20 Jun 1996 13:56:16 -0400 (EDT) Received: from samba.artinet.de by gate.artinet.de with smtp (Linux Smail3.1.29.1 #2) id m0uWnxD-0001dCC; Thu, 20 Jun 96 19:55 MET DST Received: by samba.artinet.de (Smail3.1.29.1 #3) id m0uWnxA-0007EIC; Thu, 20 Jun 96 19:55 MET DST Date: Thu, 20 Jun 1996 19:55:34 +0200 (MET DST) From: Tom Schwaller To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] ezmodule this week! (long message) In-Reply-To: <199606191709.NAA11503@maigret> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk Hi all, I am just finishing a new Python module ezmodule which is a new GUI-Library with an incorporated 3D-Api (very similar to OpenGL). It only works on X-windows, but a nice thing about it is, that you have all what you need in one very compact module. The Tk C-API should be available this year, which will finally blow out the Tcl Stuff in the Tk-Module, the Netscape plugin for Tk is also out soon, so why another toolkit? Well, look at it. It's fast, easy to use and very easy to install (compared to a Tkinter, OpenGL (Mesa), Tk/OpenGL widget installation) -rwxrwxr-x 1 et users 640870 Jun 20 16:33 ezmodule.so For a toolkit which allows you to programm in 3D (eg. with numerical Python, drawing surfaces with Gouraud shading) building GUI's (including notebooks, drawing of ppm's), that's really not bad. Setting callbacks and configuring widgets is dead easy. Here's a simple example: #!/usr/local/bin/python from EZ import * import sys def shellCB(widget, bitmap, color): global p2 widget.Configure(BITMAP_FILE=bitmap) p2.Configure(FOREGROUND=color) def main(): global p2 Initialize(1) SetApplicationName('Test1') f = Frame(label='NoteBook'); n = f.NoteBook() p1 = n.NoteBookPage(label='RadioButtons') p1.Configure(FOREGROUND="Blue", BACKGROUND="White") p1.im = p1.Label() p1.im.Configure(IMAGE='logo.ppm') p1.f = p1.Frame('Classification') p1.f.Configure(STACKING = VERTICAL, SIDE=LEFT, FOREGROUND="Red") p1.f.l1 = p1.f.RadioButton(label='VeryCcool', value=0) p1.f.l2 = p1.f.RadioButton(label='Cool', value=1) p1.f.l3 = p1.f.RadioButton(label='Ok', value=2) p1.f.l3 = p1.f.RadioButton(label='Bad', value=3) p1.f.l4 = p1.f.RadioButton(label='Very Bad', value=4) p2 = n.NoteBookPage(label='Pixmaps') p2.Configure(FOREGROUND="Red", BACKGROUND="#007f6f") p2.f = p2.Frame('Shells') p2.f.Configure(BITMAP_FILE='xterm.xpm') p2.f.Configure(STACKING = VERTICAL, SIDE=LEFT) i=0; colors=['Red', 'Green', 'Blue', 'Yellow', 'Cyan', 'Magenta'] for s in ['axp', 'blank', 'dec', 'sgi', 'sol', 'sun']: b = p2.f.RadioButton(label='xterm-'+s, gid=1, value=i) b.SetCallBack(shellCB, p2.f, 'xterm-'+s+'.xpm', colors[i]) i = i+1 p3 = n.NoteBookPage(label='Text') p3.t = p3.TextWidget() p3.t.LoadAnnotatedFile('text.txt') p4 = n.NoteBookPage(label='Misc') p4.f = p4.Frame() p4.f.Configure(BITMAP_FILE='xterm-sun.xpm') p4.b1 = p4.MenuButton(label='Menu') p4.b2 = p4.Button(label='Quit') p4.b2.SetCallBack(sys.exit) #p4.s = p4.Slider() f.Display() f.MainLoop() main() Check http://www.artinet.de/python/notebook.jpeg to see how it looks like. At http://www.artinet.de/python/EZWGL.tgz you can get EZGL from Maorong Zou Department of Mathematics The University of Texas at Austin Austin, TX 78712 mzou@ma.utexas.edu to have a first impression. I lost the annoucment. Somewehere in comp.os.linux.apps last week I used SWIG plus some handcoding for the callback and widget configuration mechanism. Fortunately I did not have to change the source code for it. I had to llok quite a bit to find a solution, but it wirks now. :-) Last but not least, here's the REDME. If you are interested, please tell me, so I'm even more motivated to work on it... ____________________________________________________________ Python ez-Module: Version 0.1 (20 June 1996) by Tom Schwaller (German Linux Magazine, tschwal@artinet.de) Tested and developped on Linux 1.2.13 (ELF) with gcc 2.7.0 Available at http://www.artinet.de/python/ez.html ----------------------------------------------------------------- This is my first version of the ez-Module, which interfaces the C-library EZWGL, done by Maorong Zou Department of Mathematics The University of Texas at Austin Austin, TX 78712 mzou@ma.utexas.edu This high-level library incorporates a GUI widget set for the X-Window System (No cross platform tool, sorry!) and Graphics commands, which resemble OpenGL a lot (except the names and (in the actual version) missing texture mapping). When I read the announcment I thought first, well another GUI library but when I saw the first GUI/3D applications with it on my screen, I immediately decided to write a Python module (had some time :-), being quite busy for the German Linux Magazine). It wrapps nearly 280 C-Functions, so it seemed ideal for testing SWIG (Simplified Wrapper and Interface Generator), done by David M. Beazley Department of Computer Science University of Utah Salt Lake City, Utah 84112 beazley@cs.utah.edu even though SWIG does not generate that kind of code I use to write myself wrapping libraries. But you can get results very fast, which is cool! Not everything is quite polished at the moment, but it seems to works. A challenge in writing the module things was the Python translation of the Widget Configuration p1 = n.NoteBookPage(label='RadioButtons') p1.Configure(FOREGROUND="Blue", BACKGROUND="White") which uses variable argument lists in the C-API. Also the callback mechnism was not straightforward, but look at the result to understand how it works: def callback0(): print 'No Argument!' def callback1(widget): print 'One Argument:', widget def callback2(widget, color): print 'Two Arguments:', widget, color a.SetCallBack(callback0) b.SetCallBack(callback1, w) c.SetCallBack(callback2, w, color) Just use the same number of arguments as in the function definition when registering the callback with SetCallBack. I found that solution very appealing, when coding it. Another problem was, that the header file ez.h is not ANSI at the moment, so I had to write the declarations myself for use with SWIG. As in the OpenGL module (Version 0.4 by James Hugunin, David Ascher and myself to follow soon I hope) I would have liked to use the matrix module, but I decided not to at the moment, so everybody interested in this module can give it a try. One will have to rewrite some of the 3d Graphics functions in the ez-module for easier use. At the moment just look at the OpenGL examples in the SWIG Distribtion to see how they work. Installation: This should be fairly easy. First compile the EZ library libEZ.a (You can do a shared library if you want) and make shure your compiler finds it while linking. Then add the following line to your Python Setup file (eg. after *noconfig*) ez ezmodule.c -DALLOW_NULL -DEZ_MAIN -lEZ -lX11 -lXext Install EZ.py somewhere in your PYTHONPATH. That's all. Now you are ready for GUI and 3D Programming. Testing: Change to the examples subdirectory and try test.py to see if it works (It really should!). You can also take a look at notebook.jpeg in the same directory to compare. More examples (GUI/3D) to follow.... I have not tested everything at the moment (there are also some small bugs in EZWGL itself, remeber it's a Beta Version), so I hope to get some feedback. In my eyes ezmodule is very well suited for medium sized Scientific applications (e.g take a look at the isosurf example in the EZGL examples subdirectory, which is taken from the Mesa distribution and rewritten in EZGL and you will see what I mean). Displaying shaded surfaces a la Matlab, Mathematica or Maple should be straightforward. Any volunteers? In conjunction with Numerical Python this module gives you a small and easy to use library for writing visualisation tools (Installing Python with Tkinter, OpenGL and some Tk/OpenGL widget is not that easy as it should be). Have fun. Enhacments, comments and contributions welcome. Please send them to tschwal@artinet.de to incorporate them in the next release. Thanks to all in the Python Community. Special thanks to Maorong Zou, for contributung EZWGL. ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Fri Jun 21 17:48:15 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id RAA10763 for matrix-sig-people; Fri, 21 Jun 1996 17:34:33 -0400 (EDT) Received: from pierce.llnl.gov (pierce.llnl.gov [128.115.18.253]) by python.org (8.7.5/8.7.3) with SMTPid RAA10758 for ; Fri, 21 Jun 1996 17:34:28 -0400 (EDT) Received: from icf.llnl.gov.llnl.gov by pierce.llnl.gov (8.6.10/LLNL-1.18/llnl.gov-03.95) id OAA02934; Fri, 21 Jun 1996 14:34:01 -0700 Received: from localhost by icf.llnl.gov.llnl.gov (4.1/SMI-4.1) id AA22428; Fri, 21 Jun 96 14:34:00 PDT Date: Fri, 21 Jun 1996 14:34:00 -0700 (PDT) From: "Tser-Yuan (Brian) Yang" To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] may be a bug Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk The following may have been posted or even corrected. If so, I apologize for sending junk mail. >>> from Numeric import * >>> a=array((1,2)) >>> import sys >>> sys.getrefcount(a) 2 >>> a 1 2 >>> sys.getrefcount(a) 5 >>> b=1 >>> sys.getrefcount(a) 4 Something is holding extra references Brian Yang ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Fri Jun 21 19:10:36 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id TAA11008 for matrix-sig-people; Fri, 21 Jun 1996 19:05:22 -0400 (EDT) Received: from pierce.llnl.gov (pierce.llnl.gov [128.115.18.253]) by python.org (8.7.5/8.7.3) with SMTPid TAA10995 for ; Fri, 21 Jun 1996 19:04:57 -0400 (EDT) Received: from icf.llnl.gov.llnl.gov by pierce.llnl.gov (8.6.10/LLNL-1.18/llnl.gov-03.95) id QAA20177; Fri, 21 Jun 1996 16:04:32 -0700 Received: from localhost by icf.llnl.gov.llnl.gov (4.1/SMI-4.1) id AA24007; Fri, 21 Jun 96 16:04:27 PDT Date: Fri, 21 Jun 1996 16:04:27 -0700 (PDT) From: "Tser-Yuan (Brian) Yang" To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Re: may be a bug Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk Brian Yang wrote: > The following may have been posted or even corrected. If so, I apologize > for sending junk mail. > >>> from Numeric import * > >>> a=array((1,2)) > >>> import sys > >>> sys.getrefcount(a) > 2 > >>> a > 1 2 > >>> sys.getrefcount(a) > 5 > >>> b=1 > >>> sys.getrefcount(a) > 4 First, I should correct myself: sys.getrefcount(a) should still be 5 after b=1, since assignment does not return a value. Some experiment showed that the extra reference counts are held by something in ArrayPrinter.py, because I commented out the following line in numeric.py: #from ArrayPrinter import arrayToString #set_print_function(arrayToString) and it hehaves like the following >>> from Numeric import * >>> a=array((1,3)) >>> a array([1,3], 'l') >>> import sys >>> sys.getrefcount(a) 3 >>> 1 1 >>> sys.getrefcount(a) 2 Brian Yang ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Mon Jun 24 13:38:41 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id NAA16727 for matrix-sig-people; Mon, 24 Jun 1996 13:25:58 -0400 (EDT) Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.7.5/8.7.3) with ESMTPid NAA16722 for ; Mon, 24 Jun 1996 13:25:49 -0400 (EDT) Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id RAA12936 (8.6.11/IDA-1.6); Mon, 24 Jun 1996 17:23:41 GMT Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id NAA28001; Mon, 24 Jun 1996 13:23:40 -0400 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id NAA05534; Mon, 24 Jun 1996 13:23:40 -0400 Date: Mon, 24 Jun 1996 13:23:40 -0400 Message-Id: <199606241723.NAA05534@cyclone.ERE.UMontreal.CA> To: fonseca@acse.shef.ac.uk CC: matrix-sig@python.org In-reply-to: <199606191838.TAA00611@serin.acse> (fonseca@acse.shef.ac.uk) Subject: Re: [PYTHON MATRIX-SIG] Matrix multiplication From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > I may have missed any previous discussion on the implementation of > matrix multiplication in Numerical Python. Anyway, I understand that > the current implementation of .matrixMultiply() requires arrays to be > 2-dimensional (has this changed?). True. As a temporary solution, I can offer a (slow) Python function that is more general: def dot(a1, a2): r1 = len(a1.shape)-1 r2 = len(a2.shape)-1 axes = [r1] + range(r1) a1 = a1.transpose(axes).copy() f = a1.reshape a1 = apply(f, a1.shape + r2*(1,)) a2 = a2.copy() f = a2.reshape a2 = apply(f, a2.shape[0:1] + r1*(1,) + a2.shape[1:]) return Numeric.add.reduce(a1*a2) ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Mon Jun 24 13:38:51 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id NAA16734 for matrix-sig-people; Mon, 24 Jun 1996 13:28:35 -0400 (EDT) Received: from condor.CC.UMontreal.CA (condor.CC.UMontreal.CA [132.204.2.103]) by python.org (8.7.5/8.7.3) with ESMTPid NAA16729 for ; Mon, 24 Jun 1996 13:28:22 -0400 (EDT) Received: from eole.ERE.UMontreal.CA (eole.ERE.UMontreal.CA [132.204.10.20]) by condor.CC.UMontreal.CA with ESMTP id RAA13006 (8.6.11/IDA-1.6); Mon, 24 Jun 1996 17:26:20 GMT Received: from cyclone.ERE.UMontreal.CA by eole.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id NAA28570; Mon, 24 Jun 1996 13:26:20 -0400 Received: by cyclone.ERE.UMontreal.CA (951211.SGI.8.6.12.PATCH1042/5.17) id NAA05614; Mon, 24 Jun 1996 13:26:19 -0400 Date: Mon, 24 Jun 1996 13:26:19 -0400 Message-Id: <199606241726.NAA05614@cyclone.ERE.UMontreal.CA> To: tbyang@icf.llnl.gov CC: matrix-sig@python.org In-reply-to: (tbyang@icf.llnl.gov) Subject: Re: [PYTHON MATRIX-SIG] Re: may be a bug From: hinsenk@ere.umontreal.ca (Konrad HINSEN) Sender: owner-matrix-sig@python.org Precedence: bulk > Some experiment showed that the extra reference counts are held by > something in ArrayPrinter.py, because I commented out the following line > in numeric.py: I can't see how the array printer could keep references, since it doesn't assign anything to a global variable. ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Mon Jun 24 14:55:56 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id OAA16902 for matrix-sig-people; Mon, 24 Jun 1996 14:53:15 -0400 (EDT) Received: from lems.brown.edu (lems.brown.edu [128.148.128.4]) by python.org (8.7.5/8.7.3) with SMTPid OAA16897 for ; Mon, 24 Jun 1996 14:53:06 -0400 (EDT) Received: from lems61.lems.brown.edu by lems.brown.edu (5.x/SMI-SVR4) id AA03312; Mon, 24 Jun 1996 14:52:34 -0400 Received: by lems61.lems.brown.edu (SMI-8.6/SMI-SVR4) id OAA00434; Mon, 24 Jun 1996 14:52:33 -0400 Date: Mon, 24 Jun 1996 14:52:32 -0400 (EDT) From: "Perry A. Stoll" X-Sender: pas@lems61 To: "Tser-Yuan (Brian) Yang" Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Re: may be a bug In-Reply-To: Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk I know of one thing that can hold references. In the interpreter, you can reference the last value returned by using an underscore. For example: >>> x = 1 >>> x 1 >>> z = _ >>> z 1 So that should account for the drop in refcount from 5 to 4 in your example and probably why you found the problem in ArrayPrinter.py. I haven't read the documentation too recently, but I don't recall seeing this - is this documented? How did I find it? just lucky. -Perry On Fri, 21 Jun 1996, Tser-Yuan (Brian) Yang wrote: > Brian Yang wrote: > > > The following may have been posted or even corrected. If so, I apologize > > for sending junk mail. > > > >>> from Numeric import * > > >>> a=array((1,2)) > > >>> import sys > > >>> sys.getrefcount(a) > > 2 > > >>> a > > 1 2 > > >>> sys.getrefcount(a) > > 5 > > >>> b=1 > > >>> sys.getrefcount(a) > > 4 > > Some experiment showed that the extra reference counts are held by > something in ArrayPrinter.py, because I commented out the following line > in numeric.py: > ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Wed Jun 26 12:11:55 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id MAA21316 for matrix-sig-people; Wed, 26 Jun 1996 12:04:28 -0400 (EDT) Received: from CNRI.Reston.VA.US (CNRI.Reston.Va.US [132.151.1.1]) by python.org (8.7.5/8.7.3) with SMTPid MAA21310 for ; Wed, 26 Jun 1996 12:04:16 -0400 (EDT) Received: from monty.cnri.reston.va.us by CNRI.Reston.VA.US id aa10104; 26 Jun 96 12:01 EDT Received: from localhost by monty via SMTP (940816.SGI.8.6.9/940406.SGI) id MAA27949; Wed, 26 Jun 1996 12:01:45 -0400 Message-Id: <199606261601.MAA27949@monty> To: "Perry A. Stoll" cc: "Tser-Yuan (Brian) Yang" , matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Re: may be a bug In-reply-to: Your message of "Mon, 24 Jun 1996 14:52:32 EDT." References: From: Guido van Rossum X-Face: S#(=*/yxQ]{^7V>yr9ElReWJZvi^x!]tR6g^m5q?|f@/ChslBZm*tWH=*aicc{$>4c/M:$. \@@o)psQitBQa!z()VjNsj+aqtDI/J0jX'a/=Qhn|gv6coKW}06%6gyuTn)oc6 Date: Wed, 26 Jun 1996 12:01:45 -0400 Sender: owner-matrix-sig@python.org Precedence: bulk > I know of one thing that can hold references. In the interpreter, you can > reference the last value returned by using an underscore. [...] > I haven't read the documentation too recently, but I don't recall seeing > this - is this documented? How did I find it? just lucky. Probably the only place is the section "Recent Additions" in the tutorial, e.g. http://www.python.org/doc/tut/node64.html --Guido van Rossum (home page: http://www.python.org/~guido/) ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Fri Jun 28 17:11:04 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id RAA27184 for matrix-sig-people; Fri, 28 Jun 1996 17:06:50 -0400 (EDT) Received: from pierce.llnl.gov (pierce.llnl.gov [128.115.18.253]) by python.org (8.7.5/8.7.3) with SMTPid RAA27179 for ; Fri, 28 Jun 1996 17:06:36 -0400 (EDT) Received: from icf.llnl.gov.llnl.gov by pierce.llnl.gov (8.6.10/LLNL-1.18/llnl.gov-03.95) id OAA18025; Fri, 28 Jun 1996 14:06:09 -0700 Received: from localhost by icf.llnl.gov.llnl.gov (4.1/SMI-4.1) id AA06092; Fri, 28 Jun 96 14:06:08 PDT Date: Fri, 28 Jun 1996 14:06:08 -0700 (PDT) From: "Tser-Yuan (Brian) Yang" To: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Re: may be a bug In-Reply-To: Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk I finally figure out what is holding the two extra reference counts in the following block of script. >>> from Numeric import * >>> a=array((1,2)) >>> import sys >>> sys.getrefcount(a) 2 >>> print a 1 2 >>> sys.getrefcount(a) 4 One is the variable 'data' in arrayToString (in ArrayPrinter.py), and the other is the exception handler. After I added 'del data' before the return of arrayToString, and commented out some statements in arrayToString: if max_line_width is None: # try: # max_line_width = sys.output_line_width # except AttributeError: max_line_width = 77 if precision is None: # try: # precision = sys.float_output_precision # except AttributeError: precision = 8 if suppress_small is None: # try: # suppress_small = sys.float_output_suppress_small # except AttributeError: suppress_small = 0 The same block of script gives: >>> from Numeric import * >>> a=array((1,2)) >>> import sys >>> print a 1 2 >>> sys.getrefcount(a) 2 I agree this is probably a feature rather than a bug, since these reference counts will be released after printing another array. Just to point this out in case some of you may be interested. Brian Yang ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Tue Jul 2 14:01:58 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id NAA06610 for matrix-sig-people; Tue, 2 Jul 1996 13:41:40 -0400 (EDT) Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.7.5/8.7.3) with SMTPid NAA06605 for ; Tue, 2 Jul 1996 13:41:34 -0400 (EDT) Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA23355; Tue, 2 Jul 96 13:36:10 -0400 Received: by maigret (950215.SGI.8.6.10/940406.SGI) for matrix-sig@python.org id NAA18467; Tue, 2 Jul 1996 13:37:58 -0400 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199607021737.NAA18467@maigret> Subject: [PYTHON MATRIX-SIG] mac numeric version? To: matrix-sig@python.org Date: Tue, 2 Jul 1996 13:37:58 -0400 (EDT) X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk Has anyone ported the numeric stuff to the mac version since we last 'spoke' on this topic? --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Tue Jul 2 14:30:20 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id OAA06716 for matrix-sig-people; Tue, 2 Jul 1996 14:22:22 -0400 (EDT) Received: from estel.uindy.edu (ESTEL.UINDY.EDU [192.146.191.197]) by python.org (8.7.5/8.7.3) with SMTPid OAA06711 for ; Tue, 2 Jul 1996 14:22:14 -0400 (EDT) Received: by estel.uindy.edu (NX5.67d/NX3.0M) id AA21589; Tue, 2 Jul 96 13:27:03 -0500 Date: Tue, 2 Jul 96 13:27:03 -0500 From: Steve Spicklemire Message-Id: <9607021827.AA21589@estel.uindy.edu> To: da@maigret.cog.brown.edu Cc: matrix-sig@python.org In-Reply-To: <199607021737.NAA18467@maigret> (da@maigret.cog.brown.edu) Subject: Re: [PYTHON MATRIX-SIG] mac numeric version? Sender: owner-matrix-sig@python.org Precedence: bulk David, I can put up a Mac version of python with numeric stuff in it if you like. It's 68K only... (that's all I have.. sorry) -steve >>>>> "David" == David Ascher writes: David> Has anyone ported the numeric stuff to the mac version David> since we last 'spoke' on this topic? David> --david David> ================= MATRIX-SIG - SIG on Matrix Math for David> Python David> send messages to: matrix-sig@python.org administrivia to: David> matrix-sig-request@python.org ================= ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Sat Jul 6 12:21:41 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id MAA15706 for matrix-sig-people; Sat, 6 Jul 1996 12:11:06 -0400 (EDT) Received: from drew.cog.brown.edu (drew.cog.brown.edu [128.148.208.10]) by python.org (8.7.5/8.7.3) with SMTPid MAA15697 for ; Sat, 6 Jul 1996 12:10:32 -0400 (EDT) Received: by drew.cog.brown.edu (5.57/Ultrix3.0-C) id AA11515; Sat, 6 Jul 96 12:05:08 -0400 Received: by maigret (950215.SGI.8.6.10/940406.SGI) for matrix-sig@python.org id MAA03009; Sat, 6 Jul 1996 12:06:59 -0400 From: da@maigret.cog.brown.edu (David Ascher) Message-Id: <199607061606.MAA03009@maigret> Subject: [PYTHON MATRIX-SIG] numeric #ifdef To: matrix-sig@python.org Date: Sat, 6 Jul 1996 12:06:59 -0400 (EDT) X-Mailer: ELM [version 2.4 PL24] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk 1. I'd like to know what #define I can use to test for the presence of the Numeric extension -- I'm writing C code which can both be used with or without the extension, but obviously I need to #ifdef out the Array calls if the numeric extension is not defined. If such a #define isn't in 0.36, I'd like it to be in 0.36+x. 2. If anyone wants the files needed to make PythonWin numeric-aware, let me know. It's just a replacement py130-3.dll and the usual lib/numeric files. If you want to know the required source changes, I can tell you that as well. I'll put it up on my web site as soon as I get the time to write a little HTML. 3. The URNG module is hard to port, because of its reliance on drand48(), which AFAIK is not available on Windows95/NT or the Mac. Is there a free drand48 somewhere? Cheers, --david ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Tue Jul 16 19:20:37 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id TAA01255 for matrix-sig-people; Tue, 16 Jul 1996 19:10:56 -0400 (EDT) Received: from pierce.llnl.gov (pierce.llnl.gov [128.115.18.253]) by python.org (8.7.5/8.7.3) with SMTPid TAA01250 for ; Tue, 16 Jul 1996 19:10:51 -0400 (EDT) Received: from icf.llnl.gov.llnl.gov by pierce.llnl.gov (8.6.10/LLNL-1.18/llnl.gov-03.95) id QAA01569; Tue, 16 Jul 1996 16:10:12 -0700 Received: from localhost by icf.llnl.gov.llnl.gov (4.1/SMI-4.1) id AA10971; Tue, 16 Jul 96 16:10:11 PDT Date: Tue, 16 Jul 1996 16:10:11 -0700 (PDT) From: "Tser-Yuan (Brian) Yang" To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] FYI: OverflowError in PrettyPrinter.py Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk FYI. A member of our group discovered that the variable max_val in the function _floatFormat of ArrayPrinter.py needed to be set to a number less than 2.15e9 on our machines. Otherwise it caused the following error message: Traceback (innermost last): File "", line 1, in ? File "./PrettyPrinter.py", line 45, in arrayToString format, item_length = _floatFormat(data, precision, suppress_small) File "./PrettyPrinter.py", line 109, in _floatFormat max_str_len = len(str(int(max_val))) + precision + 2 OverflowError: float too large to convert Brian Yang ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Tue Jul 23 15:13:01 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id PAA18099 for matrix-sig-people; Tue, 23 Jul 1996 15:05:18 -0400 (EDT) Received: from eeel.nist.gov (sparky.eeel.nist.gov [129.6.64.163]) by python.org (8.7.5/8.7.3) with SMTPid PAA18094 for ; Tue, 23 Jul 1996 15:05:11 -0400 (EDT) Received: from fermi.eeel.nist.gov by eeel.nist.gov (4.1/SMI-3.2-del.6) id AA29014; Tue, 23 Jul 96 15:04:25 EDT Received: by fermi.eeel.nist.gov (8.6.12/SMI-3.2-del.5) id OAA14652; Tue, 23 Jul 1996 14:01:42 GMT Date: Tue, 23 Jul 1996 14:01:42 GMT From: mclay@eeel.nist.gov (Michael McLay) Message-Id: <199607231401.OAA14652@fermi.eeel.nist.gov> To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Speed of opengl Sender: owner-matrix-sig@python.org Precedence: bulk Is openglmodule-0.33.c the most current version of the opengl module? There were a couple other modules available so I wasn't sure. I modified the glu, glut and opengl modules so the prefixes are no longer necessary. They seemed redundent and C'ish. I also had to patch up the example fog.py and movelight.py demonstration programs in order to get them to work. For movelight.py this involved changing: glut.MouseFunc(movelight) glut.MotionFunc(motion) glut.ReshapeFunc(reshape) glut.DisplayFunc(display) to glut.SetMouseFuncCallback(movelight) glut.MouseFunc() glut.SetMotionFuncCallback(motion) glut.MotionFunc() glut.SetReshapeFuncCallback(reshape) glut.ReshapeFunc() glut.SetDisplayFuncCallback(display) glut.DisplayFunc() glut.DisplayFunc(display) The performance of the python OpenGL module on a Linux system isn't meeting my design objectives and I was wondering if I'm doing something wrong or if it's just slow software. The system uses Mesa-1.2.8 on a Pentium 90 system with #9GXE 64Pro graphics card. When I run the the Mesa samples/speed test for the configuration the results are: antialiasing: off depthtest: off fog: off lighting: off shading: flat texturing: off Points per second: 112360 Lines per second: 53763.4 Triangles per second: 15408.3 Rects per second: 10172.9 This is pretty slow compared to an SGI. I can live with it in the application but I'd like the drawing rate to be about 4x this speed. I may even break down and buy a Acellerated-X OpenGL server for the PC, but that would be a last resort. The performance seems to drop below an acceptable speed when I use it with Python. Here's a test that draws 500 random length lines. import Ranf rlines = (Ranf.random_sample(3000) -.5)*4 t = time.time() Lines(rlines) tr = time.time() - t 1000 random lines takes 0.173418 seconds (5766.412600 lines/second 1000 random lines takes 0.041487 seconds (24103.948646 lines/second There seems to be a great difference in speed if the line length becomes very small. I did some more testing including a comparison with the Mesa sample/speed.c test. The C version draws 100 lines in a glBegin ... glEnd loop and then repeat this loop 1000 times. for (i = 0; i < repeatCount; i++) { v1[0] = 10; v1[1] = 10; v1[2] = 10; v2[0] = 20; v2[1] = 20; v2[2] = 10; glBegin(GL_LINES); for (j = 0; j < loopCount; j++) { glVertex2fv(v1); glVertex2fv(v2); } glEnd(); The results of this test reports: Lines per second: 44052.9 I wrote a Python based test that approximates this number of line draws. small = N.array([0.,.0,0.,10.0,10.0,10.0]) for i in range(13): small = small.concat(small) size = len(small)/6 t = time.time() Lines(small) tr = time.time() - t print "speed line test takes %f seconds (%f lines/second)" %( tr, size/tr) It results plotted about 8192 lines inside the Lines() command. The report printed in reshape 600 300 speed line test takes 1.377772 seconds (5945.831500 lines/second) 1000 random lines takes 0.104659 seconds (9554.843538 lines/second This test uses the same size window as the test script. The viewport is closer to the objects being drawn so the lines appear longer. The Python based version draw at a rate of ~6000 l/s compared to the C version was drawing at 44k l/s. The difference is difficult to uderstand. It looks like all the time should be spent inside the gl_Lines loop, so why isn't performance better. Did I miss something in my test? Michael ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Tue Jul 23 15:54:58 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id PAA18246 for matrix-sig-people; Tue, 23 Jul 1996 15:48:02 -0400 (EDT) Received: from goldilocks.LCS.MIT.EDU (goldilocks.lcs.mit.edu [18.27.0.167]) by python.org (8.7.5/8.7.3) with SMTPid PAA18241 for ; Tue, 23 Jul 1996 15:47:59 -0400 (EDT) Received: from ling-ling (ling-ling.lcs.mit.edu) by goldilocks.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA23705; Tue, 23 Jul 96 15:47:14 EDT Message-Id: <31F52C85.1994@sls.lcs.mit.edu> Date: Tue, 23 Jul 1996 15:48:21 -0400 From: Jim Hugunin Reply-To: jjh@goldilocks.lcs.mit.edu Organization: MIT - LCS - SLS X-Mailer: Mozilla 3.0b5aGold (WinNT; I) Mime-Version: 1.0 To: Michael McLay Cc: matrix-sig@python.org Subject: Re: [PYTHON MATRIX-SIG] Speed of opengl References: <199607231401.OAA14652@fermi.eeel.nist.gov> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-matrix-sig@python.org Precedence: bulk Michael McLay wrote: > > Is openglmodule-0.33.c the most current version of the opengl module? > There were a couple other modules available so I wasn't sure. That's the most recent publically available version. David Ascher, Tom Schwaller and myself need to find the time to clean up and distribute a final version sometime soon. > I modified the glu, glut and opengl modules so the prefixes are no > longer necessary. They seemed redundent and C'ish. I also had to ... More reason to produce a clean version so each person doesn't have to do this. > The performance of the python OpenGL module on a Linux system isn't > meeting my design objectives and I was wondering if I'm doing > something wrong or if it's just slow software. The system uses > Mesa-1.2.8 on a Pentium 90 system with #9GXE 64Pro graphics card. ... > This is pretty slow compared to an SGI. I can live with it in the > application but I'd like the drawing rate to be about 4x this speed. Well, a P-90 is pretty slow compared to an SGI, and Mesa is not exactly a speed demon. > I may even break down and buy a Acellerated-X OpenGL server for the > PC, but that would be a last resort. This is the price you'll have to pay if you want really great performance (or switch over to NT 4.0 where they have a fairly nice implementation distributed with the OS). > The performance seems to drop below an acceptable speed when I use it > with Python. Here's a test that draws 500 random length lines. Now this part I should be able to help with. First, I'd advise using the idiom: gl.Begin(GL_LINES) gl.Vertex(array_of_vertices) gl.End(GL_LINES) its a much more general approach and Lines will probably be fading out of future versions of the module. As far as the speeds that you're noticing, this is completely counter to my own experiences. Could you send me the complete source (both python and C) to your test programs so that I can look at them and run my own tests? -Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Tue Jul 23 22:12:18 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id WAA19169 for matrix-sig-people; Tue, 23 Jul 1996 22:06:58 -0400 (EDT) Received: from oasis.unl.edu (oasis.unl.edu [129.93.34.17]) by python.org (8.7.5/8.7.3) with SMTPid WAA19164 for ; Tue, 23 Jul 1996 22:06:50 -0400 (EDT) Received: by oasis.unl.edu (950911.SGI.8.6.12.PATCH825/940406.SGI) for matrix-sig@python.org id VAA04504; Tue, 23 Jul 1996 21:06:02 -0500 From: drh@oasis.unl.edu (Doug Heisterkamp) Message-Id: <199607240206.VAA04504@oasis.unl.edu> Subject: [PYTHON MATRIX-SIG] reverse function To: matrix-sig@python.org Date: Tue, 23 Jul 1996 21:06:02 -0500 (CDT) X-Mailer: ELM [version 2.4 PL23] Content-Type: text Sender: owner-matrix-sig@python.org Precedence: bulk Hi, After upgrading to version 3.6 I noticed that the reverse function is not working: >>> a = arange(100).reshape(5,20) >>> reverse(a) Traceback (innermost last): File "", line 1, in ? File "/usr/local/lib/python/numeric/Numeric.py", line 124, in reverse return m[None] #::-1] ArrayError: index must be either an int or a sequence If I use the part that is commented out: >>> a[::-1] 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 >>> Using ::-1 seems to work for 1 and 2D arrays. Is there a reason not to use it? What is the exact meaning of :: ? Thanks, Doug drh@oasis.unl.edu ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Sat Jul 27 10:55:39 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id KAA00260 for matrix-sig-people; Sat, 27 Jul 1996 10:47:43 -0400 (EDT) Received: from catch.ivab.se (root@catch.ivab.se [193.180.173.22]) by python.org (8.7.5/8.7.3) with ESMTPid FAA24568; Thu, 25 Jul 1996 05:51:56 -0400 (EDT) Received: from arnold.image.ivab.se (arnold.image.ivab.se [193.180.173.119]) by catch.ivab.se (8.6.12/8.6.12) with SMTP id LAA21657; Thu, 25 Jul 1996 11:51:04 +0200 Received: by arnold.image.ivab.se; (5.65v3.2/1.1.8.2/16Nov95-0104AM) id AA02801; Thu, 25 Jul 1996 11:51:01 +0200 Date: Thu, 25 Jul 1996 11:51:01 +0200 Message-Id: <9607250951.AA02801@arnold.image.ivab.se> From: Fredrik Lundh To: matrix-sig@python.org, image-sig@python.org Subject: [PYTHON MATRIX-SIG] [PREANNOUNCEMENT] netCDF reader for the matrix and image extensions Reply-To: fredrik_lundh@ivab.se Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk Just a little teaser ;-) I've pulled together a netCDF reader that doesn't need any C library: data = NetCdfDataFile.open("dataset.cdf") # list variables print "variables:", data.keys() # global attributes print "global attributes:", data.attr.keys() # dump all variables in the file for i in data.keys(): var = data[i] print i, data.type, data.shape print "- data:", var[:] if var.attr: print "- local attributes:", var.attr.keys() It currently sports a slightly different interface compared to Bill Noon's, but they can probably be unified. And of course, to create or modify netCDF files, you need to use the real library. - The above module can be used with the Numerical extension to load data as multidimensional arrays for further processing. var = data["foo"] foo = Numeric.array(var) # 1-dimensional foo.reshape(var.shape) # make n-dimensional - It can also be used with the next version of PIL 0.2 in order to load image data from netCDF files (PIL still supports 8-bit data only, but you can use scale/offset to fit other data into the 0..255 range). # foo must be a 2-dimensional variable var = data["foo"] im = Image.new("L", var.shape) im.putdata(var, scale, offset) im.save("output.jpg") I plan to release this with PIL 0.2. Comments and flames are welcome. /F ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Thu Aug 1 17:44:13 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id RAA03906 for matrix-sig-people; Thu, 1 Aug 1996 17:37:03 -0400 (EDT) Received: from phoenix.geog.ubc.ca (phoenix.geog.ubc.ca [137.82.51.1]) by python.org (8.7.5/8.7.3) with ESMTPid RAA03901 for ; Thu, 1 Aug 1996 17:36:58 -0400 (EDT) Received: from leghorn.geog.ubc.ca (leghorn [137.82.51.2]) by phoenix.geog.ubc.ca (8.7.3/8.7.3) with ESMTP id OAA04862 for ; Thu, 1 Aug 1996 14:36:08 -0700 (PDT) Received: (acannon@localhost) by leghorn.geog.ubc.ca (8.6.12/8.6.12) id OAA06704; Thu, 1 Aug 1996 14:36:06 -0700 Date: Thu, 1 Aug 1996 14:36:05 -0700 (PDT) From: Alex Cannon To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] Gist plots with Tk? Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk This may seem off-topic, but I couldn't decide whether this should go in matrix-sig (because of the gist tie in) or the GUI list (because of Tk). Anyhow, is it possible to get Tk and the gist module to coexist? As an example, I've modified the simple 'hello world' tkinter script to plot a line with gist. Running the script and clicking on the 'Plot' button causes the gist window to popup, but the plot doesn't finish until after quitting the Tk mainloop() (when running with -i). Is there a way to do this properly? Alex -- from Tkinter import * from gist import * class Test(Frame): def plot(self): plg([0, 1]) def createWidgets(self): self.QUIT = Button(self, {'text': 'Quit', 'fg': 'red', 'command': self.quit}) self.QUIT.pack({'side': 'left', 'fill': 'both'}) self.gist_test = Button(self, {'text': 'Plot', 'command': self.plot}) self.gist_test.pack({'side': 'left'}) def __init__(self, master=None): Frame.__init__(self, master) Pack.config(self) self.createWidgets() test = Test() test.mainloop() ------------------------------------------------------------ Alex Cannon (acannon@geog.ubc.ca) Geography 240C Atmospheric Science Programme Tel: (604) 822-2269 University of British Columbia, B.C. Fax: (604) 822-6150 ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Fri Aug 2 14:02:54 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id NAA06373 for matrix-sig-people; Fri, 2 Aug 1996 13:54:33 -0400 (EDT) Received: from yorvic.york.ac.uk (yorvic.york.ac.uk [144.32.72.22]) by python.org (8.7.5/8.7.3) with SMTPid NAA06368 for ; Fri, 2 Aug 1996 13:54:29 -0400 (EDT) From: Leo Caves Date: Fri, 2 Aug 1996 18:53:51 +0100 Message-Id: <9608021853.ZM1030@pingu> X-Mailer: Z-Mail (3.2.2 10apr95 MediaMail) To: matrix-sig@python.org Subject: [PYTHON MATRIX-SIG] "linking" variables to python Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-matrix-sig@python.org Precedence: bulk Hi! context: extending python with a legacy FORTRAN app. and have used the PyArray_FromDimsAndData() function to link FORTRAN common block arrays to python array objects. is working beautifully... Q: how to do the same with scalar variables ? eg. INTEGER N in FORTRAN declaration to python object (say) n I am trying to avoid acessing as array of length 1. Also, I would like to ask what the current opinions on the perennial array indexing question are. i.e. indexed from 0 (in python) vs. 1 in FORTRAN. My point here is that I have a whole load of integer arrays in FORTRAN which serve as indices into other arrays. Therefore these would have to be remapped to point to the correct place in the python array. Thanks, Leo Caves ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Fri Aug 9 18:28:05 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id SAA06301 for matrix-sig-people; Fri, 9 Aug 1996 18:20:23 -0400 (EDT) Received: from goldilocks.LCS.MIT.EDU (goldilocks.lcs.mit.edu [18.27.0.167]) by python.org (8.7.5/8.7.3) with SMTPid SAA06292 for ; Fri, 9 Aug 1996 18:20:17 -0400 (EDT) Received: from ling-ling.lcs.mit.edu by goldilocks.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA14084; Fri, 9 Aug 96 18:19:24 EDT Received: by ling-ling.lcs.mit.edu with Microsoft Mail id <01BB861F.8A1BDA90@ling-ling.lcs.mit.edu>; Fri, 9 Aug 1996 18:21:10 -0400 Message-Id: <01BB861F.8A1BDA90@ling-ling.lcs.mit.edu> From: Jim Hugunin To: "'matrix-sig@python.org'" Subject: [PYTHON MATRIX-SIG] Release 1.0alpha1 now available Date: Fri, 9 Aug 1996 18:21:09 -0400 Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Sender: owner-matrix-sig@python.org Precedence: bulk The first alpha release of the final 1.0 version of the Numeric = Extensions is now available. This release contains ZERO patches to the = python core. To use it you must have python1.4b2 installed on your = computer (much thanks to Guido for adding all of the needed = functionality to this release!). If you are working under Windows NT (or 95) there is a binary release = available that includes all needed files. The 1.0 number here means that I'm finally happy with the basic = functionality and design of the system and don't intend to make many = more changes. This version unfortunately changes many details from the = previous version and will break most existing code. I don't want to = ever do that again (not to you or to myself). There is an ugly, but = fairly complete piece of documentation in html for all of the functions = provided, as well as a very simple example module to help illustrate the = use of the C API. The alpha release means that this is not at all well tested. I'm = counting on you folks to track down the bugs now, and when that seems = under control, I'll send it out to the main list as a Beta release = (ideally coinciding with the first official release of python1.4). You = should expect a very rapid release schedule at this point as I = desperately want this out soon! Please beat hard on this and let me know where it breaks! For Unix users: Untar this wherever you want, and follow the instructions in INSTALL. Please try and build the dynamically loadable version, and let me know = about your success on different systems. I'd really like to include = binaries in the next alpha release (in preparation for the beta = release). ftp://ftp.sls.lcs.mit.edu/pub/jjh/NumPy-1.0a1.tar.gz There is also an fftmodule in this release. If you don't have fftpack = on your system, you can grab ftp://ftp.sls.lcs.mit.edu/pub/jjh/rfft-1.2.tar.gz=20 which is very easy to compile on almost all unix systems. This is = stolen from the rlab source code distribution site. For Windows 95/NT Unzip this into wherever you have your binary version of python = installed. This should be into a directory that already has a = py140-b1.dll in it. That will be replaced by the beta 2 version of that = library contained in the zip file. ftp://ftp.sls.lcs.mit.edu/pub/jjh/Win32-NumPy-1.0a1.zip Enjoy - Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Fri Aug 9 18:34:46 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id SAA06319 for matrix-sig-people; Fri, 9 Aug 1996 18:32:34 -0400 (EDT) Received: from goldilocks.LCS.MIT.EDU (goldilocks.lcs.mit.edu [18.27.0.167]) by python.org (8.7.5/8.7.3) with SMTPid SAA06314 for ; Fri, 9 Aug 1996 18:32:30 -0400 (EDT) Received: from ling-ling.lcs.mit.edu by goldilocks.LCS.MIT.EDU (4.1/SLS-4.1.1) id AA14165; Fri, 9 Aug 96 18:31:36 EDT Received: by ling-ling.lcs.mit.edu with Microsoft Mail id <01BB8621.3E6D8BF0@ling-ling.lcs.mit.edu>; Fri, 9 Aug 1996 18:33:22 -0400 Message-Id: <01BB8621.3E6D8BF0@ling-ling.lcs.mit.edu> From: Jim Hugunin To: "'matrix-sig@python.org'" Subject: [PYTHON MATRIX-SIG] Standard Numerical Library Date: Fri, 9 Aug 1996 18:33:21 -0400 Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Sender: owner-matrix-sig@python.org Precedence: bulk I'm getting tired of explaining to people why the Numeric Extensions = don't contain matrix inversion or fft operators. I intend to fix this = by providing a standard library with the 1.0 release of the system. The = primitive fftmodule in the current alpha1 release is my start on this. Here are my plans for this library, please comment: 1) Based on LAPACK and FFTPACK FORTRAN libraries, but only using the = subset available with the RLab distribution. This is a very nice = version of these libraries in C, and on every system I have available = they've compiled with "./configure; make" without a hitch (like some = scripting languages I know). They also seem to compile fairly easily = under NT. I'd distribute these packages as well for people without the = "real" libraries on their machines. 2) Implementing a bare minimum of functionality, in a simple, clean, = unsophisticated way. ie. fft( (0,0,1,1) ) will just work. No need to setup work areas, etc. 3) Implementing the "basic" functions 1D FFT Matrix Inversion Matrix Eigenvalues Matrix Determinant Any requests? I'm back - Jim ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Sat Aug 10 05:51:52 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id FAA07836 for matrix-sig-people; Sat, 10 Aug 1996 05:44:06 -0400 (EDT) Received: from avocado.pc.helsinki.fi (janne@avocado.pc.Helsinki.FI [128.214.75.66]) by python.org (8.7.5/8.7.3) with SMTPid FAA07831 for ; Sat, 10 Aug 1996 05:43:56 -0400 (EDT) Received: (from janne@localhost) by avocado.pc.helsinki.fi (8.6.10/8.6.9) id MAA09333; Sat, 10 Aug 1996 12:42:56 +0300 To: Jim Hugunin Cc: "'matrix-sig@python.org'" Subject: Re: [PYTHON MATRIX-SIG] Standard Numerical Library References: <01BB8621.3E6D8BF0@ling-ling.lcs.mit.edu> From: Janne Sinkkonen Date: 10 Aug 1996 12:42:55 +0300 In-Reply-To: Jim Hugunin's message of Fri, 9 Aug 1996 18:33:21 -0400 Message-ID: Lines: 19 X-Mailer: Gnus v5.2.25/XEmacs 19.14 Sender: owner-matrix-sig@python.org Precedence: bulk Jim Hugunin writes: > 3) Implementing the "basic" functions > > 1D FFT > Matrix Inversion > Matrix Eigenvalues > Matrix Determinant > Any requests? SVD is a must. Pseudoinverse would be nice and trivial after SVD. Is there a function returning the index of a minimum or a maximum? This would be nice in some ANN applications, although I don't know whether it is generally useful enough to be a standard part of the language. -- Janne Sinkkonen ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Sat Aug 10 09:49:58 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id JAA08133 for matrix-sig-people; Sat, 10 Aug 1996 09:48:39 -0400 (EDT) Received: from acds21 (acds21.physik.rwth-aachen.de [137.226.30.31]) by python.org (8.7.5/8.7.3) with SMTPid JAA08128 for ; Sat, 10 Aug 1996 09:48:07 -0400 (EDT) Received: by acds21; (5.65/1.1.8.2/01Dec95-8.2MPM) id AA03384; Sat, 10 Aug 1996 15:46:47 +0200 Date: Sat, 10 Aug 1996 15:46:47 +0200 From: Konrad Hinsen Message-Id: <9608101346.AA03384@acds21> To: jjh@goldilocks.lcs.mit.edu Cc: matrix-sig@python.org In-Reply-To: <01BB861F.8A1BDA90@ling-ling.lcs.mit.edu> (message from Jim Hugunin on Fri, 9 Aug 1996 18:21:09 -0400) Subject: Re: [PYTHON MATRIX-SIG] Release 1.0alpha1 now available Sender: owner-matrix-sig@python.org Precedence: bulk The first alpha release of the final 1.0 version of the Numeric = Extensions is now available. This release contains ZERO patches to the = Great news! That makes me eager to get back to work ;-) Unfortunately I won't have good enough net access until the end of next week (right now I have an awfully slow link at 7 cents a minute, but it works from my laptop...) Please beat hard on this and let me know where it breaks! Sounds like fun! I'm getting tired of explaining to people why the Numeric Extensions = don't contain matrix inversion or fft operators. I intend to fix this = by providing a standard library with the 1.0 release of the system. The = primitive fftmodule in the current alpha1 release is my start on this. Good idea. But we should make sure that this standard library is a subset of some "full" liraries, even if these will not be ready immediately. I don't want two incompatible matrix inversion functions! 1) Based on LAPACK and FFTPACK FORTRAN libraries, but only using the = subset available with the RLab distribution. This is a very nice = I don't know this subset. Is it fully compatible with the current versions of the full libraries? 2) Implementing a bare minimum of functionality, in a simple, clean, = unsophisticated way. ie. fft( (0,0,1,1) ) will just work. No need to setup work areas, etc. That should be the default for any numeric library! 3) Implementing the "basic" functions 1D FFT Matrix Inversion Matrix Eigenvalues Matrix Determinant I predict a lot of discussion about what is "basic". My only addition to your list would be solution of linear equations (inversion is a terribly inefficient way to do that). I would also like to have different algorithms where appropriate, a fast one for standard problems and a foolproof one for people who don't know what they are doing (i.e. SVD for matrix inversion etc.) As soon as I am back in business, I will update my LAPACK interface for the new version. I don't pretend that it is the last word on linear algebra libraries (some problems, like application to different types of matrices (symmetric etc.) have been completely ignored until now), but it has worked quite well for me in the past, and I have a couple of application algorithms that use it. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca Departement de Chimie | Tel.: +1-514-343-6111 ext. 3953 Universite de Montreal | Fax: +1-514-343-7586 C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/ Montreal (QC) H3C 3J7 | Francais (phase experimentale) ------------------------------------------------------------------------------- ================= MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org ================= From owner-matrix-sig@python.org Sat Aug 10 14:20:25 1996 Received: (from majordom@localhost) by python.org (8.7.5/8.7.3)id OAA08500 for matrix-sig-people; Sat, 10 Aug 1996 14:19:45 -0400 (EDT) Received: from relay-4.mail.demon.net (relay-4.mail.demon.net [158.152.1.108]) by python.org (8.7.5/8.7.3) with SMTPid OAA08489 for ; Sat, 10 Aug 1996 14:19:23 -0400 (EDT) Received: from post.demon.co.uk ([158.152.1.72]) by relay-4.mail.demon.net id aa19981; 10 Aug 96 18:18 GMT Received: from gaivota.demon.co.uk ([158.152.83.237]) by relay-3.mail.demon.net id aa28037; 10 Aug 96 19:16 +0100 Received: from localhost by gaivota.demon.co.uk with smtp id m0upIZT-0002inC (Debian /\oo/\ Smail3.1.29.1 #29.37); Sat, 10 Aug 96 19:15 BST Date: Sat, 10 Aug 1996 19:15:35 +0100 (BST) From: Carlos Fonseca To: Janne Sinkkonen cc: Jim Hugunin , "'matrix-sig@python.org'" Subject: Re: [PYTHON MATRIX-SIG] Standard Numerical Library In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-matrix-sig@python.org Precedence: bulk On 10 Aug 1996, Janne Sinkkonen wrote: > Is there a function returning the index of a minimum or a > maximum? This would be nice in some ANN applications, although I don't > know whether it is generally useful enough to be a standard part of > the language. > I find functions like max(), min(), and sort() (fun() for short) more limited than the corresponding argfun(), simply because the former could always be writen as take(a,argfun(a)). Unfortunate