From jjh@mama-bear.lcs.mit.edu Fri Dec 1 23:39:41 1995
From: jjh@mama-bear.lcs.mit.edu (James Hugunin)
Date: Fri, 1 Dec 95 18:39:41 -0500
Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available
Message-ID: <9512012339.AA04760@ling-ling.lcs.mit.edu.LCS.MIT.EDU>
I can't completely believe I finally finished this latest release. I'm
heading off for the python workshop on Monday, and I wanted to get this out
before I left.
The whole package can be found at the same site as before, (if you don't
know the site you need to send me mail saying that you agree to test and
review the object and I'll give you the site).
The NEWS for this release follows. The two major incompatible changes at
the very beginning are why this release took so long (that and the fact that
I have a few other demands on my time). Please let me know what you think
of these changes!
I finally feel good about the feature set and performance of the object.
If nobody has any major complaints (or REALLY good suggestions like Yorick
pseudo and rubber indices, curse them) then I don't expect the basic
structure of the object to change much in the future.
As before, don't hesitate to send me your gripes, your bugs, and your
praise (if ever merited). However, I'll be gone for most of next week at
the python workshop, so I might be slow in replying.
-Jim
**********
Release 0.20 of the matrix object
Major incompatible changes (I expect that these might elict some
interesting discussion):
1) Matrix indexing now returns by reference to the original data, not by
value. As a side effect of this change, arbitrary sequences are not allowed
in multidimensional indices, but only single indices, slices, RubberIndex
and None.
Note: there is a new method, take, which will allow you to index a matrix
with an arbitrary sequence as before.
This change was motivated both by the possible speed increases (about 40%
for some code) and more importantly by issues of clarity of expression. I
couldn't come up with any other way to make the following hold:
m[0][1] is an efficient way to index a multi-dimensional array.
m[0,1] == m[0][1]
m[0,] == m[0]
2) No more ranks of operators, instead Yorick style pseudo indices are
used. As a side effect of this, outer product is now a convenience function
rather than a method on ofuncs.
For functions of unbounded rank (currently the only kind of function
supported by my omathmodule) Yorick pseudo indices support a superset of
what was possible to express using ranks and outer products. There is no
fundamental reason not to have both pseudo indices and ranks, I just think
that it's conceptually cleaner not to mix the two up.
Here's a fun thing you can do with pseudo indices that is a lot uglier with
ranks and outer products:
x=mrange(-1, 1, 2.0/m)[All,None]
y=mrange(-1, 1, 2.0/n)
r=sqrt(x**2+y**2)
z=exp(-r**2)*cos(2*pi*r)
This is modified from a piece of Tom Schwaller's code for producing a 2d
surface plot. Notice that this is much more efficient than generating x and
y as full 2d matrices.
Major improvements:
1) This new release is about 40% faster than 0.11 (on a benchmark code I
was given by the folks at LBL).
2) No longer need to say a[[1,2,3]] for multidimensional indexing, now
a[1,2,3] works just fine (thanks Konrad!)
3) Type coercion is performed more or less according to the same rules that
python uses. So I can add a matrix of ints to one of floats, and I wind up
with a matrix of floats at the end.
4) Finally there's a matrixMultiply method for matrices that behaves
reasonably for all combinations of matrix, vector, and scalar arguments.
Minor improvements:
1) pickling of matrices works (and uses a binary format with endianness
labeled for both efficiency and portability).
2) The omath module supports trig functions as object methods (if m has
method sin, then I can take sin(m) and get the sine of m. This also works
on matrices of objects. omathmodule can be used now a a replacement for
cmathmodule and mathmodule.
3) transpose now does the appropriate arbitrary permutation of dimensions.
4) More matrix methods, and better implementations of existing ones.
Notes
1) The code hasn't actually grown to more than double it's previous size.
This is just the effect of including all of Konrad's patches to the python
Grammar as complete files rather than diffs. Hopefully, this will be able
to be cleanly seperated from the matrix object, but I just don't have the
time to deal with that right now.
Things I plan to fix ASAP
1) Integrate Chris Chase's grammar patches to replace Slice(None,None,2) with ::2.
2) Make matrixMultiply work on matrices larger than 2d (this requires a
notion of rank).
3) Fix the memory leaks
4) Make some real documentation (I already have doc strings for all the
matrix methods, I'm hoping to steal some code to convert this to "real"
documentation at the workshop).
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Sun Dec 3 16:59:56 1995
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Sun, 3 Dec 1995 11:59:56 -0500
Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available
In-Reply-To: <9512012339.AA04760@ling-ling.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Fri, 1 Dec 95 18:39:41 -0500)
Message-ID: <199512031659.LAA21925@cyclone.ERE.UMontreal.CA>
Major incompatible changes (I expect that these might elict some
interesting discussion):
OK, let's start ;-)
1) Matrix indexing now returns by reference to the original data, not by
value. As a side effect of this change, arbitrary sequences are not allowed
in multidimensional indices, but only single indices, slices, RubberIndex
and None.
I don't see any problem with this. Of course the documentation (to be written...)
should be clear so that everyone knows the implications of this, but since
Python variables in general hold references, it shouldn't be a surprise
for anyone.
This change was motivated both by the possible speed increases (about 40%
for some code) and more importantly by issues of clarity of expression. I
couldn't come up with any other way to make the following hold:
m[0][1] is an efficient way to index a multi-dimensional array.
m[0,1] == m[0][1]
m[0,] == m[0]
I don't see why these are important features, but of course they don't
do any harm.
2) No more ranks of operators, instead Yorick style pseudo indices are
used. As a side effect of this, outer product is now a convenience function
rather than a method on ofuncs.
As another side effect, we have lost some important functionality.
For functions of unbounded rank (currently the only kind of function
supported by my omathmodule) Yorick pseudo indices support a superset of
what was possible to express using ranks and outer products. There is no
fundamental reason not to have both pseudo indices and ranks, I just think
that it's conceptually cleaner not to mix the two up.
The pseudo indices are not a replacement for ranks, but just a convenient
notation for some structural functions. I like this notation very much,
but we still need ranks. And as you said, there is no reason not to have both.
Apart from the problem of functions with finite rank (which we will
certainly need e.g. for matrix inversion), it is not true that pseudo
indiced support a superset of what is possible with ranks and the
associated structural functions. In fact, it is just the opposite, as
I will demonstrate.
First let me explain how the new pseudo indices fit into the J-style
array system. J doesn't have the concept of "indexing"; instead there
are a few structural functions that can be applied with different
ranks to construct all kinds of rearrangements of array elements. All
the (pseudo) indices we currently have can be mapped onto these
structural functions as follows:
- Explicit numerical indices, lists, slices and the rubber index
are equivalent to the function take() (see my Array.py) with
various rank.
- The pseudo-index "None" is equivalent to the J-function "itemize"
that adds an axis of length one to its argument. I didn't implement
this in Array.py, but it is a trivial addition.
- The contracting rubber index (* in Yorick, I haven't found it yet
in Python)is equivalent to the function ravel() with appropriate
rank.
As an example, the outer product mentioned above could be
(inefficiently) implemented in my Array.py as:
def outer(f, a, b):
return f(reshape[0](a, shape(b)), b)
So every construction with (pseudo) indices can be emulated with three
structural functions of suitable rank. On the other hand, there are
constructions that cannot be expressed with pseudo indices, for
example all cases where a function rank is not a constant but a
variable. In fact these can be handled in Python -- as opposed to
Yorick -- by remembering that a[x,y,z] is really just a shorthand for
a[(x,y,z)] and that the index tuple can be constructed from an
expression, but that seems more like an implementation accident than a
feature to me.
More importantly, pseudo indices don't help in the least when it comes
to reductions and general inner products that require the specification
of an axis. Yorick does this by putting the function into the index
list -- a clever idea, but I'd prefer to keep a clear distinction
between functions and arguments. The current matrix module allows
a second parameter to reduce() that indicates the axis, which is
exactly how APL handles the problem.
But both these methods are restrictive. They are sufficient only
for reductions with functions of unbounded rank. I can give an
example of an actual application where this is not enough: I had
a sequence of n rotation matrices, stored in a rank-3 array D of
shape (n, 3, 3). Then the net rotation of this sequence could
be obtained by matrixMultiply.reduce(D). Since matrix
multiplication is a rank-2 function, I couldn't express this
with the current matrix module.
The bottom line: pseudo indices are convenient shorthands for
the most common structural functions, but they are no replacement
for the general function rank scheme we had in version 0.11.
And now for something completely different
------------------------------------------
1) Names
This may seem to be a minor issue, but we should tackle this before
we all get used to the current ones.
I strongly dislike the type-specific constructors (also used for
output). They should be IntegerMatrix, FloatMatrix, ComplexMatrix,
CharacterMatrix, and GeneralMatrix. Anyone is free to define shorter
names for efficient typing, if desired.
I even propose a more radical renaming. Many people associate "matrix"
with the 2D-matrices from linear algebra. So it would be better to
call our general objects "arrays", and leave the name "matrix" for
linear-algebra type objects that are restricted to rank 2 and use *
for matrix multiplication. They could be implemented in Python based
on arrays.
The pseudo-index 'None' is very confusing. An index "2" picks item
number two from an axis, an index "All" picks all items from an
axis. Consequently "None" should pick no item from an axis, which is
course is a pointless operation. What it really does is create a new
axis, so it should be named "New". "RubberIndex" is not confusing,
but a bit strange, so it's worth thinking about alternatives. How
about "Skip", in the sense "skip all following axes for explicit
consideration"? For the "*" in Yorick I propose "Contract".
2) Constructors
Currently there are two constructors, matrix() and Matrix(), with
slightly different behaviour: matrix() takes a single argument that is
a (possibly nested) Python sequence object, e.g. a list. Matrix()
takes an arbitrary number of arguments that are made into a list
before being passed to matrix(). So Matrix(1,2,3) is equivalent
to matrix([1,2,3]).
I find it confusing to have to constructors with almost the same name
and almost the same function. I propose to have only one, of whatever
name, which behaves like matrix(). The reason is that many matrix
functions accept nested lists or tuples instead of matrix arguments,
e.g. matrix([2,3]) + [5,4] works. So does matrix([2,3]) +
matrix([5,4]), which is equivalent. But Matrix([2,3]) + [5,4] and
Matrix([2,3]) + Matrix([5,4]) are not equivalent (well, they are in
this simple exaple due to broadcasting, but not in general). So
matrix() can be considered a conversion function from lists and tuples
to arrays, which is needed anyway.
I realize that the calling style of Matrix() has the advantage of
saving one pair of brackets, but I don't consider this important
enough to create the current confusion.
I guess that's enough for today ;-) I wish the lucky participants
of the workshop much fun and hope that they don't have to eat
spam for lunch ;-)
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From forrest@rose.rsoc.rockwell.com Mon Dec 4 19:25:11 1995
From: forrest@rose.rsoc.rockwell.com (Dave Forrest)
Date: Mon, 4 Dec 1995 13:25:11 -0600
Subject: [PYTHON MATRIX-SIG] Renaming and Constructors
Message-ID: <9512041925.AA00688@feynman.rsoc.rockwell.com>
> From owner-matrix-sig@python.org Mon Dec 4 12:06 CST 1995
> Date: Sun, 3 Dec 1995 11:59:56 -0500
> From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad)
[snip]
>
> I strongly dislike the type-specific constructors (also used for
> output). They should be IntegerMatrix, FloatMatrix, ComplexMatrix,
> CharacterMatrix, and GeneralMatrix. Anyone is free to define shorter
> names for efficient typing, if desired.
>
> I even propose a more radical renaming. Many people associate "matrix"
> with the 2D-matrices from linear algebra. So it would be better to
Yes!, Yes we do! (Some of us even thought that that was the reason
for having a Matrix SIG)
> call our general objects "arrays", and leave the name "matrix" for
> linear-algebra type objects that are restricted to rank 2 and use *
> for matrix multiplication. They could be implemented in Python based
> on arrays.
I thought that we had decided a while back to call it "Tensor" because
"Array" was a more general, computer-science concept.
[snip]
> 2) Constructors
>
[snip]
> I find it confusing to have to constructors with almost the same name
> and almost the same function. I propose to have only one, of whatever
> name, which behaves like matrix(). The reason is that many matrix
> functions accept nested lists or tuples instead of matrix arguments,
> e.g. matrix([2,3]) + [5,4] works. So does matrix([2,3]) +
> matrix([5,4]), which is equivalent. But Matrix([2,3]) + [5,4] and
> Matrix([2,3]) + Matrix([5,4]) are not equivalent (well, they are in
> this simple exaple due to broadcasting, but not in general). So
> matrix() can be considered a conversion function from lists and tuples
> to arrays, which is needed anyway.
I agree.
David Forrest
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Mon Dec 4 22:12:32 1995
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Mon, 4 Dec 1995 17:12:32 -0500
Subject: [PYTHON MATRIX-SIG] Renaming and Constructors
In-Reply-To: <9512041925.AA00688@feynman.rsoc.rockwell.com> (forrest@rose.rsoc.rockwell.com)
Message-ID: <199512042212.RAA14614@cyclone.ERE.UMontreal.CA>
I thought that we had decided a while back to call it "Tensor" because
"Array" was a more general, computer-science concept.
But what we have now corresponds to that concept. In contrary, a
tensor is something completely different: a mathematical object with
certain transformation behaviour under coordinate system transforms.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From stoll@atr-sw.atr.co.jp Tue Dec 5 08:23:21 1995
From: stoll@atr-sw.atr.co.jp (Perry A. Stoll)
Date: Tue, 05 Dec 1995 17:23:21 +0900
Subject: [PYTHON MATRIX-SIG] A bug fix for release 0.20
Message-ID: <199512050823.RAA27076@ciris21.atr-sw.atr.co.jp>
While Jim Hugunin is gone at the Python conference, I thought I'd
share a bug fix with the list. Finding this bug was accidental (trying
to cast a matrix of ints to types.ListType), but unavoidable (it core
dumped).
Here is a brief example which demonstrates this:
import Matrix, types
Matrix.mrange(1,5).cast(types.ListType)
The fix is simple once you've found it ;-) It is only one line in the
file Objects/matrixtypes.c. Here are the diffs:
564c566
< sizeof(PyObject), 'O', PyMatrix_OBJECT};
---
> sizeof(PyObject *), 'O', PyMatrix_OBJECT};
i.e. the size of entries in an ObjectMatrix should be the size of a
pointer to a PyObject struct, not the size of a PyObject struct.
And now for more poking around in the guts of the matrix lib...
-Perry Stoll
Research Associate, CSRL Advanced Telecommunications Research
2-2 Hikaridai, Seika-cho Soraku-gun, Kyoto 619-02 JAPAN
VOICE: +81-774-95-1217 FINGER: stoll@atrwide.atr.co.jp
FAX : +81-774 95-1208 EMAIL: stoll@atr-sw.atr.co.jp
PGP 2.6 Key fingerprint = AF 56 5C D8 5F 78 BA FD 21 6E 2A 68 C4 55 9E B0
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From mclay@eeel.nist.gov Tue Dec 5 13:28:54 1995
From: mclay@eeel.nist.gov (Michael McLay)
Date: Tue, 5 Dec 95 08:28:54 EST
Subject: [PYTHON MATRIX-SIG] Re: Python vs Perl: benchmarking for speed?
In-Reply-To: <HINSENK.95Dec4173948@cyclone.ERE.UMontreal.CA>
References: <496if1$8hg@mrnews.mro.dec.com>
<MSPAL.95Nov25111659@iws174a.MPPMU.MPG.DE>
<DIprtI.A4p@ig.co.uk>
<49m6j5$m7v@mrnews.mro.dec.com>
<49of3l$86g@csnews.cs.colorado.edu>
<HINSENK.95Dec4173948@cyclone.ERE.UMontreal.CA>
Message-ID: <9512051328.AA03465@acdc.eeel.nist.gov>
Hinsen Konrad (hinsenk@cyclone.ERE.UMontreal.CAwrites:
> It seems to me that the argument you replied to was a purely
> hypothetical one, to illustrate the usefulness of tabulated
> benchmarks. Noone suggests that Perl takes ten times more time
> for a given problem than Python. In fact, I would be very much
> surprised if the difference between Perl and Python would ever
> reach a factor of ten in any direction for any problem.
What an invitation. How about doing that benchmark comparison of
Python matrix math vs. Perl matrix math that the world has been
waiting for. I can't wait to see the results:-)
Michael
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From forrest@rose.rsoc.rockwell.com Tue Dec 5 13:41:53 1995
From: forrest@rose.rsoc.rockwell.com (Dave Forrest)
Date: Tue, 5 Dec 1995 07:41:53 -0600
Subject: [PYTHON MATRIX-SIG] Re: Python vs Perl: benchmarking for speed?
Message-ID: <9512051341.AA03099@feynman.rsoc.rockwell.com>
> From owner-matrix-sig@python.org Tue Dec 5 07:35 CST 1995
[snip]
>
> Hinsen Konrad (hinsenk@cyclone.ERE.UMontreal.CAwrites:
> > It seems to me that the argument you replied to was a purely
> > hypothetical one, to illustrate the usefulness of tabulated
> > benchmarks. Noone suggests that Perl takes ten times more time
> > for a given problem than Python. In fact, I would be very much
> > surprised if the difference between Perl and Python would ever
> > reach a factor of ten in any direction for any problem.
>
> What an invitation. How about doing that benchmark comparison of
> Python matrix math vs. Perl matrix math that the world has been
> waiting for. I can't wait to see the results:-)
IMHO, this "dialogue" is not relevant to the Matrix discussion. Please
take it elsewhere.
David Forrest
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@mama-bear.lcs.mit.edu Thu Dec 7 17:47:03 1995
From: jjh@mama-bear.lcs.mit.edu (James Hugunin)
Date: Thu, 7 Dec 95 12:47:03 -0500
Subject: [PYTHON MATRIX-SIG] Ranks and pseudo indices
References: <199512031659.LAA21925@cyclone.ERE.UMontreal.CA>
Message-ID: <9512071747.AA07309@ling-ling.lcs.mit.edu.LCS.MIT.EDU>
Konrad writes:
> 2) No more ranks of operators, instead Yorick style pseudo indices are
> used. As a side effect of this, outer product is now a convenience
function
> rather than a method on ofuncs.
>
> As another side effect, we have lost some important functionality.
>
> For functions of unbounded rank (currently the only kind of function
> supported by my omathmodule) Yorick pseudo indices support a superset of
> what was possible to express using ranks and outer products. There is no
> fundamental reason not to have both pseudo indices and ranks, I just think
> that it's conceptually cleaner not to mix the two up.
>
> The pseudo indices are not a replacement for ranks, but just a convenient
> notation for some structural functions. I like this notation very much,
> but we still need ranks. And as you said, there is no reason not to have both.
>
I'll look at your specific comments, but I don't yet find them convincing.
> So every construction with (pseudo) indices can be emulated with three
> structural functions of suitable rank. On the other hand, there are
> constructions that cannot be expressed with pseudo indices, for
> example all cases where a function rank is not a constant but a
> variable. In fact these can be handled in Python -- as opposed to
> Yorick -- by remembering that a[x,y,z] is really just a shorthand for
> a[(x,y,z)] and that the index tuple can be constructed from an
> expression, but that seems more like an implementation accident than a
> feature to me.
Actually, I do think that this is a feature, but I'd be curious for a good
example of a case of this.
> More importantly, pseudo indices don't help in the least when it comes
> to reductions and general inner products that require the specification
> of an axis. Yorick does this by putting the function into the index
> list -- a clever idea, but I'd prefer to keep a clear distinction
> between functions and arguments. The current matrix module allows
> a second parameter to reduce() that indicates the axis, which is
> exactly how APL handles the problem.
I don't think that you're actually complaining about my current solution to
this problem are you?
> But both these methods are restrictive. They are sufficient only
> for reductions with functions of unbounded rank. I can give an
> example of an actual application where this is not enough: I had
> a sequence of n rotation matrices, stored in a rank-3 array D of
> shape (n, 3, 3). Then the net rotation of this sequence could
> be obtained by matrixMultiply.reduce(D). Since matrix
> multiplication is a rank-2 function, I couldn't express this
> with the current matrix module.
This is unfortunate, but unless I hear of many more of these problems, I
don't expect this to go away any time soon. It's REALLY hard to add in
function of finite rank to the Optimized FUNCtion scheme I'm currently
using. I spent a week and a half trying to make this happen and wound up
with an ugly useless mess. Since I can imagine very few cases where this
sort of behavior is important, I'm very tempted to leave ofuncs as they are
right now, only operating on unbounded functions.
> The bottom line: pseudo indices are convenient shorthands for
> the most common structural functions, but they are no replacement
> for the general function rank scheme we had in version 0.11.
You haven't convinced me of this (except in the cases of functions with
finite rank) and I have a large number of reasons to ignore them.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@mama-bear.lcs.mit.edu Thu Dec 7 18:04:03 1995
From: jjh@mama-bear.lcs.mit.edu (James Hugunin)
Date: Thu, 7 Dec 95 13:04:03 -0500
Subject: [PYTHON MATRIX-SIG] Renaming and Constructors
References: <9512041925.AA00688@feynman.rsoc.rockwell.com>
Message-ID: <9512071804.AA07317@ling-ling.lcs.mit.edu.LCS.MIT.EDU>
> 1) Names
>
> This may seem to be a minor issue, but we should tackle this before
> we all get used to the current ones.
I agree completely!
> I strongly dislike the type-specific constructors (also used for
> output). They should be IntegerMatrix, FloatMatrix, ComplexMatrix,
> CharacterMatrix, and GeneralMatrix. Anyone is free to define shorter
> names for efficient typing, if desired.
Does a FloatMatrix contain C floats or doubles (both are possible in
matrices)? Please define a complete naming scheme that can be compared to
the current (admittedly cryptic) typecodes.
Note: The current output is only being used because nobody has yet given me
a good replacement for the PrintMatrix function. I deliberately pulled
this function out into python code so that other people could do it right
for me.
> I even propose a more radical renaming. Many people associate "matrix"
> with the 2D-matrices from linear algebra. So it would be better to
> call our general objects "arrays", and leave the name "matrix" for
> linear-algebra type objects that are restricted to rank 2 and use *
> for matrix multiplication. They could be implemented in Python based
> on arrays.
The only problem that I have with array is that python already has arrays
with a known semantics. The fundamental rule in python is to never break
existing code. I know this is annoying, but the only other option would be
to make the current matrices completely backward compatible with the
existing arrays. (Which is possibly possible...)
Any other naming suggestions? (I agree that tensor is as bad as matrix)
> The pseudo-index 'None' is very confusing. An index "2" picks item
> number two from an axis, an index "All" picks all items from an
> axis. Consequently "None" should pick no item from an axis, which is
> course is a pointless operation. What it really does is create a new
> axis, so it should be named "New". "RubberIndex" is not confusing,
> but a bit strange, so it's worth thinking about alternatives. How
> about "Skip", in the sense "skip all following axes for explicit
> consideration"? For the "*" in Yorick I propose "Contract".
I have no good opinions on this issue. The obvious reason that I chose
None is that it is a built-in python object, so it is always in the current
namespace. This namespace issue is an important one. I really don't think
that we are going to be adding any of the three of these to the python core
any time soon.
One solution would be to use "new", "ellipses", and "contract" and to have
Matrix.py map these strings to objects.
Note: I picked RubberIndex in the hopes of choosing something sufficiently
long and hard to type that nobody would assume I meant it to be the final
solution. I had hoped that this might be turned into the syntax ".." one
day, but after the python workshop I doubt that this will happen any time
soon.
> 2) Constructors
>
> Currently there are two constructors, matrix() and Matrix(), with
> slightly different behaviour: matrix() takes a single argument that is
> a (possibly nested) Python sequence object, e.g. a list. Matrix()
> takes an arbitrary number of arguments that are made into a list
> before being passed to matrix(). So Matrix(1,2,3) is equivalent
> to matrix([1,2,3]).
>
> I find it confusing to have to constructors with almost the same name
> and almost the same function. I propose to have only one, of whatever
> name, which behaves like matrix(). The reason is that many matrix
> functions accept nested lists or tuples instead of matrix arguments,
> e.g. matrix([2,3]) + [5,4] works. So does matrix([2,3]) +
> matrix([5,4]), which is equivalent. But Matrix([2,3]) + [5,4] and
> Matrix([2,3]) + Matrix([5,4]) are not equivalent (well, they are in
> this simple exaple due to broadcasting, but not in general). So
> matrix() can be considered a conversion function from lists and tuples
> to arrays, which is needed anyway.
>
> I realize that the calling style of Matrix() has the advantage of
> saving one pair of brackets, but I don't consider this important
> enough to create the current confusion.
This is a real problem. I'll repeat my reasons for creating Matrix, but
otherwise, I'm willing to go along with the consensus of the list on this
one.
I was looking for the cleanest and easiest to read notation for a matrix in
the unfortunate world where I couldn't add a new bracket type to python
(okay, there aren't even any brackets to add so what am I complaining
about). I noticed that by using the variable number of arguments notation,
I got something that looked a lot more like just a new kind of brackets.
Let me know who else agrees with Konrad here.
> I guess that's enough for today ;-) I wish the lucky participants
> of the workshop much fun and hope that they don't have to eat
> spam for lunch ;-)
Had much fun, never ate SPAM - Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Thu Dec 7 19:34:11 1995
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Thu, 7 Dec 1995 14:34:11 -0500
Subject: [PYTHON MATRIX-SIG] Ranks and pseudo indices
In-Reply-To: <9512071747.AA07309@ling-ling.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Thu, 7 Dec 95 12:47:03 -0500)
Message-ID: <199512071934.OAA10658@cyclone.ERE.UMontreal.CA>
> variable. In fact these can be handled in Python -- as opposed to
> Yorick -- by remembering that a[x,y,z] is really just a shorthand for
> a[(x,y,z)] and that the index tuple can be constructed from an
> expression, but that seems more like an implementation accident than a
> feature to me.
Actually, I do think that this is a feature, but I'd be curious for a good
example of a case of this.
OK, let's make it a feature. I can't remember an example right now,
but I know that I used variables to specify ranks in J a few times.
Besides, my philosophy is the opposite: there should be good arguments
for restrictions, not for generality. Sooner or later someone will
reach the limits of everything, so you should not impose narrow limits
arbitrarily just because you can't think of any problem affected by
them.
> between functions and arguments. The current matrix module allows
> a second parameter to reduce() that indicates the axis, which is
> exactly how APL handles the problem.
I don't think that you're actually complaining about my current solution to
this problem are you?
In the sense of being restrictive, yes. Many APL users have found this
handling of reduction restrictive, which is why J's rank scheme is
generally appreciated, even by those who are put off by J's other
features.
This is unfortunate, but unless I hear of many more of these problems, I
don't expect this to go away any time soon. It's REALLY hard to add in
function of finite rank to the Optimized FUNCtion scheme I'm currently
using. I spent a week and a half trying to make this happen and wound up
I confess that I have never looked at their implementation, but I
don't see where the problem is. It is even possible to implement
the mapping from finite-rank functions to unbounded-rank functions
in Python relatively easily, but that means for-loops and the
associated loss of efficiency.
I'll see if I can figure out how the OFUNCs work.
with an ugly useless mess. Since I can imagine very few cases where this
sort of behavior is important, I'm very tempted to leave ofuncs as they are
right now, only operating on unbounded functions.
If you want more cases, try to locate an archive of comp.lang.apl of
the time when J came out. There were long discussions about the merits
of the rank system back then.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Thu Dec 7 20:14:31 1995
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Thu, 7 Dec 1995 15:14:31 -0500
Subject: [PYTHON MATRIX-SIG] Renaming and Constructors
In-Reply-To: <9512071804.AA07317@ling-ling.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Thu, 7 Dec 95 13:04:03 -0500)
Message-ID: <199512072014.PAA13383@cyclone.ERE.UMontreal.CA>
Does a FloatMatrix contain C floats or doubles (both are possible in
matrices)? Please define a complete naming scheme that can be compared to
Really? I didn't know that. Anyway, "float" should mean what it means
in normal Python, i.e. C double. C floats could be called
"ShortFloat", for example.
As for the other typecodes, I am not sure what they all mean (is
there a list somewhere?). But let's start:
'c' Character
'u' Unsigned (is that really useful?)
'1' ??? (maybe Boolean?)
's' String
'i' Integer
'l' LongInteger
'f' ShortFloat
'd' Float
'F' ??? (maybe ShortComplex?)
'D' Complex
'O' General
The only problem that I have with array is that python already has arrays
with a known semantics. The fundamental rule in python is to never break
existing code. I know this is annoying, but the only other option would be
to make the current matrices completely backward compatible with the
existing arrays. (Which is possibly possible...)
So that means we can not call any module "array", but we can still
use the name "array" within the modules. A suggestions:
- "arrays_and_matrices" for the C module that nobody imports
directly anyway.
- "Array" for the Python wrapper with array semantics; "Array"
in the names of the constructors.
- "Matrix" for the Python wrapper with 2D-matrix semantics; "Matrix"
in the names of the constructors.
That leaves the possibility of confusion between "array" and "Array",
but I can live with that. I doubt the current Python arrays will
be attractive to anyone who has the new module.
namespace. This namespace issue is an important one. I really don't think
that we are going to be adding any of the three of these to the python core
any time soon.
They don't have to be. They just have to be in the namespace of
the modules "Array" and "Matrix".
> I guess that's enough for today ;-) I wish the lucky participants
> of the workshop much fun and hope that they don't have to eat
> spam for lunch ;-)
Had much fun, never ate SPAM - Jim
That's cheating ;-)
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From chris.chase@jhuapl.edu Fri Dec 8 20:59:02 1995
From: chris.chase@jhuapl.edu (Chris Chase S1A)
Date: Fri, 8 Dec 1995 15:59:02 -0500
Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available
In-Reply-To: <9512012339.AA04760@ling-ling.lcs.mit.edu.LCS.MIT.EDU>
References: <9512012339.AA04760@ling-ling.lcs.mit.edu.LCS.MIT.EDU>
Message-ID: <199512082050.PAA00755@python.org>
James> The NEWS for this release follows. The two major incompatible
James> changes at the very beginning are why this release took so long
James> (that and the fact that I have a few other demands on my time).
James> Please let me know what you think of these changes!
James> As before, don't hesitate to send me your gripes, your bugs,
James> and your praise (if ever merited). However, I'll be gone for
James> most of next week at the python workshop, so I might be slow in
James> replying.
James> -Jim
James> Major incompatible changes (I expect that these might elict some
James> interesting discussion):
James> 1) Matrix indexing now returns by reference to the original
James> data, not by value. As a side effect of this change,
James> arbitrary sequences are not allowed in multidimensional
James> indices, but only single indices, slices, RubberIndex and
James> None.
I personally prefer general product indexing that allow
multidimensional indexes. These could be added to the reference
version of indexing but they would require extra baggage by retaining
a copy of the index vector. This would not be very efficient but is
not a reason to eliminate it since it would not affect the efficiency
of the other types of indexes.
Are references better than copying for indexing?
I can see that a speed increase is natural when using only references
rather than creating a copy. But are references always efficient? If
a multidimensional slice reference is passed as an argument to another
routine that accesses the object multiple times then it may end up
being less efficient. To avoid this would require that these routines
make contiguous copies of these types of arguments (whether or not the
copy is needed since the reference nature of the object is transparent
to the user). Similarly a copy to contiguous storage would
need to be done before passing a reference to an external FORTRAN or C
routine.
On the other hand, I can see that references can save memory for very
large matrix objects. Additionally, a copy can be performed at user
discretion.
I would think that assignment to references that change the originally
indexed object may lead to some surprises for some users used to the
implementations of other matrix languages like Matlab and IDL.
I am not an expert on this. What are the other pros and cons about
references versus copies being returned by indexing?
James> Note: there is a new method, take, which will allow you to
James> index a matrix with an arbitrary sequence as before.
This then requires a sequence of take() plus [] indexing to obtain
general product indexing.
How does one do assignment using a multi-dimensional index vector?
James> This change was motivated both by the possible speed increases
James> (about 40% for some code) and more importantly by issues of
James> clarity of expression. I couldn't come up with any other way
James> to make the following hold:
James> m[0][1] is an efficient way to index a multi-dimensional array.
James> m[0,1] == m[0][1]
James> m[0,] == m[0]
It will be difficult to maintain a natural connection between
hierarchical index using [][] versus multi-dimensional indexing. For
example:
m[0:4,1] != m[0:4][1]
James> 2) No more ranks of operators, instead Yorick style pseudo
James> indices are used. As a side effect of this, outer product
James> is now a convenience function rather than a method on
James> ofuncs.
James> For functions of unbounded rank (currently the only kind of
James> function supported by my omathmodule) Yorick pseudo indices
James> support a superset of what was possible to express using ranks
James> and outer products. There is no fundamental reason not to have
James> both pseudo indices and ranks, I just think that it's
James> conceptually cleaner not to mix the two up.
Function ranks offer more generality than what can be done with pseudo
indices. I would prefer that the matrix module keep function ranks
rather than limit the full generality of the object. Although I do
not what it takes to add it cleanly. I have not had time to look
deeply into your code other than browsing the matrixobject.c (the
current state of the code made for rather dense reading).
James> 1) This new release is about 40% faster than 0.11 (on a
James> benchmark code I was given by the folks at LBL).
What was the benchmark?
>> 1) Names
>>
>> This may seem to be a minor issue, but we should tackle this before
>> we all get used to the current ones.
James> I agree completely!
>> I strongly dislike the type-specific constructors (also used for
>> output). They should be IntegerMatrix, FloatMatrix, ComplexMatrix,
>> CharacterMatrix, and GeneralMatrix. Anyone is free to define shorter
>> names for efficient typing, if desired.
James> Does a FloatMatrix contain C floats or doubles (both are
James> possible in matrices)? Please define a complete naming scheme
James> that can be compared to the current (admittedly cryptic)
James> typecodes.
Perhaps a naming scheme similar to FORTRAN's INTEGER*8, REAL*4 REAL*8.
The advantage of this is that the precision is known. With ANSI C the
precision of floats and doubles is not specified. ANSI C only
enforces a minimum precision on an implementation.
>> I even propose a more radical renaming. Many people associate "matrix"
>> with the 2D-matrices from linear algebra. So it would be better to
>> call our general objects "arrays", and leave the name "matrix" for
>> linear-algebra type objects that are restricted to rank 2 and use *
>> for matrix multiplication. They could be implemented in Python based
>> on arrays.
I would prefer the term "array" with a specialized "matrix" module
providing linear algebra matrix functions and a matrix multiplication
operator that take 2D arrays as arguments. But it was pointed out
that we can not use "array" as it is already a different module in
Python. I don't suppose the module produced by this SIG could
completely replace the current array module (without breadking its old
behavior)?
James> Note: I picked RubberIndex in the hopes of choosing something
James> sufficiently long and hard to type that nobody would assume I
James> meant it to be the final solution. I had hoped that this might
James> be turned into the syntax ".." one day, but after the python
James> workshop I doubt that this will happen any time soon.
What happened at the workshop to rule out the ".." or "*" syntax for
rubber indexes?
Chris
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Fri Dec 8 23:16:02 1995
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Fri, 8 Dec 1995 18:16:02 -0500
Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available
In-Reply-To: <199512082050.PAA00755@python.org> (message from Chris Chase S1A on Fri, 8 Dec 1995 15:59:02 -0500)
Message-ID: <199512082316.SAA05920@cyclone.ERE.UMontreal.CA>
It will be difficult to maintain a natural connection between
hierarchical index using [][] versus multi-dimensional indexing. For
example:
m[0:4,1] != m[0:4][1]
I agree, and we shouldn't insist on this as a feature. It might even
be better (in terms of error checking) to forbid an index that doesn't
cover all dimensions. Right now, a[0] is just a shorthand for
a[0, RubberIndex], and I don't think this shorthand is necessary.
Perhaps a naming scheme similar to FORTRAN's INTEGER*8, REAL*4 REAL*8.
The advantage of this is that the precision is known. With ANSI C the
precision of floats and doubles is not specified. ANSI C only
enforces a minimum precision on an implementation.
And since Python is written in C, we can't promise anything more.
Besides, giving a single number still doesn't specify the precision
of a floating point number. The memory space could be distributed
differently between mantissa and exponent.
My major concern is that array names correspond with the standard
Python number type names as far as possible. Anything else just
generates confusion.
operator that take 2D arrays as arguments. But it was pointed out
that we can not use "array" as it is already a different module in
As I pointed out before, it is only the module name which may not
be "array". Even "Array" is OK.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dubois1@llnl.gov Sat Dec 9 00:05:54 1995
From: dubois1@llnl.gov (Paul. Dubois)
Date: Fri, 08 Dec 1995 16:05:54 -0800
Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available
References: <199512082316.SAA05920@cyclone.ERE.UMontreal.CA>
Message-ID: <30C8D2E2.6190@llnl.gov>
A followup to the remarks about precision.
In fact, Fortran 90 goes to considerable lengths to try to improve this
situation but it isn't REAL*8 that is the solution. Fortran 90 uses
something called a Kind specifier, and Kind specifiers are calculable
from desired characteristics at compile time. Actually knowing that
something is a certain number of bits doesn't tell you what you need to
know, as there is a tradeoff between precision and dynamic range.
Anyway, you end up saying REAL(kind) x, y, z.
I believe Jim has made fundamentally the right decision in deciding that
the abstraction he is after is "things that store a homogenous bunch of
stuff in contiguous memory". This would be a parameterized type in other
languages, such as Eiffel's ARRAY[T], where one actually instantiates
ARRAY[DOUBLE], ARRAY[INTEGER], ARRAY[CHARACTER], ARRAY[TELEPHONE_BOOK],
etc. Since a large use of this abstraction is connection to C and
Fortran libraries, one has to be precise about it in a cross-language
way and I think the present proposal is correct. The question of what
language must be used to express this at the Python level is an
interesting one. While I favor names like IntegerMatrix, ..., reserving
Matrix to be a "smart" operator that decides on the type from the
arguments, there is a place for a version with a calculable type or type
code, so that one can get different things on different platforms with
the same source by doing tests on the system type.
I think Array is a better name than Matrix and if 'array' weren't
already taken would have no hesitation in going ahead. If we picked
Array as the name that is now Matrix, what would we call that which is
now 'matrix'? ArrayFromSequence ?
--
Paul F. Dubois, L-472 (510)-422-5426
Lawrence Livermore National Laboratory FAX (510)-423-9969
Livermore, CA 94550 dubois1@llnl.gov
Consulting: PaulDubois@aol.com
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Sat Dec 9 16:05:43 1995
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Sat, 9 Dec 1995 11:05:43 -0500
Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available
In-Reply-To: <30C8D2E2.6190@llnl.gov> (dubois1@llnl.gov)
Message-ID: <199512091605.LAA12973@cyclone.ERE.UMontreal.CA>
interesting one. While I favor names like IntegerMatrix, ..., reserving
Matrix to be a "smart" operator that decides on the type from the
arguments, there is a place for a version with a calculable type or type
code, so that one can get different things on different platforms with
the same source by doing tests on the system type.
The "smart" version could take an optional argument indicating the type
somehow. In addition to the typecodes, it would be nice if it would
accept the return values of the builtin function type() . Then you
could write something like
list = [1,3,2]
Array(list, type(list[0]))
The typecode will still be needed for the types that have no Python
equivalents, like C-floats.
I think Array is a better name than Matrix and if 'array' weren't
already taken would have no hesitation in going ahead. If we picked
Once more, it is only the module name "array" that we can't use. Nothing
else is global in Python.
Array as the name that is now Matrix, what would we call that which is
now 'matrix'? ArrayFromSequence ?
I still propose that "Array" should behave like "matrix" does now.
That way we need only one constructor.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Sat Dec 9 16:05:43 1995
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Sat, 9 Dec 1995 11:05:43 -0500
Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available
In-Reply-To: <30C8D2E2.6190@llnl.gov> (dubois1@llnl.gov)
Message-ID: <199512091605.LAA12973@cyclone.ERE.UMontreal.CA>
interesting one. While I favor names like IntegerMatrix, ..., reserving
Matrix to be a "smart" operator that decides on the type from the
arguments, there is a place for a version with a calculable type or type
code, so that one can get different things on different platforms with
the same source by doing tests on the system type.
The "smart" version could take an optional argument indicating the type
somehow. In addition to the typecodes, it would be nice if it would
accept the return values of the builtin function type() . Then you
could write something like
list = [1,3,2]
Array(list, type(list[0]))
The typecode will still be needed for the types that have no Python
equivalents, like C-floats.
I think Array is a better name than Matrix and if 'array' weren't
already taken would have no hesitation in going ahead. If we picked
Once more, it is only the module name "array" that we can't use. Nothing
else is global in Python.
Array as the name that is now Matrix, what would we call that which is
now 'matrix'? ArrayFromSequence ?
I still propose that "Array" should behave like "matrix" does now.
That way we need only one constructor.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From da@maigret.cog.brown.edu Sat Dec 9 21:12:04 1995
From: da@maigret.cog.brown.edu (David Ascher)
Date: Sat, 9 Dec 1995 16:12:04 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] Display of 1 & 2D matrices?
In-Reply-To: <199510021627.MAA14222@cyclone.ERE.UMontreal.CA> from "Hinsen Konrad" at Oct 2, 95 12:27:22 pm
Message-ID: <199512092112.QAA05656@maigret>
Pardon this digression please:
Has anyone started working on display modules for 1 and 2D matrices
using any of the gui systems available for Python (Tk-based, X-based,
whatever-based)?
--david
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From da@maigret.cog.brown.edu Sat Dec 9 21:12:04 1995
From: da@maigret.cog.brown.edu (David Ascher)
Date: Sat, 9 Dec 1995 16:12:04 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] Display of 1 & 2D matrices?
In-Reply-To: <199510021627.MAA14222@cyclone.ERE.UMontreal.CA> from "Hinsen Konrad" at Oct 2, 95 12:27:22 pm
Message-ID: <199512092112.QAA05656@maigret>
Pardon this digression please:
Has anyone started working on display modules for 1 and 2D matrices
using any of the gui systems available for Python (Tk-based, X-based,
whatever-based)?
--david
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Mon Dec 11 13:39:43 1995
From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey)
Date: Mon, 11 Dec 1995 08:39:43 -0500
Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available
In-Reply-To: "Paul. Dubois" <dubois1@llnl.gov>
"Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available" (Dec 8, 4:05pm)
References: <199512082316.SAA05920@cyclone.ERE.UMontreal.CA>
<30C8D2E2.6190@llnl.gov>
Message-ID: <9512110839.ZM26537@dsdbqvarsa.er.usgs.gov>
One minor note on these naming discussions.
It is necessary for names to be unique without regard to case. That is,
although Python is case sensitive, some of the operating systems it functions
in are not, and since module names are found in the local file systems, the
names like array and Array may conflict.
Jim
On Dec 8, 4:05pm, Paul. Dubois wrote:
> Subject: Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is availabl
> A followup to the remarks about precision.
> In fact, Fortran 90 goes to considerable lengths to try to improve this
> situation but it isn't REAL*8 that is the solution. Fortran 90 uses
> something called a Kind specifier, and Kind specifiers are calculable
> from desired characteristics at compile time. Actually knowing that
> something is a certain number of bits doesn't tell you what you need to
> know, as there is a tradeoff between precision and dynamic range.
> Anyway, you end up saying REAL(kind) x, y, z.
>
> I believe Jim has made fundamentally the right decision in deciding that
> the abstraction he is after is "things that store a homogenous bunch of
> stuff in contiguous memory". This would be a parameterized type in other
> languages, such as Eiffel's ARRAY[T], where one actually instantiates
> ARRAY[DOUBLE], ARRAY[INTEGER], ARRAY[CHARACTER], ARRAY[TELEPHONE_BOOK],
> etc. Since a large use of this abstraction is connection to C and
> Fortran libraries, one has to be precise about it in a cross-language
> way and I think the present proposal is correct. The question of what
> language must be used to express this at the Python level is an
> interesting one. While I favor names like IntegerMatrix, ..., reserving
> Matrix to be a "smart" operator that decides on the type from the
> arguments, there is a place for a version with a calculable type or type
> code, so that one can get different things on different platforms with
> the same source by doing tests on the system type.
>
> I think Array is a better name than Matrix and if 'array' weren't
> already taken would have no hesitation in going ahead. If we picked
> Array as the name that is now Matrix, what would we call that which is
> now 'matrix'? ArrayFromSequence ?
>
> --
> Paul F. Dubois, L-472 (510)-422-5426
> Lawrence Livermore National Laboratory FAX (510)-423-9969
> Livermore, CA 94550 dubois1@llnl.gov
> Consulting: PaulDubois@aol.com
>
> =================
> MATRIX-SIG - SIG on Matrix Math for Python
>
> send messages to: matrix-sig@python.org
> administrivia to: matrix-sig-request@python.org
> =================
>-- End of excerpt from Paul. Dubois
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Mon Dec 11 14:32:05 1995
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Mon, 11 Dec 1995 09:32:05 -0500
Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available
In-Reply-To: <9512110839.ZM26537@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov)
Message-ID: <199512111432.JAA26952@cyclone.ERE.UMontreal.CA>
It is necessary for names to be unique without regard to case. That is,
although Python is case sensitive, some of the operating systems it functions
in are not, and since module names are found in the local file systems, the
names like array and Array may conflict.
Does that mean that Python doesn't do any clever filename mapping
on sytems that do not support case distinction and/or long file names?
I never checked, but I always assumed that it does something like
the GNU library that maps Unix filenames to short DOS-style filenames
(i.e. provide a mapping that is as unique as possible, even if that
produces strange-looking filenames).
So, in effect, Python modules names are restricted to 8 lower-case
characters? That would create a few problems with the Python code I
have produced so far...
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From et@appl-math.tu-muenchen.de Mon Dec 11 14:11:45 1995
From: et@appl-math.tu-muenchen.de (Thomas Schwaller)
Date: Mon, 11 Dec 1995 15:11:45 +0100
Subject: [PYTHON MATRIX-SIG] plmodule.c, openglmodule.c
Message-ID: <9512111511.ZM4023@piranha.appl-math.tu-muenchen.de>
Hi Matrix SIG'ers,
1) First a comment on the Array, array, matrix discussion...
What about the name "multiarray" for Array. When I see uppercase letters I
always think I can derive from that class, but for Array this would not
be the case. Just an idea, not really important.
2) I found a bug in my plmodule.c. When you use just matrices everything is ok
but when tuple or list input is transformed to matrices these are
dereferenced too early. Fixed that bug.
ftp://ftp.appl-math.tu-muenchen.de/pub/et/plmodule.c.0.3.gz
Will "try" to put it on our matrix location. Connections are nearly
impossible for me here! :-(
3) Completely reworked my OpenGl module. This was quite old, used the old
naming convention and the cgen script for producing the glmodule.
Now everything works with matrices as input, not just tuples or lists.
The module is not completely finished yet, but you can already do a whole
bunch of things with it (257 procedures up to now in the module).
For GLUT or OpenGL widgets and general information about this openGL module
check also the Info collected by David Ascher at
http://maigret.cog.brown.edu:80/python/opengl/
If you want to have a look at this preversion check:
ftp://ftp.appl-math.tu-muenchen.de/pub/et/openglmodule.c.0.1.gz
If there's enough interest in that I will certainly contribute a lot
of examples.
4) Heard about the visual toolkit C++ classes and the Tcl binding for it.
Coolest thing for OpenGl graphics at the moment (at least for me! :-))
Ken Martin (one of the authors) agreed to to a python binding with me.
The Tcl binding works well (I already did some wrappers for Python
`a la Tkinter, but we will do a native Python binding) and at the moment
he is doing a Java binding. To map the quite complex data hierarchy
to python I will certainly need advice from people having done this before.
Unfortunately startup time is quite high and comiling the classes
and working with them shows the limits of C++ compiler technology
(at least the one I have, g++ and SGI CC). Using the code in the classes
directly in a Python module would be great but why developping C++ classes
you will ask. Well, ... good question ...
>>THIS WAS JUST FOR YOUR INFORMATION<<
Hope it's not boring you to death! ;_)
5) How was the feedback at the python workshop concernig the matrixmodule.
Would be nice to hear a word or two.
6) Read about the sprse matrix stuff in the paper for the workshop. Is this
making any progress or is it a long term story not been started yet.
Sorry I'm just curious!
Tom
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From fredrik_lundh@ivab.se Mon Dec 11 15:14:15 1995
From: fredrik_lundh@ivab.se (Fredrik Lundh)
Date: Mon, 11 Dec 1995 16:14:15 +0100
Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available
In-Reply-To: <199512111432.JAA26952@cyclone.ERE.UMontreal.CA>
(hinsenk@ere.umontreal.ca)
Message-ID: <9512111514.AA28909@kalle.image.ivab.se>
> Does that mean that Python doesn't do any clever filename mapping
> on sytems that do not support case distinction and/or long file names?
As with most other things, Python leaves it to the underlying system
to handle this.
Would think that MS-DOS and Windows 3.1 is the worst case, where
filenames are _silently truncated_ to fit into the 8.3 scheme. So
"import ContainerIO" will in look for "containe.py". Since my editor
does the same, I have no problems when writing modules, as long as I
make sure that the 8 first characters are unique.
This is no problem under Windows 95/NT, even on FAT file systems, nor
on Macs. Don't know about Amigas.
(Generally, I think its a bad idea to let lowercase and capitalized
versions of the same name mean different things.)
/F
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Mon Dec 11 15:16:20 1995
From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey)
Date: Mon, 11 Dec 1995 10:16:20 -0500
Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available
In-Reply-To: hinsenk@ERE.UMontreal.CA (Hinsen Konrad)
"Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available" (Dec 11, 9:32am)
References: <199512111432.JAA26952@cyclone.ERE.UMontreal.CA>
Message-ID: <9512111016.ZM27294@dsdbqvarsa.er.usgs.gov>
On Dec 11, 9:32am, Hinsen Konrad wrote:
> Subject: Re: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is availabl
>
> It is necessary for names to be unique without regard to case. That is,
> although Python is case sensitive, some of the operating systems it
functions
> in are not, and since module names are found in the local file systems,
the
> names like array and Array may conflict.
>
> Does that mean that Python doesn't do any clever filename mapping
> on sytems that do not support case distinction and/or long file names?
> I never checked, but I always assumed that it does something like
> the GNU library that maps Unix filenames to short DOS-style filenames
> (i.e. provide a mapping that is as unique as possible, even if that
> produces strange-looking filenames).
>
> So, in effect, Python modules names are restricted to 8 lower-case
> characters? That would create a few problems with the Python code I
> have produced so far...
Actually, the 8-character limit is not as serious, since:
1. For most dos/windows-3.1x programs, including python, long file names are
automatically truncated, and
2. Windows NT and Windows-95 do not have the 8-character limitation.
Still, Windows-NT and Windows 95 are both case-insensitive, and when Python is
told to import "Spam", it will happily import from Spam.py, spam.py. SPAM.PY,
...
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From mclay@eeel.nist.gov Mon Dec 11 17:57:22 1995
From: mclay@eeel.nist.gov (Michael McLay)
Date: Mon, 11 Dec 95 12:57:22 EST
Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available
In-Reply-To: <199512111432.JAA26952@cyclone.ERE.UMontreal.CA>
References: <9512110839.ZM26537@dsdbqvarsa.er.usgs.gov>
<199512111432.JAA26952@cyclone.ERE.UMontreal.CA>
Message-ID: <9512111757.AA00826@acdc.eeel.nist.gov>
Hinsen Konrad writes:
>
> It is necessary for names to be unique without regard to case. That is,
> although Python is case sensitive, some of the operating systems it functions
> in are not, and since module names are found in the local file systems, the
> names like array and Array may conflict.
>
> Does that mean that Python doesn't do any clever filename mapping
> on sytems that do not support case distinction and/or long file names?
> I never checked, but I always assumed that it does something like
> the GNU library that maps Unix filenames to short DOS-style filenames
> (i.e. provide a mapping that is as unique as possible, even if that
> produces strange-looking filenames).
>
> So, in effect, Python modules names are restricted to 8 lower-case
> characters? That would create a few problems with the Python code I
> have produced so far...
I don't think a universally agreed upon solution has ever been reached
on the filename mapping issue. It wouldn't be to hard to implement a
platform independent mechanism that allowed case sensitive names. It
could all be done by redefining the import statement so that filenames
are looked up at run time on some platforms. The filesystem on
CD-ROMs uses this technique for allowing "portable" file names.
Python might even use the same encoding rules. This will have to be
done when the PSA develops a multi-platform CD-ROM distribution
of Python.
Michael
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Mon Dec 11 23:27:45 1995
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Mon, 11 Dec 1995 18:27:45 -0500
Subject: [PYTHON MATRIX-SIG] Release 0.20 of matrix object is available
In-Reply-To: <9512111016.ZM27294@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov)
Message-ID: <199512112327.SAA05548@cyclone.ERE.UMontreal.CA>
Actually, the 8-character limit is not as serious, since:
1. For most dos/windows-3.1x programs, including python, long file names are
automatically truncated, and
Bad enough. I have lots of module names that do not differ in the
first eight characters. Not much of a problem for me, because
I am not going to use 8-character systems any more unless forced with
a gun, but a problem for Python portability in general.
Still, Windows-NT and Windows 95 are both case-insensitive, and when Python is
told to import "Spam", it will happily import from Spam.py, spam.py. SPAM.PY,
...
Another portability problem. Although in general I wouldn't use upper-
and lowercase variants of the same name together, this can happen by
accident, and then I have code that works only on some platforms.
Since Pyton uses case-sensitive names everywhere, it should try to
enforce this as much as possible also for module names.
Michael McLay on the same topic:
I don't think a universally agreed upon solution has ever been reached
on the filename mapping issue. It wouldn't be to hard to implement a
platform independent mechanism that allowed case sensitive names. It
could all be done by redefining the import statement so that filenames
are looked up at run time on some platforms. The filesystem on
That would be a good solution. Is there any SIG we can make
responsible for this? ;-)
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Tue Dec 12 23:38:35 1995
From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey)
Date: Tue, 12 Dec 1995 18:38:35 -0500
Subject: [PYTHON MATRIX-SIG] [comp.lang.python] Proposal for sharing C interfaces between extension modules
Message-ID: <199512122338.XAA14048@qvarsx.er.usgs.GOV>
The following message was posted on the Python list. One of the
driving reasons for this proposal is to simplify the configuration of
the matrix object implementation and the many extension modules that
will use it.
My idea is that the matrix object can be included in a module and that
the C interface to the matrix object type can be exported as a
collection of CObjects or through a single CObject that is an array
or struct (or other data structure) of C function pointers. Then
extension modules that want to use the C interface to the matrix
object can import the matrix module and use PyObject_GetAttrString to
get access to the C interface.
In this way, the matrix object need not be munged into the standard
source tree.
-- Jim
------- Forwarded Message
From: jfulton@disqvarsa.er.usgs.GOV (Jim Fulton )
Newsgroups: comp.lang.python
Subject: Proposal for sharing C interfaces between extension modules
Date: 12 Dec 1995 19:36:04 GMT
Organization: U.S. Geological Survey
NNTP-Posting-Host: disqvarsa.er.usgs.gov
Problem: It is sometimes the case that an extension module needs to
export a C interface to other extension modules. This is especially
true for modules that define useful built-in types.
If modules are dynamically linked, sharing C interfaces between
modules can get someone complicated, because the extension modules
need to get linked together and get linked with Python. How this is
done can vary significantly from system to system.
Python already provides a dynamic linking facility, via import. An
extension module can import other modules, including other extension
modules. So, for example, a Matrix module could import a complex
number module and use PyObject_GetAttrString to get any objects
(classes, functions, etc) exported by the complex module. This
mechanism is nice because it doesn't depend in any way how the module
being imported was linked. Python handles the linking for you.
Unfortunately, this currently only works with Python objects exported
by a module.
(Fortunately, if only a Python interface to a module is needed, this
mechanism allows an extension module to "link" to another module
regardless of whether the other module is a Python module or an
extension module. Cool. :)
I propose adding a very small new type to the core language, called
CObject, that is a Python wrapper for C objects. On my system, this
new type adds a total of 204 bytes to the text portion of the
interpreter. :-)
If an extension module wishes to export a C object (such as a C
function, or an array of C functions) to other extension modules, the
extension module would use the function CObject_FromVoidPtr to create
a Python object that wraps a pointer to the C object. The module
would then insert this object in the module's dictionary during module
initialization.
Here is the declaration for CObject_FromVoidPtr:
/* Create a CObject from a pointer to a C object and an optional
destrutor function. If the second argument is non-null, then
it will be called with the first argument if and when the CObject
is destroyed.
*/
extern PyObject *
CObject_FromVoidPtr Py_PROTO((void *cobj, void (*destruct)(void*)));
An extension module that wishes to use a C object exported by another
extension module, would:
- Import the other module,
- Use PyObject_GetAttrString to get the CObject, using a published
name, and
- Use CObject_AsVoidPtr to obtain a pointer to the C object,
Here is the declaration for CObject_AsVoidPtr:
/* Retrieve a pointer to a C object from a CObject. */
extern void *
CObject_AsVoidPtr Py_PROTO((PyObject *));
Of course, CObjects exported by a module can be accessed from Python,
but they provide no useful behavior that can be accessed directly from
Python scripts. They can be assigned to Python variables and passed
to extension functions or extension type methods.
This small extension to the language should significantly ease the
configuration of modules that need or want to interface with each
other via C.
--
-- Jim Fulton jfulton@usgs.gov (703) 648-5622
U.S. Geological Survey, Reston VA 22092
This message is being posted to obtain or provide technical information
relating to my duties at the U.S. Geological Survey.
------- End of Forwarded Message
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@mama-bear.lcs.mit.edu Wed Dec 13 17:51:32 1995
From: jjh@mama-bear.lcs.mit.edu (James Hugunin)
Date: Wed, 13 Dec 95 12:51:32 -0500
Subject: [PYTHON MATRIX-SIG] Re: Version 2.0
References: <9512011859.AA10898@kristen.llnl.gov>
<9512011916.AA04586@ling-ling.lcs.mit.edu.LCS.MIT.EDU>
<30BF59D4.1E33@llnl.gov>
<9512071830.AA07326@ling-ling.lcs.mit.edu.LCS.MIT.EDU>
<30C8AAE4.616E@llnl.gov>
Message-ID: <9512131751.AA09345@ling-ling.lcs.mit.edu.LCS.MIT.EDU>
Paul sent me this as private comments and I'm going to violate all rules of
Netiquette by replying to the entire group. I think that there are some
important things in here to comment on.
> a. It seems like if I don't do the import of omath then 2.*x doesn't
> work. After a from omath import * it does. (?)
This is true. You have to import omath somehow, or matrices do not have
any numeric behavior (notice that if you use import Matrix, or from Matrix
import *. this gets done for you). Here are the reasons...
1) This makes the matrixobject code a lot smaller, and therefore likelier
to be included in the base python distribution (fingers crossed)
2) This makes add.reduce(m) possible.
3) This makes it possible to have a vectorized_ofuncmodule (or something
like that) that defines high performance versions of these operators for a
particular machine, and then have them be used when you say "a+b".
> b. Matrix(1,2,3) produces a Matrix_l, or at least that is what it
> claims it is. Should be a Matrix_i?
This will be seen as a BIG issue, and I'll write more about it later (I've
a few other demands on my time right now).
It should either be Matrix_b (a matrix of bytes, because 1,2, and 3 can all
be stored in a single byte) or it should be Matrix_l (a matrix of longs
because a python integer technically holds a C long).
> c. I preferred the [1.,2.,3.] rather than the Matrix_d(...). A lot. A
> whole lot. Provide a query function for those who really want to know
> the flavor of matrix.
The printing of matrices can be GREATLY improved. I changed the
matrix_print function so that it goes out and calls a python routine to do
the actual printing. Check out the PrintMatrix function in Matrix.py.
Currently this function just returns the string representation of the object
(for which Matrix_d is a good choice modulo naming conventions).
Anybody who has good ideas about how to print a matrix can go in and modify
this python function to get whatever print behavior they like. If you come
up with a good print function, send it to me, and I'll include the best one
in the next release.
> d. I really like the way Matrix decides on a type.
Thanks.
> e. I like the idea that others have suggested of IntegerMatrix(...)
> instead of a constructor that takes a character to decide type. However,
> there are times when the latter is useful because you are COMPUTING the
> type (like choose a precision based on the system, for example) so I
> wouldn't get rid of the character method.
> The fact that Matrix() does the right thing and that everything
> coerces mitigates most of this to the margins, however.
And here I decided to respond to your message in order to stay out of the
naming issue. I like both approaches, and I'm waiting to see what things
look like after the dust has settled.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Wed Dec 13 19:04:10 1995
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Wed, 13 Dec 1995 14:04:10 -0500
Subject: [PYTHON MATRIX-SIG] Re: Version 2.0
In-Reply-To: <9512131751.AA09345@ling-ling.lcs.mit.edu.LCS.MIT.EDU> (message from James Hugunin on Wed, 13 Dec 95 12:51:32 -0500)
Message-ID: <199512131904.OAA20843@cyclone.ERE.UMontreal.CA>
This is true. You have to import omath somehow, or matrices do not have
any numeric behavior (notice that if you use import Matrix, or from Matrix
import *. this gets done for you). Here are the reasons...
If omath is essential to the module "matrix", then this module should
do the import. All it takes is
PyImport_ImportModule("omath");
somewhere in the module initialization.
> b. Matrix(1,2,3) produces a Matrix_l, or at least that is what it
> claims it is. Should be a Matrix_i?
This will be seen as a BIG issue, and I'll write more about it later (I've
a few other demands on my time right now).
It should either be Matrix_b (a matrix of bytes, because 1,2, and 3 can all
be stored in a single byte) or it should be Matrix_l (a matrix of longs
because a python integer technically holds a C long).
I'd vote for "long" ("IntegerMatrix"). The idea is that the existence
of all the types that Python normally doesn't have should be hidden
as far as possible; those who need the smaller types to save memory
should be the only ones who have to know about it.
This guarantees that code like
x = Matrix(a,b)
x[0] = c
will always work correctly as long as a, b, and c are of the same
Python type. If the "smallest possible" rule were used, then
this would show an unexpected behaviour if a=1, b=2 and c=10000.
> c. I preferred the [1.,2.,3.] rather than the Matrix_d(...). A lot. A
> whole lot. Provide a query function for those who really want to know
> the flavor of matrix.
I agree!
Anybody who has good ideas about how to print a matrix can go in and modify
this python function to get whatever print behavior they like. If you come
up with a good print function, send it to me, and I'll include the best one
in the next release.
The first contest in this SIG! ;-)
But before everyone starts coding essentially the same stuff,
let's first discuss what the output format should look like and
then search a volunteer to do the implementation.
The two esential problems to be solved are:
1) How to print arrays of rank > 2.
2) How to deal with large arrays that don't fit on the screen.
Personally I have a certain bias towards the APL output format,
which is based on the following principles:
- all elements of an array are printed using the same format,
to make them line up nicely
- there is no border decoration whatsoever, just elements
separated by spaces
- one-dimensional arays are printed in a line, two-dimensional
arrays as a sequence of lines, N-dimensional arrays as
sequences of (N-1)-dimensional arrays separated by
(N-2) blank lines
- lines longer than the terminal width (an adjustable parameter)
are wrapped to the following line with an indentation of
six characters
This format is space-saving and very readable. The only
problem is that there is no visible difference between
a scalar and a higher-dimensional array with exactly
one element, but this has not turned out to be a problem
in practice.
And here I decided to respond to your message in order to stay out of the
naming issue. I like both approaches, and I'm waiting to see what things
look like after the dust has settled.
My suggestion for constructors:
Array(sequence, typecode=None)
transforms the sequence into an array of the given
typecode; if the typecode is None, it determines
the type by itself. Typecodes can be letters or
type objects (as returned by type()).
IntegerArray(sequence)
transforms the given sequence into an array of
integers.
ShortIntegerArray(sequence)
transforms the given sequence into an array of
C-integers.
and so on for the other specific types. Output of repr() should be
IntegerArray() etc., output of str() should be the "pretty" format to
be discussed separately.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@mama-bear.lcs.mit.edu Wed Dec 13 19:20:11 1995
From: jjh@mama-bear.lcs.mit.edu (James Hugunin)
Date: Wed, 13 Dec 95 14:20:11 -0500
Subject: [PYTHON MATRIX-SIG] plmodule.c, openglmodule.c
References: <9512111511.ZM4023@piranha.appl-math.tu-muenchen.de>
Message-ID: <9512131920.AA09380@ling-ling.lcs.mit.edu.LCS.MIT.EDU>
It's great to see these modules! This object is primarily intended as a
vehicle to make it easier to extend python with these sorts of numerically
intensive libraries. The fact that writing them has gone well so far is a
great sign.
> 5) How was the feedback at the python workshop concernig the matrixmodule.
> Would be nice to hear a word or two.
Everybody seemed very positive about it. They loved the surface plot you
sent me. Many people (including Guido) seemed rather excited when I
mentioned that you were also doing an openglmodule "right" using matrices as
inputs.
A few people seemed skeptical about the speed claims, but hopefully those
of you who have been using the object believe me. Check out the main list
for a discussion of complex numbers that was motivated by the matrix
discussion at the meeting.
> 6) Read about the sprse matrix stuff in the paper for the workshop. Is this
> making any progress or is it a long term story not been started yet.
> Sorry I'm just curious!
I'm writing a sparse matrix object in python on top of the matrix object.
This is partially a proof of concept, but it is also because my speech
recognition code desperately needs sparse matrices. Currently it's good
enough for my applications. I'll try and get a rough version of this
assembled for public consumption by the 0.21 release of the matrix object.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@mama-bear.lcs.mit.edu Wed Dec 13 19:20:11 1995
From: jjh@mama-bear.lcs.mit.edu (James Hugunin)
Date: Wed, 13 Dec 95 14:20:11 -0500
Subject: [PYTHON MATRIX-SIG] plmodule.c, openglmodule.c
References: <9512111511.ZM4023@piranha.appl-math.tu-muenchen.de>
Message-ID: <9512131920.AA09380@ling-ling.lcs.mit.edu.LCS.MIT.EDU>
It's great to see these modules! This object is primarily intended as a
vehicle to make it easier to extend python with these sorts of numerically
intensive libraries. The fact that writing them has gone well so far is a
great sign.
> 5) How was the feedback at the python workshop concernig the matrixmodule.
> Would be nice to hear a word or two.
Everybody seemed very positive about it. They loved the surface plot you
sent me. Many people (including Guido) seemed rather excited when I
mentioned that you were also doing an openglmodule "right" using matrices as
inputs.
A few people seemed skeptical about the speed claims, but hopefully those
of you who have been using the object believe me. Check out the main list
for a discussion of complex numbers that was motivated by the matrix
discussion at the meeting.
> 6) Read about the sprse matrix stuff in the paper for the workshop. Is this
> making any progress or is it a long term story not been started yet.
> Sorry I'm just curious!
I'm writing a sparse matrix object in python on top of the matrix object.
This is partially a proof of concept, but it is also because my speech
recognition code desperately needs sparse matrices. Currently it's good
enough for my applications. I'll try and get a rough version of this
assembled for public consumption by the 0.21 release of the matrix object.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Wed Dec 13 23:23:46 1995
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Wed, 13 Dec 1995 18:23:46 -0500
Subject: [PYTHON MATRIX-SIG] Another naming issue
Message-ID: <199512132323.SAA07693@cyclone.ERE.UMontreal.CA>
While we are discussing names, let me point out another one that is
not perfect: omath. From the source code I know that "o" is supposed
to mean "optimized", which may well be true, but the characterising
feature from a user's point of view is generality (compared to
"math"). So how about "gmath" (general math) or "umath" (universal
math)?
As a motivation for a quick decision, I promise to make two of my
Python modules that use Xmath (no, *not* Xmas!) available as soon as
there is a definite name. One implements 3D-Vectors (in the sense of
geometrical objects, not vectors in the sense of linear algebra), the
other automatic differentiation of arbitrary functions.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Wed Dec 13 23:24:31 1995
From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey)
Date: Wed, 13 Dec 1995 18:24:31 -0500
Subject: [PYTHON MATRIX-SIG] Re: Version 2.0
In-Reply-To: James Hugunin <jjh@mama-bear.lcs.mit.edu>
"[PYTHON MATRIX-SIG] Re: Version 2.0" (Dec 13, 12:51pm)
References: <9512011859.AA10898@kristen.llnl.gov>
<9512011916.AA04586@ling-ling.lcs.mit.edu.LCS.MIT.EDU>
<30BF59D4.1E33@llnl.gov>
<9512071830.AA07326@ling-ling.lcs.mit.edu.LCS.MIT.EDU>
<30C8AAE4.616E@llnl.gov>
<9512131751.AA09345@ling-ling.lcs.mit.edu.LCS.MIT.EDU>
Message-ID: <9512131824.ZM11092@dsdbqvarsa.er.usgs.gov>
On Dec 13, 12:51pm, James Hugunin wrote:
> Subject: [PYTHON MATRIX-SIG] Re: Version 2.0
>
> Paul sent me this as private comments and I'm going to violate all rules of
> Netiquette by replying to the entire group. I think that there are some
> important things in here to comment on.
>
> > a. It seems like if I don't do the import of omath then 2.*x doesn't
> > work. After a from omath import * it does. (?)
>
> This is true. You have to import omath somehow, or matrices do not have
> any numeric behavior (notice that if you use import Matrix, or from Matrix
> import *. this gets done for you). Here are the reasons...
>
> 1) This makes the matrixobject code a lot smaller, and therefore likelier
> to be included in the base python distribution (fingers crossed)
I don't see that this makes any difference. The matrix module can and should
be in the standard "distribution", but it need not be linked statically into
the interpreter on systems that support dynamic linking, which includes most or
all of the interesting ones. If it is not linked statically into the
interpreter, but is available as a dynamically linked module, then the size is
not so critical. I don't think the space issue justifies the weird side-effect
dependencies indicated above.
>
> 2) This makes add.reduce(m) possible.
This doesn't make any sense to me. But I haven't kept up with this list
either. How does importing or not importing a module have any impact on the
amove expression or it's feasabity?
> 3) This makes it possible to have a vectorized_ofuncmodule (or something
> like that) that defines high performance versions of these operators for a
> particular machine, and then have them be used when you say "a+b".
I don't get this either. Could you elaborate.
> > b. Matrix(1,2,3) produces a Matrix_l, or at least that is what it
> > claims it is. Should be a Matrix_i?
>
> This will be seen as a BIG issue, and I'll write more about it later (I've
> a few other demands on my time right now).
>
> It should either be Matrix_b (a matrix of bytes, because 1,2, and 3 can all
> be stored in a single byte) or it should be Matrix_l (a matrix of longs
> because a python integer technically holds a C long).
>
> > c. I preferred the [1.,2.,3.] rather than the Matrix_d(...). A lot. A
> > whole lot. Provide a query function for those who really want to know
> > the flavor of matrix.
>
> The printing of matrices can be GREATLY improved. I changed the
> matrix_print function so that it goes out and calls a python routine to do
> the actual printing. Check out the PrintMatrix function in Matrix.py.
> Currently this function just returns the string representation of the object
> (for which Matrix_d is a good choice modulo naming conventions).
>
> Anybody who has good ideas about how to print a matrix can go in and modify
> this python function to get whatever print behavior they like. If you come
> up with a good print function, send it to me, and I'll include the best one
> in the next release.
>
> > d. I really like the way Matrix decides on a type.
>
> Thanks.
>
> > e. I like the idea that others have suggested of IntegerMatrix(...)
> > instead of a constructor that takes a character to decide type. However,
> > there are times when the latter is useful because you are COMPUTING the
> > type (like choose a precision based on the system, for example) so I
> > wouldn't get rid of the character method.
I agree.
> > The fact that Matrix() does the right thing and that everything
> > coerces mitigates most of this to the margins, however.
>
> And here I decided to respond to your message in order to stay out of the
> naming issue. I like both approaches, and I'm waiting to see what things
> look like after the dust has settled.
I think both approaches should be provided.
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@mama-bear.lcs.mit.edu Thu Dec 14 18:43:08 1995
From: jjh@mama-bear.lcs.mit.edu (James Hugunin)
Date: Thu, 14 Dec 95 13:43:08 -0500
Subject: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath
References: <199512132323.SAA07693@cyclone.ERE.UMontreal.CA>
Message-ID: <9512141843.AA09889@ling-ling.lcs.mit.edu.LCS.MIT.EDU>
This relates to both Konrad's and Jim's recent posts. Also, this is a bit
of a rambling set of comments rather than a coherent argument. I just want
people to understand where this weird ofunc thing came from, and what it
does now.
1) What is an Optimized FUNCtion?
An ofunc is a python object that contains C code to execute a function on
arrays of raw C numbers (or on raw python objects, but that's another story)
at close to compiled speed. Each function takes an arbitrary number of
arguments of given types, and produces an arbitrary number of results of
given types. For every desired combination of types that the function
should operate on, there is another bit of C code to do the actual
computation.
2) Why not bundle it into the matrix object?
For one thing, I'd like to eventually clean things up and document them
well enough that anybody can create an ofunc. They're really useful little
critters for doing very fast computations on arrays of numbers.
Also, there are things like generalized reductions that need to have
something to grab onto as the function to reduce over. Without ofuncs as
real python objects, I don't have any good way to express something like:
add.reduce(m)
(equivalent to but much faster than reduce(lambda x, y: x+y, m) ).
There are also many of these creatures that don't really belong with the
matrixobject, but which should behave in exactly the same way as "built-in"
functions like add. These include things like hypot.
3) Why the current weird dependency on import omath?
I wanted to have a module which defined add and subtract as ofuncs so that
they could be used in things like add.reduce.
I also wanted to be able to play with different implementations of the
ofuncs for different purposes. For example, it should be possible to do the
usual numeric tricks of special casing the currently very general inner
loops to gain speed improvements. Using the current setup, this can be done
by creating a new module, and using the ofuncs for things like add and
subtract defined in this new module in the matrix object.
Also, I noticed that the matrix object was quite useful without any math
functions at all, and I thought that it was a nice coincidence that this
arrangement made it possible to have a "stripped-down" matrixobject with no
numerical capabilites if you so desired.
4) How this turned into a generalized math package
I started to notice a certain duplication of code going on. For example,
mathmodule now had a function to take the sin of a double, cmathmodule had
one to take the sin of a complex number, and omathmodule (for these ofuncs
I'd created) had one to take the sin of a matrix of doubles, complex, or
even of generic python objects (that had a sin method). This made it a very
small jump to allow a single double or complex to be treated as a 0d
matrix, and completely replace the code in mathmodule and cmathmodule with
omathmodule.
This is what turned into the generic math module that now exists. I have
no arguments with giving it a name more fitting of its new role.
If nobody has any objections, or better ideas, I'll change the name to
"umath" (gxxxx always makes me think of gnu code) for the upcoming bug-fix
release.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Thu Dec 14 20:03:13 1995
From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey)
Date: Thu, 14 Dec 1995 15:03:13 -0500
Subject: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath
In-Reply-To: James Hugunin <jjh@mama-bear.lcs.mit.edu>
"[PYTHON MATRIX-SIG] A ramble on ofunc's and omath" (Dec 14, 1:43pm)
References: <199512132323.SAA07693@cyclone.ERE.UMontreal.CA>
<9512141843.AA09889@ling-ling.lcs.mit.edu.LCS.MIT.EDU>
Message-ID: <9512141503.ZM11806@dsdbqvarsa.er.usgs.gov>
Hm. This made be reread:
> > a. It seems like if I don't do the import of omath then 2.*x doesn't
> > work. After a from omath import * it does. (?)
> This is true. You have to import omath somehow, or matrices do not have
> any numeric behavior (notice that if you use import Matrix, or from Matrix
> import *. this gets done for you). Here are the reasons...
Let me guess: x is an object returned from a module other than the matrix
module that creates matrix objects, right? In the above example, Matrix has
not been imported yet?
Then the initialization code for Matrix causes the ofunc stuff to get wired
into the matrix type?
I'm more and more inclined to think that extension types (types not
built into the language) should always be hosted and exported by an
extension module and accessed from other modules via the import
mechanism. If this were the case, then any extension module that
created matrices would have to import the Matrix module to get access
to a matrix constructor. The Matrix module would then import other
needed modules, like omath. (See additional comment below.)
I intend to rewrite FIDL so that modules that it creates import Matrix and use
PyObject_GetAttrString to obtain matrix constructors. So if you get a
matrix from a FIDL-generated module, Matrix and therefore omath, will
have been imported and matrices will have numeric behavior.
On Dec 14, 1:43pm, James Hugunin wrote:
> Subject: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath
>
> This relates to both Konrad's and Jim's recent posts. Also, this is a bit
> of a rambling set of comments rather than a coherent argument. I just want
> people to understand where this weird ofunc thing came from, and what it
> does now.
>
> 1) What is an Optimized FUNCtion?
>
> An ofunc is a python object that contains C code to execute a function on
> arrays of raw C numbers (or on raw python objects, but that's another story)
> at close to compiled speed. Each function takes an arbitrary number of
> arguments of given types, and produces an arbitrary number of results of
> given types. For every desired combination of types that the function
> should operate on, there is another bit of C code to do the actual
> computation.
>
>
> 2) Why not bundle it into the matrix object?
>
> For one thing, I'd like to eventually clean things up and document them
> well enough that anybody can create an ofunc. They're really useful little
> critters for doing very fast computations on arrays of numbers.
>
> Also, there are things like generalized reductions that need to have
> something to grab onto as the function to reduce over. Without ofuncs as
> real python objects, I don't have any good way to express something like:
>
> add.reduce(m)
> (equivalent to but much faster than reduce(lambda x, y: x+y, m) ).
>
> There are also many of these creatures that don't really belong with the
> matrixobject, but which should behave in exactly the same way as "built-in"
> functions like add. These include things like hypot.
>
>
> 3) Why the current weird dependency on import omath?
>
> I wanted to have a module which defined add and subtract as ofuncs so that
> they could be used in things like add.reduce.
>
> I also wanted to be able to play with different implementations of the
> ofuncs for different purposes. For example, it should be possible to do the
> usual numeric tricks of special casing the currently very general inner
> loops to gain speed improvements. Using the current setup, this can be done
> by creating a new module, and using the ofuncs for things like add and
> subtract defined in this new module in the matrix object.
>
> Also, I noticed that the matrix object was quite useful without any math
> functions at all, and I thought that it was a nice coincidence that this
> arrangement made it possible to have a "stripped-down" matrixobject with no
> numerical capabilites if you so desired.
But you now have a weird interface to the matrix module. Assuming
that on most systems, the matrix module will be dynamically linked
anyway, I wouldn't worry about size. Just let the matrix module
import omath so that the user who doesn't need omath can still use
matrix math without having to be aware of your implementation decision
to use omath.
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Thu Dec 14 23:07:57 1995
From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey)
Date: Thu, 14 Dec 1995 18:07:57 -0500
Subject: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath
In-Reply-To: hinsenk@ERE.UMontreal.CA (Hinsen Konrad)
"Re: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath" (Dec 14, 5:33pm)
References: <199512142233.RAA15710@cyclone.ERE.UMontreal.CA>
Message-ID: <9512141807.ZM12630@dsdbqvarsa.er.usgs.gov>
On Dec 14, 5:33pm, Hinsen Konrad wrote:
> Subject: Re: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath
>
> I'm more and more inclined to think that extension types (types not
> built into the language) should always be hosted and exported by an
> extension module and accessed from other modules via the import
> mechanism. If this were the case, then any extension module that
>
> That is certainly preferable. However, I am not sure whether it
> is possible in all cases with the current implementation. Clearly
> there is not much provision for C modules that depend on each other.
> Your proposed CObjects solve one problem, but there may be more.
There may be, but I haven't though of any yet. If you come up with any, I'd be
happy to try to come up with a solution to them too. :-]
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Thu Dec 14 22:33:36 1995
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Thu, 14 Dec 1995 17:33:36 -0500
Subject: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath
In-Reply-To: <9512141503.ZM11806@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov)
Message-ID: <199512142233.RAA15710@cyclone.ERE.UMontreal.CA>
I'm more and more inclined to think that extension types (types not
built into the language) should always be hosted and exported by an
extension module and accessed from other modules via the import
mechanism. If this were the case, then any extension module that
That is certainly preferable. However, I am not sure whether it
is possible in all cases with the current implementation. Clearly
there is not much provision for C modules that depend on each other.
Your proposed CObjects solve one problem, but there may be more.
But you now have a weird interface to the matrix module. Assuming
that on most systems, the matrix module will be dynamically linked
anyway, I wouldn't worry about size. Just let the matrix module
import omath so that the user who doesn't need omath can still use
matrix math without having to be aware of your implementation decision
to use omath.
I agree. omath should be usable without matrix, but the reverse is not
so important.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Fri Dec 15 14:05:15 1995
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Fri, 15 Dec 1995 09:05:15 -0500
Subject: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath
In-Reply-To: <9512141807.ZM12630@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov)
Message-ID: <199512151405.JAA23256@cyclone.ERE.UMontreal.CA>
> That is certainly preferable. However, I am not sure whether it
> is possible in all cases with the current implementation. Clearly
> there is not much provision for C modules that depend on each other.
> Your proposed CObjects solve one problem, but there may be more.
There may be, but I haven't though of any yet. If you come up with any, I'd be
happy to try to come up with a solution to them too. :-]
It seems that you can do everything necessary with these CObjects. After
all, you can make all non-static variables and functions known in this
way, and then you have published everything that other statically
linked C modules have access to.
A potential problem is that another module can do whatever it wants
with the CObjects, including something wrong (like calling a variable).
This can be avoided (except for intentional misuse) by providing
a header file that decribes the public features of a module.
One could produce a tool (in Python, of course!) that produces such
a header file with the prototypes of the original module code.
It could be made so easy to use that except for an additional import
call at the beginning, another module could use all functions and
variables exactly as if they were statically linked. And then one
could pack the whole interface for a module into a single CObject,
which helps keeping a module's name space clean.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Fri Dec 15 14:40:16 1995
From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey)
Date: Fri, 15 Dec 1995 09:40:16 -0500
Subject: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath
In-Reply-To: hinsenk@ERE.UMontreal.CA (Hinsen Konrad)
"Re: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath" (Dec 15, 9:05am)
References: <199512151405.JAA23256@cyclone.ERE.UMontreal.CA>
Message-ID: <9512150940.ZM14205@dsdbqvarsa.er.usgs.gov>
On Dec 15, 9:05am, Hinsen Konrad wrote:
> Subject: Re: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath
>
> > That is certainly preferable. However, I am not sure whether it
> > is possible in all cases with the current implementation. Clearly
> > there is not much provision for C modules that depend on each other.
> > Your proposed CObjects solve one problem, but there may be more.
>
> There may be, but I haven't though of any yet. If you come up with any,
I'd be
> happy to try to come up with a solution to them too. :-]
>
> It seems that you can do everything necessary with these CObjects. After
> all, you can make all non-static variables and functions known in this
> way,
Actually, with CObjects, all of the exported C objects can be static. So
essentially, the only non-static variable in a module is the module
initialization function. This is a plus when the module is statically linked,
because you don't have to worry about global name colisions. You get the
benefits of Python's namespace mechanisms even for your C code. :-)
> and then you have published everything that other statically
> linked C modules have access to.
>
> A potential problem is that another module can do whatever it wants
> with the CObjects, including something wrong (like calling a variable).
This is a problem with C in general. Of course ...
> This can be avoided (except for intentional misuse) by providing
> a header file that decribes the public features of a module.
Absolutely.
> One could produce a tool (in Python, of course!) that produces such
> a header file with the prototypes of the original module code.
> It could be made so easy to use that except for an additional import
> call at the beginning, another module could use all functions and
> variables exactly as if they were statically linked. And then one
> could pack the whole interface for a module into a single CObject,
> which helps keeping a module's name space clean.
Yup, there are plenty of opportunities to build on and improve the basic
mechanism. Alot of this depends on how complex the C interface being exported
is.
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dubois1@llnl.gov Fri Dec 15 19:45:37 1995
From: dubois1@llnl.gov (Paul. Dubois)
Date: Fri, 15 Dec 1995 11:45:37 -0800
Subject: [PYTHON MATRIX-SIG] Re: Version 2.0
References: <199512131904.OAA20843@cyclone.ERE.UMontreal.CA>
Message-ID: <30D1D061.5045@llnl.gov>
In a previous message I asked (stupidly)
> b. Matrix(1,2,3) produces a Matrix_l, or at least that is what it
> claims it is. Should be a Matrix_i?
This question provoked a long response from Jim which taught me that I
had simply made a mistake; I didn't know that a Python integer
corresponded to a C long. If it does, Matrix_l is correct.
Sorry.
--
Paul F. Dubois, L-472 (510)-422-5426
Lawrence Livermore National Laboratory FAX (510)-423-9969
Livermore, CA 94550 dubois1@llnl.gov
Consulting: PaulDubois@aol.com
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Sat Dec 16 16:09:38 1995
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Sat, 16 Dec 1995 11:09:38 -0500
Subject: [PYTHON MATRIX-SIG] A ramble on ofunc's and omath
In-Reply-To: <9512150940.ZM14205@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov)
Message-ID: <199512161609.LAA02631@cyclone.ERE.UMontreal.CA>
Yup, there are plenty of opportunities to build on and improve the basic
mechanism. Alot of this depends on how complex the C interface being exported
is.
Variables and functions are all there is to export; anything else (typedefs,
macros etc.) would have to be exported via header files even in the case
of statically linked C programs.
If something like this is to become an official feature of Python
it would be nice to add a minimum of support to CObjects, even if that
increases the code size too 300 bytes ;-) The aim should be to make
this feature as easy to use as possible and reduce potential
problems to a minimum.
For example, one could have a standard typedef for a structure that
contains the complete C interface to a module. Plus a standard special
name for the variable that holds the C interface (perhaps "__C__"),
and an access function that looks for this variable and retrieves
the pointer. Then add conventions for header files, and the only
difference between this mechanism and acess to a statically linked
module is two lines of code: one for the import, and one to
retrieve the C interface.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From Emmanuel.Viennet@laforia.ibp.fr Sun Dec 17 22:45:18 1995
From: Emmanuel.Viennet@laforia.ibp.fr (VIENNET Emmanuel 48.06.27.97 Equipe Gallinari)
Date: Sun, 17 Dec 1995 23:45:18 +0100
Subject: [PYTHON MATRIX-SIG] bug: assignment from string
Message-ID: <199512172245.XAA15748@lpia2.ibp.fr>
While playing with some matrices of char:
-----------------------------------------------------------------------------
[~/softs/Python-1.3/Modules/]% python
Python 1.3 (Dec 17 1995) [GCC 2.5.8]
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> import Matrix
>>>
>>> m1 = Matrix.fromString( '\001\002\003\004', 4, 'c' )
>>>
>>> m2 = m1.reshape( 2, 2 )
>>>
>>> m2[1,1]
'\004'
>>>
>>> m2[1,1] = 'e'
Segmentation fault (core dumped)
[~/softs/Python-1.3/Modules/]% gdb ../python core
GDB 4.13 (i486-slackware-linux),
Copyright 1994 Free Software Foundation, Inc...
Core was generated by `python'.
#0 0x64fc4 in optimize_slices (dest_strides=0xbffff37c,
dest_dimensions=0xbffff378, dest_nd=0xbffff374, src_strides=0xbffff370,
src_dimensions=0xbffff36c, src_nd=0xbffff368, elsize=0xbffff364,
copies=0xbffff360) at matrixobject.c:127
127 if (((*dest_strides)[*dest_nd-1] == *elsize) && ((*src_strides)[*src_nd-1] == *elsize)) {
(gdb) where
#0 0x64fc4 in optimize_slices (dest_strides=0xbffff37c,
dest_dimensions=0xbffff378, dest_nd=0xbffff374, src_strides=0xbffff370,
src_dimensions=0xbffff36c, src_nd=0xbffff368, elsize=0xbffff364,
copies=0xbffff360) at matrixobject.c:127
#1 0x68949 in PyMatrix_CopyMatrix (dest=0x164080, src=0x164200)
at matrixobject.c:262
#2 0x68a21 in PyMatrix_CopyObject (dest=0x164080, src_object=0x187b20)
at matrixobject.c:278
#3 0x6a0a1 in matrix_ass_sub (self=0x164140, index=0x174600, op=0x187b20)
at matrixobject.c:673
#4 0x3531f in assign_subscript (w=0x164140, key=0x174600, v=0x187b20)
at ceval.c:2600
-----------------------------------------------------------------------------
The problem is with the string in the rhs: m2[1,1] = 'e'
(m1[1,1] = 66 works).
I suppose the bug is in matrix_ass_sub(), but I can't figure where.
Also, I wonder why can't we specify all the dimensions as arguments to
fromString(), avoiding to call reshape() ?
Emmanuel
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@mama-bear.lcs.mit.edu Mon Dec 18 20:56:36 1995
From: jjh@mama-bear.lcs.mit.edu (James Hugunin)
Date: Mon, 18 Dec 95 15:56:36 -0500
Subject: [PYTHON MATRIX-SIG] bug: assignment from string
References: <199512172245.XAA15748@lpia2.ibp.fr>
Message-ID: <9512182056.AA12077@ling-ling.lcs.mit.edu.LCS.MIT.EDU>
> (Core dump data ommitted)
As a small point, these detailed listings of the results of a core dump are
probably not at all useful to any member of this list except myself (to
whom the are EXTREMELY helpful) so it might be a better idea to just send
them directly to me and I'll fix them ASAP. However, I don't want to say
anything that will discourage people from sending in bug reports.
The other comments are great for the list.
> The problem is with the string in the rhs: m2[1,1] = 'e'
> (m1[1,1] = 66 works).
>
> I suppose the bug is in matrix_ass_sub(), but I can't figure where.
There's actually something rather strange going on here, and while it
shouldn't dump core, my current interpretation of strings says that this
should raise an exception.
Here's what's going on:
'hello' gets converted to a 5-length vector ['h', 'e', 'l', 'l', 'o']
'e' gets converted to a 1-length vector ['e']
66 is a 0-dimensional scalar
So, m[1,1] = 'e' is trying to assign a 1-length vector to a 0-dimensional
scalar, which is an exception.
I can fix this by treating 1-length strings as 0-dimensional scalars if you
think this is the appropriate choice.
Note: I NEVER use arrays of strings personally, so I'm depending on the
other people on this list to tell me how you want them to behave. I can't
imagine a use for arrays of strings except for interfacing to existing
libraries.
> Also, I wonder why can't we specify all the dimensions as arguments to
> fromString(), avoiding to call reshape() ?
This is a simple oversight on my part. I'm not convinced that the current
form of "fromString" is exactly the way it should finally be, and I'm
reluctant to churn out C-code for an interface that might change.
Nevertheless, I'll add a function to Matrix.py to fix this in the next
release.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Tue Dec 19 14:50:26 1995
From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey)
Date: Tue, 19 Dec 1995 09:50:26 -0500
Subject: [PYTHON MATRIX-SIG] bug: assignment from string
In-Reply-To: James Hugunin <jjh@mama-bear.lcs.mit.edu>
"Re: [PYTHON MATRIX-SIG] bug: assignment from string" (Dec 18, 3:56pm)
References: <199512172245.XAA15748@lpia2.ibp.fr>
<9512182056.AA12077@ling-ling.lcs.mit.edu.LCS.MIT.EDU>
Message-ID: <9512190950.ZM19947@dsdbqvarsa.er.usgs.gov>
On Dec 18, 3:56pm, James Hugunin wrote:
> Subject: Re: [PYTHON MATRIX-SIG] bug: assignment from string
>
(snip)
> The other comments are great for the list.
>
> > The problem is with the string in the rhs: m2[1,1] = 'e'
> > (m1[1,1] = 66 works).
> >
> > I suppose the bug is in matrix_ass_sub(), but I can't figure where.
>
> There's actually something rather strange going on here, and while it
> shouldn't dump core, my current interpretation of strings says that this
> should raise an exception.
>>
> Here's what's going on:
>
> 'hello' gets converted to a 5-length vector ['h', 'e', 'l', 'l', 'o']
> 'e' gets converted to a 1-length vector ['e']
> 66 is a 0-dimensional scalar
>
> So, m[1,1] = 'e' is trying to assign a 1-length vector to a 0-dimensional
> scalar, which is an exception.
>
> I can fix this by treating 1-length strings as 0-dimensional scalars if you
> think this is the appropriate choice.
Yes. This is necessary because Python does not have a character type. 1-length
strings must be used as "characters". Witness the fact that 'hello'[1] wants
to return a character, but returns a string instead.
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From stoll@atr-sw.atr.co.jp Wed Dec 20 08:32:39 1995
From: stoll@atr-sw.atr.co.jp (Perry A. Stoll)
Date: Wed, 20 Dec 1995 17:32:39 +0900
Subject: [PYTHON MATRIX-SIG] __doc__ string problems?
Message-ID: <199512200832.RAA18484@ciris21.atr-sw.atr.co.jp>
Matrix-sig'ers,
Sorry to disturb everyone. Is anyone else having a problem using __
doc__ strings since installing the Matrix module? Before rebuilding, I
thought I'd check with others on this list. Could it be due to some of
the changes in the distribution?
As a short example, the following does nothing in my python with the
Matrix library but works fine on the Mac:
def foo(x):
"""Some documentation string."""
return x
foo.__doc__
Ideas?
-Perry Stoll
Perry A. Stoll
Research Associate, CSRL Advanced Telecommunications Research
2-2 Hikaridai, Seika-cho Soraku-gun, Kyoto 619-02 JAPAN
VOICE: +81-774-95-1217 FINGER: stoll@atrwide.atr.co.jp
FAX : +81-774 95-1208 EMAIL: stoll@atr-sw.atr.co.jp
PGP 2.6 Key fingerprint = AF 56 5C D8 5F 78 BA FD 21 6E 2A 68 C4 55 9E B0
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsen@physik.rwth-aachen.de Thu Dec 21 15:02:53 1995
From: hinsen@physik.rwth-aachen.de (Konrad Hinsen)
Date: Thu, 21 Dec 1995 16:02:53 +0100
Subject: [PYTHON MATRIX-SIG] __doc__ string problems?
In-Reply-To: <199512200832.RAA18484@ciris21.atr-sw.atr.co.jp> (stoll@atr-sw.atr.co.jp)
Message-ID: <9512211502.AA06882@acds21>
Sorry to disturb everyone. Is anyone else having a problem using __
doc__ strings since installing the Matrix module? Before rebuilding, I
My fault! It has to do with the power operator patch. I have already
sent the fix to Guido, but obviously forgotten that other people on
the Matrix SIG might need it.
You have to add one line in the function get_docstring() in Python/compile.c;
I'll include the full function:
static object *
get_docstring(n)
node *n;
{
int i;
switch (TYPE(n)) {
case suite:
if (NCH(n) == 1)
return get_docstring(CHILD(n, 0));
else {
for (i = 0; i < NCH(n); i++) {
node *ch = CHILD(n, i);
if (TYPE(ch) == stmt)
return get_docstring(ch);
}
}
break;
case file_input:
for (i = 0; i < NCH(n); i++) {
node *ch = CHILD(n, i);
if (TYPE(ch) == stmt)
return get_docstring(ch);
}
break;
case stmt:
case simple_stmt:
case small_stmt:
return get_docstring(CHILD(n, 0));
case expr_stmt:
case testlist:
case test:
case and_test:
case not_test:
case comparison:
case expr:
case xor_expr:
case and_expr:
case shift_expr:
case arith_expr:
case term:
case factor:
case power:
if (NCH(n) == 1)
return get_docstring(CHILD(n, 0));
break;
case atom:
if (TYPE(CHILD(n, 0)) == STRING)
return parsestrplus(n);
break;
}
return NULL;
}
The new line is "case power:". Briefly, I had to introduce a new node
type to give the power operator a higher priority than multiplication,
and get_docstring() needs a complete list of node types in its case
statements in order to work correctly.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de Chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================