From da@maigret.cog.brown.edu Tue Jan 9 16:50:12 1996
From: da@maigret.cog.brown.edu (David Ascher)
Date: Tue, 9 Jan 1996 11:50:12 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] hello?
Message-ID: <199601091650.LAA22547@maigret>
I got a bounce from the list the last time I tried. I'll try again.
Content part of mesage: How is the doc for the matrix module
progressing? Anyone?
--david
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Fri Jan 12 22:47:43 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Fri, 12 Jan 1996 17:47:43 -0500
Subject: [PYTHON MATRIX-SIG] Fast evaluation of numerical expressions
Message-ID: <199601122247.RAA02079@cyclone.ERE.UMontreal.CA>
The following is not directly related to matrices, but I suppose most
of us are also interested in more general numerical applications of
Python.
A frequent problem with Python code for numerical applications is
speed, and the matrix module will help only in those cases where an
algorithm can be expressed in terms of matrix operations on
sufficiently large matrices. Many applications also require
the evaluation of many "simple" expressions, i.e. functions of
scalar values or small arrays.
I have thought about possibilities to speed up such calculations
without writing a special C extension module. One idea that looks
promising to me is to write a special compiler for numerical
expressions. This compiler would generate code for a simple
(and fast) stack-based expression evaluator that would be written
in C; the compiler itself could be written in Python. The
expression evaluator would not have to deal with type checks
etc., and would know the standard mathematical functions.
It could therefore be a lot faster than the general expression
evaluator in Python. In addition, the expression compiler
could be followed by an optimizer that takes care of constant
subexpressions and other things.
The compiler could most easily be constructed by defining
a new type "compiled expression" and define the usual
functions on that type. Then a user could write something
like
x = variable(types.FloatType, 0)
n = variable(types.IntType, 1)
y = variable(types.FloatType, 2)
compiled_function = sin(x**n+y)
print compiled_function(2.5, 3, 0.3)
The function "variable" would return an object of type "compiled
expression" that pushes the n-th argument to the evaluation stack. All
the other operations would just append their code to whatever code
they get in their arguments. Variable types could be restricted to
integers, floats, complex numbers, and arrays of these types. To
compensate the lack of control structures it would probably be
necessary to introduce a conditional function; loops would be replaced
by array operations.
Does this seem like a reasonable idea? And would anyone be willing
to collaborate on an implementation? Or does anyone have a better
idea to achieve fast expression evaluation?
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From guido@CNRI.Reston.VA.US Fri Jan 12 23:45:32 1996
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Fri, 12 Jan 1996 18:45:32 -0500
Subject: [PYTHON MATRIX-SIG] Fast evaluation of numerical expressions
In-Reply-To: Your message of "Fri, 12 Jan 1996 17:47:43 EST."
<199601122247.RAA02079@cyclone.ERE.UMontreal.CA>
References: <199601122247.RAA02079@cyclone.ERE.UMontreal.CA>
Message-ID: <199601122345.SAA23753@monty>
This sounds like a useful idea. Please proceed.
(I particularly like the idea of using a 'variable' data type which
can be subjected to normal operations, yielding a data structure
representing the expression structure. This could be used in a
symbolic algebra package, as well...)
--Guido van Rossum <guido@CNRI.Reston.VA.US>
URL: <http://www.python.org/~guido/>
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Sat Jan 13 14:32:56 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Sat, 13 Jan 1996 09:32:56 -0500
Subject: [PYTHON MATRIX-SIG] Fast evaluation of numerical expressions
In-Reply-To: <199601122345.SAA23753@monty> (message from Guido van Rossum on Fri, 12 Jan 1996 18:45:32 -0500)
Message-ID: <199601131432.JAA07188@cyclone.ERE.UMontreal.CA>
This sounds like a useful idea. Please proceed.
As soon as I find some spare time...
(I particularly like the idea of using a 'variable' data type which
can be subjected to normal operations, yielding a data structure
representing the expression structure. This could be used in a
symbolic algebra package, as well...)
Indeed. Symbolic algebra in Python would be neat...
A nice feature of this approach is that existing code can be compiled
without modification. Any Python function foo(x) that consists of a
linear sequence of statements (no conditionals, no loops) can be
compiled as simply as
compiled_foo = foo(variable(types.FloatType))
The tricky part will be the optimizer. I'd like to have at least
extraction of common subexpressions, which occur rather often
in numerical expressions.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Tue Jan 16 14:50:00 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Tue, 16 Jan 96 09:50:00 EST
Subject: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging
Message-ID: <9601161450.AA16343@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Hi all. I just got back in town from an extended Christmas vacation
(I also got married, so I had a good excuse for being gone so long).
Now that I'm back in the office again, I've been thinking that it's
about time that the matrix object made it out into beta release.
There are a few naming conventions and packaging decisions to be made
before this happens. Below are the conventions that I propose to use.
Once this object is released in beta form I have no intention of
changing the naming conventions without a REALLY good reason, so
please let me know your opinions now.
Every name in quotes should be considered a proposed name open for
discussion.
-----
The "NumericPython" package contains the following major pieces.
1) Konrad's numeric patches to the python core (incorporated into the
working version of python by Guido) will be required and included with
the distribution. These will be the only patches to the python core
required.
2) One C module that must be statically linked called "multiarraymodule.c"
Having this particular module statically linked will eliminate the
need for getting the CObject proposal working before release.
Use "PyArray_" as the name of the Matrix Object. This is a simple
renaming of the existing "PyMatrix_".
Use "array(sequence, typecode='d')" as the default
constructor for this new C type.
This PyArray C type will not implement automatic type-coercion (unlike the
current implementation). The reason for this is that I have decided
type-coercion is a major pain for arrays of floats (as opposed to
doubles) and I happen to use arrays of floats for most of my work. If
somebody can give me a good suggestion for how to keep automatic
type-coercion and have Matrix_f(1,2,3)*1.2 == Matrix_f(1.2,2.4,3.6)
then I might consider changing this decision. See later note on
Array.py for an alternative.
The include files "arrayobject.h", and "ofuncobject.h" will provide
the needed C interface to the array and optimized function objects.
3) Two dynamically linkable modules called "umathmodule.c", and "ieee_umathmodule.c"
These will both provide "universal" math support, providing the basic
functions of mathmodule, plus things like "greater" and
"booleanOr" for matrices, floats, complex, ints, and generic python
objects.
The basic "umath" will cause python exceptions in the event of an
overflow or divide-by-zero, etc. (this means it will be slow).
"ieee_umath" will not check the arguments, or the results of its
computations, and this should result in standard IEEE overflow, and
NaN values occuring in arrays as well as unpredictable modulo effects
for integer overflows. (this will be the fast version).
4) Two python objects, "Array.py" and "Matrix.py"
Array is essentially a python binding around the underlying C type,
and this will also provide for automatic type-coercion and will
generally assume that it is only working with arrays of type long,
double, and complex double (the three types of arrays that are
equivalent to python objects). In my initial tests of this object on
LLNL's simple benchmark, I found that the performance was only 5%
slower than using the C object directly.
Matrix will inherit almost everything from Array, however it will be
limited to 2 or fewer dimensions, and m1*m2 where m1 and m2 are
matrices will perform matrix style multiplication. If the linear
algebra people would like, other changes can be made (ie. ~m1 ==
m1.transpose(), ...). Based on the experiments with Array, the
performance penalty for this approach should be minimal.
In order to support these python objects (and others like them), two
special data members will be added, "__array__", and "__object__". If
an object has the member "__array__", then the C functions that handle
matrices will attempt to retrieve the matrix from this member when
passed in a python object. In addition, they will attempt to convert
their result to an object of class "__object__" upon return. This
means that umath.sin(Array([0, pi/2, pi])) == Array([0.,1.,0.]).
Hopefully, this convention will allow these python objects to coexist
well with any numeric libraries.
5) A standard library "Numeric.py" which will be the standard way of
importing multiarray, Array, Matrix, umath, etc. It will include the
inverted trig functions ("sec", "csc", "arcsec", ...) as well as most
of the standard functions currently in Matrix.py
6) Great documentation and tutorials (hopefully written by Paul
DuBois).
7) A standard test suite.
8) A new version of pickle.py which is aware of matrix objects. This
will be suggested as a replacement for the existing pickle.py. Matrix
objects are pickled using a binary format that is endian-aware, so
pickling a matrix object is a very efficient and portable way of both
storing them and sending them around the network.
9?) A "numericmodule.c" which contains reasonably efficient fft,
matrix inversion, convolution, random numbers, eigenvalues and
filtering functions stolen from existing C code so that the package
can be viewed as a "complete" numerical computing system?
Well, that's all I can think of for now, let me know your opinions
before I start my final burst of coding and get this thing polished up
and released.
Planned schedule:
Comments on this proposal until 1/22
Final alpha release 0.30 1/26
Massive use of release 0.30, and lots of good bug reports from users
Bugfixed alpha release 0.31 2/2
More testing, and hopefully final forms of documentation and tutorials
First beta release 1.0beta1 announced to general newsgroup 2/12
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From guido@CNRI.Reston.VA.US Tue Jan 16 15:19:28 1996
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Tue, 16 Jan 1996 10:19:28 -0500
Subject: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging
In-Reply-To: Your message of "Tue, 16 Jan 1996 09:50:00 EST."
<9601161450.AA16343@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
References: <9601161450.AA16343@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <199601161519.KAA05252@monty>
Looks good!
No comments here, except:
> 1) Konrad's numeric patches to the python core (incorporated into the
> working version of python by Guido) will be required and included with
> the distribution. These will be the only patches to the python core
> required.
I changed one thing in Konrad's final patches: I took out support for
'i' and 'I' to indicate imaginary constants -- you have to use 'j' or
'J'. I don't like to offer gratuitous choices (the lower/upper case
equivalency is a C legacy that I keep for consistency with other
Python numerical constants) and I expect that engineers will whine
bitterly if I chose 'i' while mathematicians will accept 'j' without
complaints... (Note that unless you never read code written by
someone else, you must learn both alternatives if they are both
present in the language. This is my reason for wanting to offer only
one option -- not the savings in instruction size or cycles.)
> 2) One C module that must be statically linked called "multiarraymodule.c"
>
> Having this particular module statically linked will eliminate the
> need for getting the CObject proposal working before release.
I do have a working version of CObject which you can use if you like.
It's up to you.
--Guido van Rossum <guido@CNRI.Reston.VA.US>
URL: <http://www.python.org/~guido/>
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Tue Jan 16 16:31:56 1996
From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey)
Date: Tue, 16 Jan 1996 11:31:56 -0500
Subject: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging
In-Reply-To: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
"[PYTHON MATRIX-SIG] Final matrix object renaming and packaging" (Jan 16, 9:50am)
References: <9601161450.AA16343@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <9601161131.ZM4105@dsdbqvarsa.er.usgs.gov>
On Jan 16, 9:50am, James Hugunin wrote:
> Subject: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging
>
> Hi all. I just got back in town from an extended Christmas vacation
> (I also got married, so I had a good excuse for being gone so long).
Congradulations!
> Now that I'm back in the office again, I've been thinking that it's
> about time that the matrix object made it out into beta release.
>
> There are a few naming conventions and packaging decisions to be made
> before this happens. Below are the conventions that I propose to use.
> Once this object is released in beta form I have no intention of
> changing the naming conventions without a REALLY good reason, so
> please let me know your opinions now.
Fair enough.
>
> Every name in quotes should be considered a proposed name open for
> discussion.
>
> -----
>
> The "NumericPython" package contains the following major pieces.
>
> 1) Konrad's numeric patches to the python core (incorporated into the
> working version of python by Guido) will be required and included with
> the distribution. These will be the only patches to the python core
> required.
>
>
> 2) One C module that must be statically linked called "multiarraymodule.c"
By statically linked, I assume you mean statically linked into the interpreter.
Right?
> Having this particular module statically linked will eliminate the
> need for getting the CObject proposal working before release.
As Guido said, the CObject proposal is working. I'll send it to you in a
separate note.
I finished the (very small) CObject implementation very soon after the
workshop because I feel it is important to use it for the Matrix (Numeric)
module.
I feel strongly that the Matrix module should export it's C interface
using CObjects so that modules using matrices to not require that any
of the matrix software be statically linked. In my distribution of
Python, I statically link as little as possible to keep the
interpreter and the interpreter start-up time small. I
plan to modify FIDL to use the CObject-exported interface.
I'd be happy to assist you with this if you wish.
>
> Use "PyArray_" as the name of the Matrix Object. This is a simple
> renaming of the existing "PyMatrix_".
>
> Use "array(sequence, typecode='d')" as the default
> constructor for this new C type.
Is this a replacement for the existing array type?
> This PyArray C type will not implement automatic type-coercion (unlike the
> current implementation). The reason for this is that I have decided
> type-coercion is a major pain for arrays of floats (as opposed to
> doubles) and I happen to use arrays of floats for most of my work. If
> somebody can give me a good suggestion for how to keep automatic
> type-coercion and have Matrix_f(1,2,3)*1.2 == Matrix_f(1.2,2.4,3.6)
> then I might consider changing this decision. See later note on
> Array.py for an alternative.
>
> The include files "arrayobject.h", and "ofuncobject.h" will provide
> the needed C interface to the array and optimized function objects.
>
>
> 3) Two dynamically linkable modules called "umathmodule.c", and
"ieee_umathmodule.c"
> These will both provide "universal" math support, providing the basic
> functions of mathmodule, plus things like "greater" and
> "booleanOr" for matrices, floats, complex, ints, and generic python
> objects.
>
> The basic "umath" will cause python exceptions in the event of an
> overflow or divide-by-zero, etc. (this means it will be slow).
> "ieee_umath" will not check the arguments, or the results of its
> computations, and this should result in standard IEEE overflow, and
> NaN values occuring in arrays as well as unpredictable modulo effects
> for integer overflows. (this will be the fast version).
>
>
> 4) Two python objects, "Array.py" and "Matrix.py"
Are these imported by Numeric, or would the user be importing these?
Is the current built-in array module going away? If not, then there
is a name conflict on case-insensitive file systems. I like the idea
of having the user import Numeric and then use the
Numeric.Matrix_d(...) or Numeric.Array_f(...) rather than than
importing Matrix and Array. Is there any reason for the user to
import Array directly? I am strongly opposed to using "Array" unless
the "array" module goes away.
>
> Array is essentially a python binding around the underlying C type,
> and this will also provide for automatic type-coercion and will
> generally assume that it is only working with arrays of type long,
> double, and complex double (the three types of arrays that are
> equivalent to python objects). In my initial tests of this object on
> LLNL's simple benchmark, I found that the performance was only 5%
> slower than using the C object directly.
>
> Matrix will inherit almost everything from Array, however it will be
> limited to 2 or fewer dimensions, and m1*m2 where m1 and m2 are
> matrices will perform matrix style multiplication. If the linear
> algebra people would like, other changes can be made (ie. ~m1 ==
> m1.transpose(), ...). Based on the experiments with Array, the
> performance penalty for this approach should be minimal.
>
> In order to support these python objects (and others like them), two
> special data members will be added, "__array__", and "__object__". If
> an object has the member "__array__", then the C functions that handle
> matrices will attempt to retrieve the matrix from this member when
> passed in a python object.
Are we taking about python members or C structure members? Is the
__array__ member supposed to be the C pointer to a block of memory?
> In addition, they will attempt to convert
> their result to an object of class "__object__" upon return.
Class __object__? So __object__ is a pointer to a Python class
object? Or is it a type code? How does the function manage to do
this? Presumably the function needs to call some sort of constructor
function. How does it do this? Is this explained in the header
files?
> This
> means that umath.sin(Array([0, pi/2, pi])) == Array([0.,1.,0.]).
OK. This makes sense
> Hopefully, this convention will allow these python objects to coexist
> well with any numeric libraries.
Could you provide some additional details?
> 5) A standard library "Numeric.py" which will be the standard way of
> importing multiarray, Array, Matrix, umath, etc. It will include the
> inverted trig functions ("sec", "csc", "arcsec", ...) as well as most
> of the standard functions currently in Matrix.py
So someone who wants to create a matrix object will do something like:
import Numeric
m=Numeric.Matrix_d(...)
Right? I like it. :-)
Did you keep a "Matrix" constructor that has a type code as an argument?
>
>
> 6) Great documentation and tutorials (hopefully written by Paul
> DuBois).
Wat cool. Will we also get doc strings?
>
> 7) A standard test suite.
>
>
> 8) A new version of pickle.py which is aware of matrix objects. This
> will be suggested as a replacement for the existing pickle.py. Matrix
> objects are pickled using a binary format that is endian-aware, so
> pickling a matrix object is a very efficient and portable way of both
> storing them and sending them around the network.
>
>
> 9?) A "numericmodule.c" which contains reasonably efficient fft,
> matrix inversion, convolution, random numbers, eigenvalues and
> filtering functions stolen from existing C code so that the package
> can be viewed as a "complete" numerical computing system?
>
>
> Well, that's all I can think of for now, let me know your opinions
> before I start my final burst of coding and get this thing polished up
> and released.
>
> Planned schedule:
>
> Comments on this proposal until 1/22
> Final alpha release 0.30 1/26
> Massive use of release 0.30, and lots of good bug reports from users
> Bugfixed alpha release 0.31 2/2
> More testing, and hopefully final forms of documentation and tutorials
> First beta release 1.0beta1 announced to general newsgroup 2/12
Thanks for all the great work.
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Jan 16 17:10:28 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Tue, 16 Jan 1996 12:10:28 -0500
Subject: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging
In-Reply-To: <9601161450.AA16343@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199601161710.MAA07614@cyclone.ERE.UMontreal.CA>
Hi all. I just got back in town from an extended Christmas vacation
(I also got married, so I had a good excuse for being gone so long).
Congratulations!
1) Konrad's numeric patches to the python core (incorporated into the
working version of python by Guido) will be required and included with
the distribution. These will be the only patches to the python core
required.
There has been a slight modification in the meantime (there will be
only 'j' for imaginary constants, not 'i') and two corrections that
perhaps not everybody has received. I'll prepare a new set of patch
files for the beta release.
This PyArray C type will not implement automatic type-coercion (unlike the
current implementation). The reason for this is that I have decided
type-coercion is a major pain for arrays of floats (as opposed to
doubles) and I happen to use arrays of floats for most of my work. If
somebody can give me a good suggestion for how to keep automatic
type-coercion and have Matrix_f(1,2,3)*1.2 == Matrix_f(1.2,2.4,3.6)
then I might consider changing this decision. See later note on
Array.py for an alternative.
One way to solve your "float" problem would be to write
Matrix_f(1,2,3)*Matrix_f(1.2),
i.e. cast 1.2 explicitly to float. I don't see how you can avoid
that cast anyway; without type coercion, Matrix_f(1,2,3)*1.2 would
be an error.
I'd prefer to have type coercion everywhere for consistency. If the
"high-level" classes Array and Matrix are really fast enough (about
which I have doubts, see below), then it doesn't matter that much, but
in general I'd give top priority to consistency and put the burden of
more complicated coding on those who want top speed (which of course
must be possible).
3) Two dynamically linkable modules called "umathmodule.c", and "ieee_umathmodule.c"
I don't think that using "ieee" in the name is a good idea. After all,
there is no guarantee that this will really use IEEE arithmetic (a
Vax, for example, doesn't have IEEE arithmetic hardware). Why not
just "fast_umath"?
4) Two python objects, "Array.py" and "Matrix.py"
Array is essentially a python binding around the underlying C type,
and this will also provide for automatic type-coercion and will
generally assume that it is only working with arrays of type long,
double, and complex double (the three types of arrays that are
equivalent to python objects). In my initial tests of this object on
LLNL's simple benchmark, I found that the performance was only 5%
slower than using the C object directly.
Nevertheless the Python wrapper could cause a significant overhead
for small arrays, which will also be used heavily. For example,
I am planning to change my vector class (3d vectors for geometry
etc.) to use an array of length 3 as its internal representation.
Do you have any data on the speed penalty for such small arrays?
Matrix will inherit almost everything from Array, however it will be
limited to 2 or fewer dimensions, and m1*m2 where m1 and m2 are
matrices will perform matrix style multiplication. If the linear
algebra people would like, other changes can be made (ie. ~m1 ==
m1.transpose(), ...). Based on the experiments with Array, the
performance penalty for this approach should be minimal.
I suppose some discussion will be necessary about the details of
this class, which until now didn't exist. Given that many people
seem to prefer a distinction between row and column vectors, it
would be a good idea to include separate constructors for them.
5) A standard library "Numeric.py" which will be the standard way of
importing multiarray, Array, Matrix, umath, etc. It will include the
inverted trig functions ("sec", "csc", "arcsec", ...) as well as most
of the standard functions currently in Matrix.py
I am not sure it is a good idea to have such an all-encompassing
module. It would contain a rather arbitrary selection of things,
i.e. those which exist at the time it is introduced. I expect
that other numerical modules will appear later. So it would be
better to group things according to functions or applications.
9?) A "numericmodule.c" which contains reasonably efficient fft,
matrix inversion, convolution, random numbers, eigenvalues and
filtering functions stolen from existing C code so that the package
can be viewed as a "complete" numerical computing system?
Again this should be several modules: linear algebra, random
numbers, fft/convolution. We won't achieve anything "complete"
anyway, so let's not pretend we do.
Otherwise, your proposal sounds fine to me.
Konrad.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Jan 16 17:17:41 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Tue, 16 Jan 1996 12:17:41 -0500
Subject: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging
In-Reply-To: <9601161131.ZM4105@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov)
Message-ID: <199601161717.MAA08000@cyclone.ERE.UMontreal.CA>
I feel strongly that the Matrix module should export it's C interface
using CObjects so that modules using matrices to not require that any
of the matrix software be statically linked. In my distribution of
I agree in principle, but that doesn't have to happen in the
first beta release.
Are these imported by Numeric, or would the user be importing these?
Is the current built-in array module going away? If not, then there
is a name conflict on case-insensitive file systems. I like the idea
Why? The current "array" in a C module, so "import Array" will
never load it, independent of the file name conventions. Which
means that "import Array" will always obtain an external module,
and as long as ours is the only one, it hardly matters if it
is called "Array.py" or "array.py".
importing Matrix and Array. Is there any reason for the user to
import Array directly? I am strongly opposed to using "Array" unless
Certainly, e.g. if the user doesn't want anything else. Besides, as I
pointed out in my last mail, a general "Numeric" module is not such a
terrific idea.
So someone who wants to create a matrix object will do something like:
import Numeric
m=Numeric.Matrix_d(...)
Speaking about constructors... I still prefer (strongly) more
meaningful names, like IntegerMatrix. Noone should be forced
to learn these silly type codes for standard applications.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Tue Jan 16 17:39:17 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Tue, 16 Jan 96 12:39:17 EST
Subject: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging
In-Reply-To: <9601161131.ZM4105@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov)
Message-ID: <9601161739.AA19446@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: "Jim Fulton, U.S. Geological Survey" <jfulton@usgs.gov>
> 2) One C module that must be statically linked called "multiarraymodule.c"
By statically linked, I assume you mean statically linked into the interpreter.
Right?
> Having this particular module statically linked will eliminate the
> need for getting the CObject proposal working before release.
As Guido said, the CObject proposal is working. I'll send it to you in a
separate note.
I finished the (very small) CObject implementation very soon after the
workshop because I feel it is important to use it for the Matrix (Numeric)
module.
I feel strongly that the Matrix module should export it's C interface
using CObjects so that modules using matrices to not require that any
of the matrix software be statically linked. In my distribution of
Python, I statically link as little as possible to keep the
interpreter and the interpreter start-up time small. I
plan to modify FIDL to use the CObject-exported interface.
I'd be happy to assist you with this if you wish.
I guess we run on faster networks at MIT, I never bother with dynamic
linking, and find that 4MB binaries launch as fast as I wish. I'd be
more than happy to have somebody else implement the CObject interface,
but I just don't see the time it would take (including getting myself
up to speed on the vagaries of dynamic linking) is worthwhile. If you
(Jim Fulton) want to try to add this interface, I'd be happy to help.
>
> Use "PyArray_" as the name of the Matrix Object. This is a simple
> renaming of the existing "PyMatrix_".
>
> Use "array(sequence, typecode='d')" as the default
> constructor for this new C type.
Is this a replacement for the existing array type?
I decided it was not worth the huge set of compromises that would have
been necessary to make the matrix/array object truly compatible with
the existing array object. Still, the right name for this object
really is array.
There's no problem with using PyArray as the C name for the object
because the existing array object does not export an interface. Also,
there's no problem with using the name "array" as a constructor
because avoiding these sorts of naming conflicts is why python has a
module system in the first place. No existing code that imports
"arraymodule" will be broken, but hopefully people in the future will
start using the new multiarraymodule for the same tasks.
> 4) Two python objects, "Array.py" and "Matrix.py"
Are these imported by Numeric, or would the user be importing
these?
I plan to have everything in the basic distribution imported into a
flat name space under the Numeric module. However, there's nothing to
stop people from importing these independently if they wish.
Is the current built-in array module going away? If not, then there
is a name conflict on case-insensitive file systems.
The array module is not going away. I'm confused about the problem of
case-insensitive file systems, what do they do with tkintermodule and
TkInter? If this is in fact an issue, then "Array.py" can be changed
to "UserArray.py" in the spirit of UserList, etc.
I like the idea
of having the user import Numeric and then use the
Numeric.Matrix_d(...) or Numeric.Array_f(...) rather than than
importing Matrix and Array. Is there any reason for the user to
import Array directly? I am strongly opposed to using "Array" unless
the "array" module goes away.
I agree with all this (except the last line, which I'm willing to
conceed after you explain my tkinter question).
> In order to support these python objects (and others like them), two
> special data members will be added, "__array__", and "__object__". If
> an object has the member "__array__", then the C functions that handle
> matrices will attempt to retrieve the matrix from this member when
> passed in a python object.
Are we taking about python members or C structure members? Is the
__array__ member supposed to be the C pointer to a block of memory?
The __array__ member is a member of a python object which is expected
to contain a python object of type array (the type created in C that
this whole thing is based on).
> In addition, they will attempt to convert
> their result to an object of class "__object__" upon return.
Class __object__? So __object__ is a pointer to a Python class
object?
This is still a python member. In python what it would do is call
m.__object__(new_array). I assume that a similar thing can be done in C
(I haven't implemented this in C yet).
> This
> means that umath.sin(Array([0, pi/2, pi])) == Array([0.,1.,0.]).
OK. This makes sense
Remember that Array is a python object here, that's the trick I'm
trying to make work out.
> Hopefully, this convention will allow these python objects to coexist
> well with any numeric libraries.
Could you provide some additional details?
Here's a bit of code for a unary function expecting a single PyArray
argument of type "double" of two dimensions:
PyObject *op;
PyArrayObject *ap, *rp;
TRY(PyArg_ParseTuple(args, "O", &op));
TRY(ap = PyArray_ContiguousFromObject(op, PyArray_DOUBLE, 2, 2));
// Do something with ap to get rp
Py_DECREF(ap);
return PyArray_Return(rp, op);
With the exception of the second argument to PyArray_Return, this is
the current way of writing such a chunk of code.
PyArray_ContiguousFromObject will convert any python sequence type to
an array of the appropriate type and dimensions if possible. If the
argument is already an array of the appropriate type and dimensions,
then that array will be increfed and returned (unless its data points
to a discontiguous chunk of memory in which case it will be copied
into a new array with contiguous memory).
The new feature that I want to add to this function is that if its
argument is a python object with the attribute "__array__", then this
function wil return the PyArrayObject contained in that attribute (if
this is indeed the case).
PyArray_Return is used because some operations wind up producing a
0-dimensional array. These will be converted to the appropriate
python scalars on return.
The new feature that I want to add here is that if the second argument
has a "__class__" attribute, then the constructor for that class will
be used to return a new python object with the returned PyArrayObject
in its "__array__" attribute.
This is the simplest method I could come up with to get my
"sin(Array())" example to work.
>
> 6) Great documentation and tutorials (hopefully written by Paul
> DuBois).
Wat cool. Will we also get doc strings?
doc strings are already done (probably could use some polishing, but...
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Tue Jan 16 17:53:52 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Tue, 16 Jan 96 12:53:52 EST
Subject: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging
In-Reply-To: <199601161710.MAA07614@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA)
Message-ID: <9601161753.AA19495@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Date: Tue, 16 Jan 1996 12:10:28 -0500
From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad)
1) Konrad's numeric patches to the python core (incorporated into the
working version of python by Guido) will be required and included with
the distribution. These will be the only patches to the python core
required.
There has been a slight modification in the meantime (there will be
only 'j' for imaginary constants, not 'i') and two corrections that
perhaps not everybody has received. I'll prepare a new set of patch
files for the beta release.
Great, just upload the patch files to "the site" when you're ready.
This PyArray C type will not implement automatic type-coercion (unlike the
current implementation). The reason for this is that I have decided
type-coercion is a major pain for arrays of floats (as opposed to
doubles) and I happen to use arrays of floats for most of my work. If
somebody can give me a good suggestion for how to keep automatic
type-coercion and have Matrix_f(1,2,3)*1.2 == Matrix_f(1.2,2.4,3.6)
then I might consider changing this decision. See later note on
Array.py for an alternative.
One way to solve your "float" problem would be to write
Matrix_f(1,2,3)*Matrix_f(1.2),
i.e. cast 1.2 explicitly to float. I don't see how you can avoid
that cast anyway; without type coercion, Matrix_f(1,2,3)*1.2 would
be an error.
I'm willing to be a little bit sloppy with some of the details of
types, and force the type of a scalar argument to correspond to the
type of the matrix. This is essentially what I'm doing anyway when I
say Matrix_f(1.2) [Note, I'll have another post to come about
constructor function].
I'd prefer to have type coercion everywhere for consistency. If the
"high-level" classes Array and Matrix are really fast enough (about
which I have doubts, see below), then it doesn't matter that much, but
in general I'd give top priority to consistency and put the burden of
more complicated coding on those who want top speed (which of course
must be possible).
We seem to just have a difference of opinion here, I'll wait and see
if more opinions come in.
3) Two dynamically linkable modules called "umathmodule.c", and "ieee_umathmodule.c"
I don't think that using "ieee" in the name is a good idea. After all,
there is no guarantee that this will really use IEEE arithmetic (a
Vax, for example, doesn't have IEEE arithmetic hardware). Why not
just "fast_umath"?
Good point, actually I was originally considering "slow_umath" and
"fast_umath", but I knew you wouldn't like that. So, the new proposed
name is "umath" and "fast_umath".
4) Two python objects, "Array.py" and "Matrix.py"
Array is essentially a python binding around the underlying C type,
and this will also provide for automatic type-coercion and will
generally assume that it is only working with arrays of type long,
double, and complex double (the three types of arrays that are
equivalent to python objects). In my initial tests of this object on
LLNL's simple benchmark, I found that the performance was only 5%
slower than using the C object directly.
Nevertheless the Python wrapper could cause a significant overhead
for small arrays, which will also be used heavily. For example,
I am planning to change my vector class (3d vectors for geometry
etc.) to use an array of length 3 as its internal representation.
Do you have any data on the speed penalty for such small arrays?
No data yet on that, if you want to send me a nice simple "real"
benchmark code for 3d vectors I'd be happy to add it to my bag of
tricks. Right now everything is being optimized for LLNL style
physics code because they sent me such a nice benchmark to run.
Matrix will inherit almost everything from Array, however it will be
limited to 2 or fewer dimensions, and m1*m2 where m1 and m2 are
matrices will perform matrix style multiplication. If the linear
algebra people would like, other changes can be made (ie. ~m1 ==
m1.transpose(), ...). Based on the experiments with Array, the
performance penalty for this approach should be minimal.
I suppose some discussion will be necessary about the details of
this class, which until now didn't exist. Given that many people
seem to prefer a distinction between row and column vectors, it
would be a good idea to include separate constructors for them.
Now that I've brought things out into python classes, I think I'll
probably just let somebody who's into linear algebra code worry about
those sorts of refinements.
5) A standard library "Numeric.py" which will be the standard way of
importing multiarray, Array, Matrix, umath, etc. It will include the
inverted trig functions ("sec", "csc", "arcsec", ...) as well as most
of the standard functions currently in Matrix.py
I am not sure it is a good idea to have such an all-encompassing
module. It would contain a rather arbitrary selection of things,
i.e. those which exist at the time it is introduced. I expect
that other numerical modules will appear later. So it would be
better to group things according to functions or applications.
I disagree here. I find that there is a certain core of
functions that are very nice to be able to grab with a single import
statement. I would hate to have to start my typical script with
import umath, UMathMore, Matrix, Array, ArrayConstructors,
ArrayUtilities, ...
9?) A "numericmodule.c" which contains reasonably efficient fft,
matrix inversion, convolution, random numbers, eigenvalues and
filtering functions stolen from existing C code so that the package
can be viewed as a "complete" numerical computing system?
Again this should be several modules: linear algebra, random
numbers, fft/convolution. We won't achieve anything "complete"
anyway, so let's not pretend we do.
You're obviously right here. Part of what I was hoping to do with
this was to create a simple module so that people (even those without
a FORTRAN compiler) could take the inverse of a matrix in a reasonable
amount of time, and possibly to set up some sort of API, so that when
somebody does write the ultimate linear algebra module they will be
likely include a function called "inverse" so that I could then
trivially plug in this new more powerful system.
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Tue Jan 16 18:00:45 1996
From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey)
Date: Tue, 16 Jan 1996 13:00:45 -0500
Subject: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging
In-Reply-To: hinsenk@ERE.UMontreal.CA (Hinsen Konrad)
"Re: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging" (Jan 16, 12:17pm)
References: <199601161717.MAA08000@cyclone.ERE.UMontreal.CA>
Message-ID: <9601161300.ZM4174@dsdbqvarsa.er.usgs.gov>
On Jan 16, 12:17pm, Hinsen Konrad wrote:
> Subject: Re: [PYTHON MATRIX-SIG] Final matrix object renaming and packagin
>
> I feel strongly that the Matrix module should export it's C interface
> using CObjects so that modules using matrices to not require that any
> of the matrix software be statically linked. In my distribution of
>
> I agree in principle, but that doesn't have to happen in the
> first beta release.
Perhaps not. I'll put this together myself when I get the next alpha
release.
>
> Are these imported by Numeric, or would the user be importing these?
> Is the current built-in array module going away? If not, then there
> is a name conflict on case-insensitive file systems. I like the idea
>
> Why? The current "array" in a C module, so "import Array" will
> never load it, independent of the file name conventions. Which
> means that "import Array" will always obtain an external module,
> and as long as ours is the only one, it hardly matters if it
> is called "Array.py" or "array.py".
Actually, at least in 1.2, if a C module and a Python module are in the same
directory, the C module gets loaded first. But this will be most likely a
function of what order libraries appear in sys.path. I just think that it
would be best to avoid this problem altogther by using a different name.
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Tue Jan 16 18:07:03 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Tue, 16 Jan 96 13:07:03 EST
Subject: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging
In-Reply-To: <199601161717.MAA08000@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA)
Message-ID: <9601161807.AA19538@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Date: Tue, 16 Jan 1996 12:17:41 -0500
From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad)
importing Matrix and Array. Is there any reason for the user to
import Array directly? I am strongly opposed to using "Array" unless
Certainly, e.g. if the user doesn't want anything else. Besides, as I
pointed out in my last mail, a general "Numeric" module is not such a
terrific idea.
So someone who wants to create a matrix object will do something like:
import Numeric
m=Numeric.Matrix_d(...)
Speaking about constructors... I still prefer (strongly) more
meaningful names, like IntegerMatrix. Noone should be forced
to learn these silly type codes for standard applications.
Well, actually I guess I wasn't clear enough about my choice for
constructors.
The array constructor (for PyArrayObjects):
array([1,2,3], 'd') OR array([1,2,3], types.FloatType)
will both produce a 1d array of doubles.
Similarly for Arrays (and Matrix's):
Array([1,2,3], 'd') OR Array([1.,2.,3.]) OR
Array([1,2,3], types.FloatType)
One dilemma left is what should be used as the standard string
representation of an Array (or an array).
I vote for Array([1,2,3], 'd'), but I'm open to other proposals.
Now, I happen to really like the old notation, and I'd love to come up
with a way to keep it around as a shorthand input notation. I really
did find that the removal of a set of parenthesis made some of my
denser code a lot easier to read.
So, I propose to add A_d(1,2,3) as a convenient shorthand for
Array([1,2,3], "d"). Please, let me know how terrible an idea this is.
I'd also be happy to have FloatArray(1,2,3) instead. If something
like this is desired, then I might even choose to use it as the
standard string representation.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Tue Jan 16 18:42:37 1996
From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey)
Date: Tue, 16 Jan 1996 13:42:37 -0500
Subject: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging
In-Reply-To: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
"Re: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging" (Jan 16, 12:39pm)
References: <9601161739.AA19446@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <9601161342.ZM4236@dsdbqvarsa.er.usgs.gov>
On Jan 16, 12:39pm, James Hugunin wrote:
> Subject: Re: [PYTHON MATRIX-SIG] Final matrix object renaming and packagin
>
> From: "Jim Fulton, U.S. Geological Survey" <jfulton@usgs.gov>
>
> > 2) One C module that must be statically linked called
"multiarraymodule.c"
>
> By statically linked, I assume you mean statically linked into the
interpreter.
> Right?
>
> > Having this particular module statically linked will eliminate the
> > need for getting the CObject proposal working before release.
>
> As Guido said, the CObject proposal is working. I'll send it to you in a
> separate note.
>
> I finished the (very small) CObject implementation very soon after the
> workshop because I feel it is important to use it for the Matrix (Numeric)
> module.
>
> I feel strongly that the Matrix module should export it's C interface
> using CObjects so that modules using matrices to not require that any
> of the matrix software be statically linked. In my distribution of
> Python, I statically link as little as possible to keep the
> interpreter and the interpreter start-up time small. I
> plan to modify FIDL to use the CObject-exported interface.
>
> I'd be happy to assist you with this if you wish.
>
> I guess we run on faster networks at MIT, I never bother with dynamic
> linking, and find that 4MB binaries launch as fast as I wish.
Just out of curiousity, how fast is that? What do you get from:
time python -c ''
On some of our slower systems, this takes about a second, even with
almost all modules dynamically loaded. This is much slower than I'd
like it to be, since Python is often used here for very small scripts
in which this startup time is a significant part of the overall
execution time. Starting up another version of the interpreter with
Tk linked in takes nearly twice as long.
> I'd be
> more than happy to have somebody else implement the CObject interface,
> but I just don't see the time it would take (including getting myself
> up to speed on the vagaries of dynamic linking) is worthwhile. If you
> (Jim Fulton) want to try to add this interface, I'd be happy to help.
I'll take a crack at this when you release 0.3.
> >
> > Use "PyArray_" as the name of the Matrix Object. This is a simple
> > renaming of the existing "PyMatrix_".
> >
> > Use "array(sequence, typecode='d')" as the default
> > constructor for this new C type.
>
> Is this a replacement for the existing array type?
>
> I decided it was not worth the huge set of compromises that would have
> been necessary to make the matrix/array object truly compatible with
> the existing array object. Still, the right name for this object
> really is array.
>
> There's no problem with using PyArray as the C name for the object
> because the existing array object does not export an interface. Also,
> there's no problem with using the name "array" as a constructor
> because avoiding these sorts of naming conflicts is why python has a
> module system in the first place. No existing code that imports
> "arraymodule" will be broken, but hopefully people in the future will
> start using the new multiarraymodule for the same tasks.
But existing imports of array on some systems may be broken. Even though the
array module is stored in arraymodule._, it is imported with "import array".
> > 4) Two python objects, "Array.py" and "Matrix.py"
>
> Are these imported by Numeric, or would the user be importing
> these?
>
> I plan to have everything in the basic distribution imported into a
> flat name space under the Numeric module. However, there's nothing to
> stop people from importing these independently if they wish.
>
> Is the current built-in array module going away? If not, then there
> is a name conflict on case-insensitive file systems.
>
> The array module is not going away. I'm confused about the problem of
> case-insensitive file systems, what do they do with tkintermodule and
> TkInter?
Nothing, since Tkinter doesn't work on these systems yet. :-)
> If this is in fact an issue, then "Array.py" can be changed
> to "UserArray.py" in the spirit of UserList, etc.
I think it should.
> I like the idea
> of having the user import Numeric and then use the
> Numeric.Matrix_d(...) or Numeric.Array_f(...) rather than than
> importing Matrix and Array. Is there any reason for the user to
> import Array directly? I am strongly opposed to using "Array" unless
> the "array" module goes away.
>
> I agree with all this (except the last line, which I'm willing to
> conceed after you explain my tkinter question).
Great. ;-)
> > In order to support these python objects (and others like them), two
> > special data members will be added, "__array__", and "__object__". If
> > an object has the member "__array__", then the C functions that handle
> > matrices will attempt to retrieve the matrix from this member when
> > passed in a python object.
>
> Are we taking about python members or C structure members? Is the
> __array__ member supposed to be the C pointer to a block of memory?
>
> The __array__ member is a member of a python object which is expected
> to contain a python object of type array (the type created in C that
> this whole thing is based on).
>
> > In addition, they will attempt to convert
> > their result to an object of class "__object__" upon return.
>
> Class __object__? So __object__ is a pointer to a Python class
> object?
>
> This is still a python member. In python what it would do is call
> m.__object__(new_array). I assume that a similar thing can be done in C
> (I haven't implemented this in C yet).
And new_array is one of the new built-in array objects?
> > This
> > means that umath.sin(Array([0, pi/2, pi])) == Array([0.,1.,0.]).
>
> OK. This makes sense
>
> Remember that Array is a python object here, that's the trick I'm
> trying to make work out.
So all of this is really about being able to derive Python classes from
built-in types? That is, you want an Array (which is an instance of a Python
class) to store it's data in an array (which is an object of type PyArrayType),
and you want functions that you pass an Array to to get at it's array. Have I
got this right?
(BTW, you should export the actual type objects.)
> > Hopefully, this convention will allow these python objects to coexist
> > well with any numeric libraries.
>
> Could you provide some additional details?
>
> Here's a bit of code for a unary function expecting a single PyArray
> argument of type "double" of two dimensions:
>
> PyObject *op;
> PyArrayObject *ap, *rp;
>
> TRY(PyArg_ParseTuple(args, "O", &op));
> TRY(ap = PyArray_ContiguousFromObject(op, PyArray_DOUBLE, 2, 2));
>
> // Do something with ap to get rp
>
> Py_DECREF(ap);
>
> return PyArray_Return(rp, op);
>
> With the exception of the second argument to PyArray_Return, this is
> the current way of writing such a chunk of code.
>
> PyArray_ContiguousFromObject will convert any python sequence type to
> an array of the appropriate type and dimensions if possible. If the
> argument is already an array of the appropriate type and dimensions,
> then that array will be increfed and returned (unless its data points
> to a discontiguous chunk of memory in which case it will be copied
> into a new array with contiguous memory).
>
> The new feature that I want to add to this function is that if its
> argument is a python object with the attribute "__array__", then this
> function wil return the PyArrayObject contained in that attribute (if
> this is indeed the case).
OK. If my statement above is right, then I understand this.
> PyArray_Return is used because some operations wind up producing a
> 0-dimensional array. These will be converted to the appropriate
> python scalars on return.
>
> The new feature that I want to add here is that if the second argument
> has a "__class__" attribute, then the constructor for that class will
You mean __object__? (I like __class__, or maybe even
__return_constructor__ better.)
> be used to return a new python object with the returned PyArrayObject
> in its "__array__" attribute.
So PyArray_Return checks to see of op has a callable __object__ member and if
it does, returns the result of calling this member with rp as an argument.
Right?
> This is the simplest method I could come up with to get my
> "sin(Array())" example to work.
Whew. I need to think about this. I'm not faulting your approach, but it
feels a bit complicated.
What if you had a function with multiple arguments and you wanted the
returned object to have the same type as the arguments? For example,
what if you wanted
spam(some_Array, some_other_Array) to return an Array and
spam(some_Matric, some_other_Matrix) to return a Matrix?
Would you use the first argument's __object__ or the second's?
I'll probably have more to say about this after I take some time to mull it
over.
>
> >
> > 6) Great documentation and tutorials (hopefully written by Paul
> > DuBois).
>
> Wat cool. Will we also get doc strings?
>
> doc strings are already done (probably could use some polishing, but...
Great!
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Jan 16 18:43:07 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Tue, 16 Jan 1996 13:43:07 -0500
Subject: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging
In-Reply-To: <9601161753.AA19495@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199601161843.NAA13986@cyclone.ERE.UMontreal.CA>
I'm willing to be a little bit sloppy with some of the details of
types, and force the type of a scalar argument to correspond to the
type of the matrix. This is essentially what I'm doing anyway when I
Does that mean that in "2.5*array([1,2,3], types.Integer)" the
prefactor will be silently cast to int? That would be a most
undesirable form of automatic type conversion.
One of the principles of operation should be that scalars are
considered as arrays of rank 0. From that it follows that they can be
cast explicitly to a rank-0 array of any type (including C-float), and
there are no more type problems.
BTW, that reminds me of another old constructor debate: should
array([1,2,3]) be a rank-1 or a rank-2 array? I still propose that
all constructors should take exactly one argument, scalar or
list, and that the rank of the array depends on the nesting of the
list. And now I have another argument for this: your current
constructors don't permit the construction of rank-0 arrays.
No data yet on that, if you want to send me a nice simple "real"
benchmark code for 3d vectors I'd be happy to add it to my bag of
Unfortunately that doesn't exist yet...
tricks. Right now everything is being optimized for LLNL style
physics code because they sent me such a nice benchmark to run.
Which is hardly a good way to ensure the general usefulness
of a program ;-)
I disagree here. I find that there is a certain core of
functions that are very nice to be able to grab with a single import
statement. I would hate to have to start my typical script with
import umath, UMathMore, Matrix, Array, ArrayConstructors,
ArrayUtilities, ...
Me too. But I'd rather write this "collection" module myself,
with everything I want in it, instead of having a standard
module that doesn't meet my needs.
You're obviously right here. Part of what I was hoping to do with
this was to create a simple module so that people (even those without
a FORTRAN compiler) could take the inverse of a matrix in a reasonable
amount of time, and possibly to set up some sort of API, so that when
somebody does write the ultimate linear algebra module they will be
likely include a function called "inverse" so that I could then
trivially plug in this new more powerful system.
We should learn a lesson from the old array module: never make an
interim solution a standard module. If we now quickly hack together
some collection of "useful" routines, then people will use this
interface and have problems later when they discover that they need
something better. Of course the implementation can always be
improved, but if we define a standard interface for anything,
it should be a good one.
For linear algebra, the best solution would probably be a module based
on LAPACK, which is free and also available in C (although the Fortran
version is more efficient on most machines). I don't know if
similar good and free packages exist for other applications, but
if we can't find out, we should leave this to other people.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Jan 16 18:46:30 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Tue, 16 Jan 1996 13:46:30 -0500
Subject: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging
In-Reply-To: <9601161300.ZM4174@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov)
Message-ID: <199601161846.NAA14142@cyclone.ERE.UMontreal.CA>
function of what order libraries appear in sys.path. I just think that it
would be best to avoid this problem altogther by using a different name.
I'd rather investigate a bit more and choose a reasonable name if at
all possible. After all, we are creating an API to be used by
(hopefully ;-) millions of users, who probably don't want to dig into
Python history to find out why something called an "array" everywhere
has some strange name in Python. The compromises made necessary
by Python's strange comparison implementation are bad enough.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Jan 16 18:46:30 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Tue, 16 Jan 1996 13:46:30 -0500
Subject: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging
In-Reply-To: <9601161300.ZM4174@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov)
Message-ID: <199601161846.NAA14142@cyclone.ERE.UMontreal.CA>
function of what order libraries appear in sys.path. I just think that it
would be best to avoid this problem altogther by using a different name.
I'd rather investigate a bit more and choose a reasonable name if at
all possible. After all, we are creating an API to be used by
(hopefully ;-) millions of users, who probably don't want to dig into
Python history to find out why something called an "array" everywhere
has some strange name in Python. The compromises made necessary
by Python's strange comparison implementation are bad enough.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Tue Jan 16 19:07:25 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Tue, 16 Jan 96 14:07:25 EST
Subject: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging
In-Reply-To: <9601161342.ZM4236@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov)
Message-ID: <9601161907.AA21369@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: "Jim Fulton, U.S. Geological Survey" <jfulton@usgs.gov>
> I guess we run on faster networks at MIT, I never bother with dynamic
> linking, and find that 4MB binaries launch as fast as I wish.
Just out of curiousity, how fast is that? What do you get from:
time python -c ''
On some of our slower systems, this takes about a second, even with
almost all modules dynamically loaded. This is much slower than I'd
like it to be, since Python is often used here for very small scripts
in which this startup time is a significant part of the overall
execution time. Starting up another version of the interpreter with
Tk linked in takes nearly twice as long.
~:jjh@baribal: ls -l ~/PythonSun4/python
-rwxrwxr-x 1 jjh 3227648 Jan 16 11:09 /usr/users/jjh/PythonSun4/python*
~:jjh@baribal: time ~/PythonSun4/python -c ''
0.060u 0.060s 0:00.14 85.7% 0+234k 0+0io 0pf+0w
If you're curious about the details:
All our active files are stored on a Pentium box running a
high-performance NFS server kernel (no real operating system) this
links into the network through a 100MB/s link to a switched hub.
> I'd be
> more than happy to have somebody else implement the CObject interface,
> but I just don't see the time it would take (including getting myself
> up to speed on the vagaries of dynamic linking) is worthwhile. If you
> (Jim Fulton) want to try to add this interface, I'd be happy to help.
I'll take a crack at this when you release 0.3.
Great!
> >
> > Use "PyArray_" as the name of the Matrix Object. This is a simple
> > renaming of the existing "PyMatrix_".
> >
> > Use "array(sequence, typecode='d')" as the default
> > constructor for this new C type.
>
> Is this a replacement for the existing array type?
>
> I decided it was not worth the huge set of compromises that would have
> been necessary to make the matrix/array object truly compatible with
> the existing array object. Still, the right name for this object
> really is array.
>
> There's no problem with using PyArray as the C name for the object
> because the existing array object does not export an interface. Also,
> there's no problem with using the name "array" as a constructor
> because avoiding these sorts of naming conflicts is why python has a
> module system in the first place. No existing code that imports
> "arraymodule" will be broken, but hopefully people in the future will
> start using the new multiarraymodule for the same tasks.
But existing imports of array on some systems may be broken. Even though the
array module is stored in arraymodule._, it is imported with "import array".
You're talking about something else here. In my setup you'd do something like:
from multiarray import array
this would not conflict in any way with the existing arraymodule. Now
arguments about Array.py on systems with case-insensitive filesystems
are a different matter.
> > In order to support these python objects (and others like them), two
> > special data members will be added, "__array__", and "__object__". If
> > an object has the member "__array__", then the C functions that handle
> > matrices will attempt to retrieve the matrix from this member when
> > passed in a python object.
>
> Are we taking about python members or C structure members? Is the
> __array__ member supposed to be the C pointer to a block of memory?
>
> The __array__ member is a member of a python object which is expected
> to contain a python object of type array (the type created in C that
> this whole thing is based on).
>
> > In addition, they will attempt to convert
> > their result to an object of class "__object__" upon return.
>
> Class __object__? So __object__ is a pointer to a Python class
> object?
>
> This is still a python member. In python what it would do is call
> m.__object__(new_array). I assume that a similar thing can be done in C
> (I haven't implemented this in C yet).
And new_array is one of the new built-in array objects? Exactly.
> > This
> > means that umath.sin(Array([0, pi/2, pi])) == Array([0.,1.,0.]).
>
> OK. This makes sense
>
> Remember that Array is a python object here, that's the trick I'm
> trying to make work out.
So all of this is really about being able to derive Python classes from
built-in types? That is, you want an Array (which is an instance of a Python
class) to store it's data in an array (which is an object of type PyArrayType),
and you want functions that you pass an Array to to get at it's array. Have I
got this right?
Yep. In addition, I want these functions to return an Array instead of
an array. (Even I'm beggining to doubt the value of this case-based
differentiation of names by now).
(BTW, you should export the actual type objects.)
Of course, but this still doesn't get me the behavior I want.
> > Hopefully, this convention will allow these python objects to coexist
> > well with any numeric libraries.
>
> Could you provide some additional details?
>
> Here's a bit of code for a unary function expecting a single PyArray
> argument of type "double" of two dimensions:
>
> PyObject *op;
> PyArrayObject *ap, *rp;
>
> TRY(PyArg_ParseTuple(args, "O", &op));
> TRY(ap = PyArray_ContiguousFromObject(op, PyArray_DOUBLE, 2, 2));
>
> // Do something with ap to get rp
>
> Py_DECREF(ap);
>
> return PyArray_Return(rp, op);
>
> With the exception of the second argument to PyArray_Return, this is
> the current way of writing such a chunk of code.
>
> PyArray_ContiguousFromObject will convert any python sequence type to
> an array of the appropriate type and dimensions if possible. If the
> argument is already an array of the appropriate type and dimensions,
> then that array will be increfed and returned (unless its data points
> to a discontiguous chunk of memory in which case it will be copied
> into a new array with contiguous memory).
>
> The new feature that I want to add to this function is that if its
> argument is a python object with the attribute "__array__", then this
> function wil return the PyArrayObject contained in that attribute (if
> this is indeed the case).
OK. If my statement above is right, then I understand this.
> PyArray_Return is used because some operations wind up producing a
> 0-dimensional array. These will be converted to the appropriate
> python scalars on return.
>
> The new feature that I want to add here is that if the second argument
> has a "__class__" attribute, then the constructor for that class will
You mean __object__? (I like __class__, or maybe even
__return_constructor__ better.)
I guess I still haven't finalized the name of this yet. I like
__return_constructor__, so I'll stick with that for now.
> be used to return a new python object with the returned PyArrayObject
> in its "__array__" attribute.
So PyArray_Return checks to see of op has a callable __object__ member and if
it does, returns the result of calling this member with rp as an argument.
Right?
Right, except of course now it checks for a __return_constructor__ member.
> This is the simplest method I could come up with to get my
> "sin(Array())" example to work.
Whew. I need to think about this. I'm not faulting your approach, but it
feels a bit complicated.
What if you had a function with multiple arguments and you wanted the
returned object to have the same type as the arguments? For example,
what if you wanted
spam(some_Array, some_other_Array) to return an Array and
spam(some_Matric, some_other_Matrix) to return a Matrix?
Would you use the first argument's __object__ or the second's?
If they had different __return_constructor__'s, then I'd raise an
exception. If they were the same, or only one of them had one, then
I'd use the one that was present.
I'll probably have more to say about this after I take some time to mull it
over.
Please say more about this. I'm just trying to come up with a way to
make it reasonable to subclass the array/Array object for purposes
like creating a Matrix object with as little pain as possible.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dubois1@llnl.gov Tue Jan 16 19:27:31 1996
From: dubois1@llnl.gov (Paul. Dubois)
Date: Tue, 16 Jan 1996 11:27:31 -0800
Subject: [PYTHON MATRIX-SIG] Thoughts on latest messages
References: <9601161807.AA19538@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <30FBFC23.6D98@llnl.gov>
Congratulations on your marriage and best wishes to you and your wife.
We all hope you can "juggle" the matrix work in with the tasks you will
receive from The Boss.
Some thoughts on the latest messages. I think it best if I simply phrase
them as what I need to do and let you mull over how this fits with
various schemes. In what follows I use the name Thing to stand for "one
of them thar matrix objects however it turns out you name it")
a. In a C extension module, I need to create a Thing. The creation
routines you have supplied in the present release are sufficient to my
needs.
b. Given a block of memory representing an array stored in column-major
order, I need to turn it into a Thing. Things are natively row-major but
given the current implementation I can diddle with the strides in the
Thing and it works. This means Fortran interfacing is easy. Cool. Try
not to lose this, and maybe we could make a constructor that did the
kludge officially.
c. If you want to make a Whatzit in Python that encapsulates a Thing but
adds some special semantics (say, for example, arrays whose lower index
is not 0) you can almost do it perfectly with a wrapper class but the
one thing you can't spoof is when the object is an argument to something
that expects a Thing. viz, sqrt(m). If I understood you, the idea would
be that all routines expecting a Thing would, finding that they hadn't
gotten one, try to do getattr(m, '__array__') so that objects that
wished to pretend they are Things could do so. I think this will prove
very valuable and can testify that I already ran into a case where it
would have been a nice solution for me.
Opinions:
I like the coercion properties of the current package VERY much. If you
need to do something special like keep expressions from drifting up from
float to double, then some explicit handling of scalar constants is
clearly required in either case, so I'm not persuaded anything would be
gained by undoing your good work.
About the packaging of mathematical software. A glance at the humongous
documentation for Nag or IMSL will suggest that organizing the software
into "clusters" by subject would be a good idea. In the EiffelMath
library we have clusters for ODES, probability and statistics, linear
solvers, optimization, root finding, time series analysis, etc. This
helps people find what they need and also not have to bring in too much
code just to get at a given piece.
The Thing class is very likely to get imported via 'from Thing import *'
because of the need to make the sqrt etc. work right; therefore, this
namespace should be kept minimal, with more specialized software kept in
other modules. Since it is easy to make supermodules that import sets of
other modules, one can package the facilities to suit the style of work
one does.
If it is really true that you can make a PythonThing level object that
has performance only 5% off of Thing, then that would be useful since we
could inherit from it. Especially if (c) is done.
--
Paul F. Dubois, L-472 (510)-422-5426
Lawrence Livermore National Laboratory FAX (510)-423-9969
Livermore, CA 94550 dubois1@llnl.gov
Consulting: PaulDubois@aol.com
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dubois1@llnl.gov Tue Jan 16 19:36:55 1996
From: dubois1@llnl.gov (Paul. Dubois)
Date: Tue, 16 Jan 1996 11:36:55 -0800
Subject: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging
References: <199601161846.NAA14142@cyclone.ERE.UMontreal.CA>
Message-ID: <30FBFE57.4A68@llnl.gov>
Before we get too concerned about the old "array", has anyone checked
the library and found out how many places it is used? Could the new one
be plugged in pretty easily in most places that use it now? (I suspect
so.) Could someone with access to the contributer library grep it?
While requiring users to change existing code is very undesireable, this
needs to be balanced by asking how many users, how much code, how much
trouble. In the case of a central facility like this one, it is worth a
little pain to use the right name, which surely is array.
--
Paul F. Dubois, L-472 (510)-422-5426
Lawrence Livermore National Laboratory FAX (510)-423-9969
Livermore, CA 94550 dubois1@llnl.gov
Consulting: PaulDubois@aol.com
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From da@maigret.cog.brown.edu Tue Jan 16 20:47:50 1996
From: da@maigret.cog.brown.edu (David Ascher)
Date: Tue, 16 Jan 1996 15:47:50 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] Thoughts on latest messages
In-Reply-To: <30FBFC23.6D98@llnl.gov> from "Paul. Dubois" at Jan 16, 96 11:27:31 am
Message-ID: <199601162047.PAA12207@maigret>
Paul Dubois:
>c. If you want to make a Whatzit in Python that encapsulates a Thing but
>adds some special semantics (say, for example, arrays whose lower index
>is not 0) you can almost do it perfectly with a wrapper class but the
>one thing you can't spoof is when the object is an argument to something
>that expects a Thing. viz, sqrt(m). If I understood you, the idea would
>be that all routines expecting a Thing would, finding that they hadn't
>gotten one, try to do getattr(m, '__array__') so that objects that
>wished to pretend they are Things could do so. I think this will prove
>very valuable and can testify that I already ran into a case where it
>would have been a nice solution for me.
Same here. It's a scheme similar to that used by Brian Werkentine in
Rivet (you can call rivet functions w/ rivetobjs or with classes with a
__widget attribute which is a rivetobj). It makes lots of things much
much simpler.
Examples that I've been thinking about are "documented matrices", (class
instances with names and properties for the various dimensions) and
"visible matrices" (class instances with state information for displayed
matrices.
--david
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Jan 16 18:52:27 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Tue, 16 Jan 1996 13:52:27 -0500
Subject: [PYTHON MATRIX-SIG] Final matrix object renaming and packaging
In-Reply-To: <9601161807.AA19538@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199601161852.NAA14638@cyclone.ERE.UMontreal.CA>
The array constructor (for PyArrayObjects):
array([1,2,3], 'd') OR array([1,2,3], types.FloatType)
will both produce a 1d array of doubles.
OK.
Similarly for Arrays (and Matrix's):
Array([1,2,3], 'd') OR Array([1.,2.,3.]) OR
Array([1,2,3], types.FloatType)
Also OK.
One dilemma left is what should be used as the standard string
representation of an Array (or an array).
I vote for Array([1,2,3], 'd'), but I'm open to other proposals.
For the high-level arrays, I'd prefer simply Array([1,2,3])
(which is the simplest equivalent input format). For the
low-level arrays, printing the type code is probably the best
solution.
Now, I happen to really like the old notation, and I'd love to come up
with a way to keep it around as a shorthand input notation. I really
did find that the removal of a set of parenthesis made some of my
denser code a lot easier to read.
So, I propose to add A_d(1,2,3) as a convenient shorthand for
Array([1,2,3], "d"). Please, let me know how terrible an idea this is.
Terrible enough for me... But do we need such abbreviations
as part of the standard modules? It would probably add to the
confusion of new users. Since it's a trivial Python definition,
without speed penalties later on, why not let everyone define
his own preferred abbreviations?
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Fri Jan 19 15:43:53 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Fri, 19 Jan 96 10:43:53 EST
Subject: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully)
Message-ID: <9601191543.AA13543@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Well, the comments on my previous post seem to have fallen off, so
I'll try to summarize again and see if everybody's more or less happy.
1) No argument about including Konrad's latest patches (which I now
have)
2) Naming conventions for the new module (this is new)
I think that it's important to understand the value of python's module
system in avoiding conflicts with the current array module, while
still using the name array.
The new array module will be called "multiarraymodule.c". To use this
module I need to do the following:
import multiarray
a = multiarray.array([1,2,3])
Thus, the creation operator uses the name array (which has a different
meaning in the old arraymodule).
I will reccommend that people actually use the following style instead
to import this module:
import Numeric
a = Numeric.array([1,2,3]) #an array of longs
b = Numeric.array([1,2,3], types.FloatType) #an array of doubles
c = Numeric.array([1,2,3], 'f') #an array of floats
This C type WILL continue to implement automatic type coercion, but I
reserve the right to remove it if I continue to have problems making
my floating point code work in floats instead of doubles.
I'll see how Jim Fulton's CObjects work out before making a final
decision on when to add them to the release.
An array of doubles will have the standard string representation of:
array([1,2,3], 'd')
As before, the print function is coded in python, and I encourage
people to design friendly array print functions.
3) The two math modules will be umath and fast_umath.
4) Two python object UserArray.py and Matrix.py where Matrix
inherits from UserArray.
The two special data members "__array__", and "__return_constructor__"
will be supported for python objects that want to pretend to be an
array.
5) I will have an overall library Numeric.py including what I consider
to be the basic functions desired by a numeric program. Note: after
considering Konrad's and Paul's comments, this set of basic functions
will be kept to a minimum.
6) doc strings are still the only docs for 0.30, this should change in
BETA1
7) Test suite remains
8) pickle remains
9) "numericmodule.c" is scrapped for now
One last question on installation:
I plan to package this up as an addition to the standard python
distribution, therefore, installation will consist of adding files to
Modules, Lib, and Include. Does this strike anybody as an
unreasonable thing to do?
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Fri Jan 19 16:17:33 1996
From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey)
Date: Fri, 19 Jan 1996 11:17:33 -0500
Subject: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully)
In-Reply-To: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
"[PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully)" (Jan 19, 10:43am)
References: <9601191543.AA13543@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <9601191117.ZM12373@dsdbqvarsa.er.usgs.gov>
On Jan 19, 10:43am, James Hugunin wrote:
> Subject: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully)
>
> Well, the comments on my previous post seem to have fallen off, so
> I'll try to summarize again and see if everybody's more or less happy.
(snip)
> The two special data members "__array__", and "__return_constructor__"
> will be supported for python objects that want to pretend to be an
> array.
After thinking about this a bit, I'm quite comformable with the
__array__ member idea, because there is a precedent with __float__,
__int__, and __long__. However, I have a *strong* opinion that
__array__ should be a member *function* that returns an instance as an
array. This is in keeping with __float__, __long__, etc. If a
user-defined type wants to store it's data in an array data member and
return the member, it can, but other user-defined classes may want to
compute and return array data on demand.
I would also recomment that the __array__ member function should have
an optional type code argument so that an extension that wants, say, a
double precision array can request one.
With an __array__ member function, I could, for example, immagine database
table objects that extracted their numeric data as an array on demand.
This could be quite useful.
I'm not so comfortable with the __return_constructor__ idea. It seems
to only make sense for single-argument functions. I have an alternate
proposal. Suppose that certain functions allowed an optional
(keyword?) argument that provided this constructor. So, for example,
instead of:
bar=sin(foo)
where bar and foo are both Spams (e.g. Arrays) you would have:
bar=sin(foo,Spam)
Now, you don't want to have to type ",Spam" everytime, not because you
don't like typing, but because you don't want the clutter, so you
define a "sin" method in Spam:
def sin(self): return sin(self,self.class)
so then you could do:
bar=foo.sin()
I realize that this is not quite as pretty as the first version (or is
it?), but it feels cleaner overall to me. In the case of
multi-parameter functions, it will be clear which object (the
reciever of the message) determines the return type.
(snip)
> One last question on installation:
>
> I plan to package this up as an addition to the standard python
> distribution, therefore, installation will consist of adding files to
> Modules, Lib, and Include. Does this strike anybody as an
> unreasonable thing to do?
For now, it's OK, but the released version should be a stand-alone module.
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Fri Jan 19 16:17:33 1996
From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey)
Date: Fri, 19 Jan 1996 11:17:33 -0500
Subject: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully)
In-Reply-To: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
"[PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully)" (Jan 19, 10:43am)
References: <9601191543.AA13543@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <9601191117.ZM12373@dsdbqvarsa.er.usgs.gov>
On Jan 19, 10:43am, James Hugunin wrote:
> Subject: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully)
>
> Well, the comments on my previous post seem to have fallen off, so
> I'll try to summarize again and see if everybody's more or less happy.
(snip)
> The two special data members "__array__", and "__return_constructor__"
> will be supported for python objects that want to pretend to be an
> array.
After thinking about this a bit, I'm quite comformable with the
__array__ member idea, because there is a precedent with __float__,
__int__, and __long__. However, I have a *strong* opinion that
__array__ should be a member *function* that returns an instance as an
array. This is in keeping with __float__, __long__, etc. If a
user-defined type wants to store it's data in an array data member and
return the member, it can, but other user-defined classes may want to
compute and return array data on demand.
I would also recomment that the __array__ member function should have
an optional type code argument so that an extension that wants, say, a
double precision array can request one.
With an __array__ member function, I could, for example, immagine database
table objects that extracted their numeric data as an array on demand.
This could be quite useful.
I'm not so comfortable with the __return_constructor__ idea. It seems
to only make sense for single-argument functions. I have an alternate
proposal. Suppose that certain functions allowed an optional
(keyword?) argument that provided this constructor. So, for example,
instead of:
bar=sin(foo)
where bar and foo are both Spams (e.g. Arrays) you would have:
bar=sin(foo,Spam)
Now, you don't want to have to type ",Spam" everytime, not because you
don't like typing, but because you don't want the clutter, so you
define a "sin" method in Spam:
def sin(self): return sin(self,self.class)
so then you could do:
bar=foo.sin()
I realize that this is not quite as pretty as the first version (or is
it?), but it feels cleaner overall to me. In the case of
multi-parameter functions, it will be clear which object (the
reciever of the message) determines the return type.
(snip)
> One last question on installation:
>
> I plan to package this up as an addition to the standard python
> distribution, therefore, installation will consist of adding files to
> Modules, Lib, and Include. Does this strike anybody as an
> unreasonable thing to do?
For now, it's OK, but the released version should be a stand-alone module.
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Fri Jan 19 16:39:18 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Fri, 19 Jan 96 11:39:18 EST
Subject: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully)
In-Reply-To: <9601191117.ZM12373@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov)
Message-ID: <9601191639.AA13724@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: "Jim Fulton, U.S. Geological Survey" <jfulton@usgs.gov>
> The two special data members "__array__", and "__return_constructor__"
> will be supported for python objects that want to pretend to be an
> array.
After thinking about this a bit, I'm quite comformable with the
__array__ member idea, because there is a precedent with __float__,
__int__, and __long__. However, I have a *strong* opinion that
__array__ should be a member *function* that returns an instance as an
array. This is in keeping with __float__, __long__, etc. If a
user-defined type wants to store it's data in an array data member and
return the member, it can, but other user-defined classes may want to
compute and return array data on demand.
This makes sense.
I would also recomment that the __array__ member function should have
an optional type code argument so that an extension that wants, say, a
double precision array can request one.
I don't agree with this. I really think that the __array__ member
function should return whatever the internal array is used by the
object, and let the extension convert it as it sees fit.
With an __array__ member function, I could, for example, immagine database
table objects that extracted their numeric data as an array on demand.
This could be quite useful.
This sounds useful.
I'm not so comfortable with the __return_constructor__ idea. It seems
to only make sense for single-argument functions. I have an alternate
proposal. Suppose that certain functions allowed an optional
(keyword?) argument that provided this constructor. So, for example,
instead of:
bar=sin(foo)
where bar and foo are both Spams (e.g. Arrays) you would have:
bar=sin(foo,Spam)
Now, you don't want to have to type ",Spam" everytime, not because you
don't like typing, but because you don't want the clutter, so you
define a "sin" method in Spam:
def sin(self): return sin(self,self.class)
so then you could do:
bar=foo.sin()
I realize that this is not quite as pretty as the first version (or is
it?), but it feels cleaner overall to me. In the case of
multi-parameter functions, it will be clear which object (the
reciever of the message) determines the return type.
This is actually the version that I currently have running (more or
less) it can infact be implemented entirely in python as a wrapper
around the underlying math functions. My principle problem with this
approach is that it requires me to know about all of the available
numeric functions and define equivalents for them whenever I define a
new Spam. This just seems unreasonable to me.
For example, let's say I make a OneIndexArray (FORTRAN style indexes
start at 1). Obviously, I want this array to work with all of the
existing operations on arrays, and to return a OneIndexArray back.
I think that Jim F sees a problem with
power(OneIndexArray, Matrix).
I would simply have this function call raise an exception saying that
it was called with arrays having different __return_constructor__'s.
> One last question on installation:
>
> I plan to package this up as an addition to the standard python
> distribution, therefore, installation will consist of adding files to
> Modules, Lib, and Include. Does this strike anybody as an
> unreasonable thing to do?
For now, it's OK, but the released version should be a stand-alone module.
Maybe I'll let somebody else take care of these final packaging details.
Jim
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Fri Jan 19 16:58:21 1996
From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey)
Date: Fri, 19 Jan 1996 11:58:21 -0500
Subject: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully)
In-Reply-To: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
"Re: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully)" (Jan 19, 11:39am)
References: <9601191639.AA13724@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <9601191158.ZM12402@dsdbqvarsa.er.usgs.gov>
On Jan 19, 11:39am, James Hugunin wrote:
> Subject: Re: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully)
>
> From: "Jim Fulton, U.S. Geological Survey" <jfulton@usgs.gov>
>
> > The two special data members "__array__", and "__return_constructor__"
> > will be supported for python objects that want to pretend to be an
> > array.
>
> After thinking about this a bit, I'm quite comformable with the
> __array__ member idea, because there is a precedent with __float__,
> __int__, and __long__. However, I have a *strong* opinion that
> __array__ should be a member *function* that returns an instance as an
> array. This is in keeping with __float__, __long__, etc. If a
> user-defined type wants to store it's data in an array data member and
> return the member, it can, but other user-defined classes may want to
> compute and return array data on demand.
>
> This makes sense.
>
> I would also recomment that the __array__ member function should have
> an optional type code argument so that an extension that wants, say, a
> double precision array can request one.
>
> I don't agree with this. I really think that the __array__ member
> function should return whatever the internal array is used by the
> object, and let the extension convert it as it sees fit.
I don't feel too strongly about this second point, however,
I suspect that there could be many interesting classes that don't store their
data in arrays at all. These will have to make some decision about what to
return and this could cause an extra array to be allocated. Also, having these
conversion functions take a type argument is sort of symetric with the type
argument in the array constructor.
>
> With an __array__ member function, I could, for example, immagine database
> table objects that extracted their numeric data as an array on demand.
> This could be quite useful.
>
> This sounds useful.
>
> I'm not so comfortable with the __return_constructor__ idea. It seems
> to only make sense for single-argument functions. I have an alternate
> proposal. Suppose that certain functions allowed an optional
> (keyword?) argument that provided this constructor. So, for example,
> instead of:
>
> bar=sin(foo)
>
> where bar and foo are both Spams (e.g. Arrays) you would have:
>
> bar=sin(foo,Spam)
>
> Now, you don't want to have to type ",Spam" everytime, not because you
> don't like typing, but because you don't want the clutter, so you
> define a "sin" method in Spam:
>
> def sin(self): return sin(self,self.class)
>
> so then you could do:
>
> bar=foo.sin()
>
> I realize that this is not quite as pretty as the first version (or is
> it?), but it feels cleaner overall to me. In the case of
> multi-parameter functions, it will be clear which object (the
> reciever of the message) determines the return type.
>
> This is actually the version that I currently have running (more or
> less) it can infact be implemented entirely in python as a wrapper
> around the underlying math functions. My principle problem with this
> approach is that it requires me to know about all of the available
> numeric functions and define equivalents for them whenever I define a
> new Spam. This just seems unreasonable to me.
>
> For example, let's say I make a OneIndexArray (FORTRAN style indexes
> start at 1). Obviously, I want this array to work with all of the
> existing operations on arrays, and to return a OneIndexArray back.
>
> I think that Jim F sees a problem with
Actually, I'm having more of a "gut" reaction.
>
> power(OneIndexArray, Matrix).
>
> I would simply have this function call raise an exception saying that
> it was called with arrays having different __return_constructor__'s.
I think I'd rather have this thing return an array in this case. I guess I
think it would be better to make this explicit, like:
OneIndexArray.power(Matrix)
or like:
OneIndexArray(power(OneIndexArray, Matrix))
>
> > One last question on installation:
> >
> > I plan to package this up as an addition to the standard python
> > distribution, therefore, installation will consist of adding files to
> > Modules, Lib, and Include. Does this strike anybody as an
> > unreasonable thing to do?
>
> For now, it's OK, but the released version should be a stand-alone module.
>
> Maybe I'll let somebody else take care of these final packaging details.
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Fri Jan 19 16:58:58 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Fri, 19 Jan 1996 11:58:58 -0500
Subject: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully)
In-Reply-To: <9601191543.AA13543@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199601191658.LAA12068@cyclone.ERE.UMontreal.CA>
This C type WILL continue to implement automatic type coercion, but I
reserve the right to remove it if I continue to have problems making
my floating point code work in floats instead of doubles.
But at some point it will have to be stable!
4) Two python object UserArray.py and Matrix.py where Matrix
inherits from UserArray.
I wonder about the meaning of "User" in the name "UserArray". Does it
mean "the arrays that users really want"? My first idea when seeing
such a name would be "user-defined arrays", which is not at all what
we are talking about. I still propose to explore whether it is really
definitely impossible to use the name "Array", but if that is the
case, then something like "NumericArray" would make more sense than
"UserArray".
I plan to package this up as an addition to the standard python
distribution, therefore, installation will consist of adding files to
Modules, Lib, and Include. Does this strike anybody as an
unreasonable thing to do?
It's probably the only reasonable thing to do for now. When my patches
are part of the standard distribution (i.e. in Python 1.4), it should
be possible to package everything as a standard extension.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Fri Jan 19 17:03:00 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Fri, 19 Jan 1996 12:03:00 -0500
Subject: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully)
In-Reply-To: <9601191117.ZM12373@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov)
Message-ID: <199601191703.MAA13454@cyclone.ERE.UMontreal.CA>
After thinking about this a bit, I'm quite comformable with the
__array__ member idea, because there is a precedent with __float__,
__int__, and __long__. However, I have a *strong* opinion that
__array__ should be a member *function* that returns an instance as an
array. This is in keeping with __float__, __long__, etc. If a
user-defined type wants to store it's data in an array data member and
return the member, it can, but other user-defined classes may want to
compute and return array data on demand.
I agree that this would be better, but I worry about efficiency.
Function calls in Python are notoriously slow and should be avoided.
Now, you don't want to have to type ",Spam" everytime, not because you
don't like typing, but because you don't want the clutter, so you
define a "sin" method in Spam:
def sin(self): return sin(self,self.class)
so then you could do:
bar=foo.sin()
I realize that this is not quite as pretty as the first version (or is
it?), but it feels cleaner overall to me. In the case of
Actually, when you import "sin" from umath, sin(foo) will automatically
be translated into foo.sin(). Again, I worry about the additional
overhead of a Python function call.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Fri Jan 19 17:36:30 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Fri, 19 Jan 96 12:36:30 EST
Subject: [PYTHON MATRIX-SIG] Finalized 0.3 interface (hopefully)
In-Reply-To: <199601191658.LAA12068@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA)
Message-ID: <9601191736.AA14476@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Date: Fri, 19 Jan 1996 11:58:58 -0500
From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad)
This C type WILL continue to implement automatic type coercion,
but I reserve the right to remove it if I continue to have
problems making my floating point code work in floats instead of
doubles.
But at some point it will have to be stable!
I agree. It will be stable in the BETA1 release. I'll use release
0.30 to see how well things work for me.
4) Two python object UserArray.py and Matrix.py where Matrix
inherits from UserArray.
I wonder about the meaning of "User" in the name "UserArray". Does
it mean "the arrays that users really want"? My first idea when
seeing such a name would be "user-defined arrays", which is not at
all what we are talking about. I still propose to explore whether
it is really definitely impossible to use the name "Array", but if
that is the case, then something like "NumericArray" would make
more sense than "UserArray".
This is just an extension of the current UserList.py and
UserDictionary.py names in the standard python distribution to refer
to python classes that are derived from C-types. The principle
purpose for these classes (and for UserArray as well) is to be used as
a superclass for python defined subclasses. I don't think this
convention is too bad, so I think it's best to stay consistent.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dubois1@llnl.gov Fri Jan 19 19:29:04 1996
From: dubois1@llnl.gov (Paul. Dubois)
Date: Fri, 19 Jan 1996 11:29:04 -0800
Subject: [PYTHON MATRIX-SIG] Attributes vs. methods
References: <9601191639.AA13724@baribal.LCS.MIT.EDU.LCS.MIT.EDU> <9601191158.ZM12402@dsdbqvarsa.er.usgs.gov>
Message-ID: <30FFF100.7203@llnl.gov>
Pardon in advance if this gets too pedantic; I figured the range of OOP
experience varies so I go somewhat slow. The question before the SIG is
whether, for example, it should be x.shape or x.shape().
First, speaking in abstract OOP terms, a class can have two kinds of
members, attributes and methods. The former is data, the latter is
executable. In some languages (but not all) you can tell which kind of
access is being made just by looking at it:
x.f -- f is data
x.f() -- f is a method
(Aside: this is not a GoodThing. It forces the implementor of a class to
commit to a representation for f, ie. to decide once and for all how
clients will refer to f. For example, if f represents a certain
quantity, say pressure, a decision is made immediately as to whether f
is to be a primary state variable or whether to calculate it from
something else, like the temperature. It is better if a data attribute
is indistinguishable from the client's point of view from a function
that takes no arguments. Eiffel has this right.)
That said, one then has to ask whether a certain language allows
statements of the form
x.f = something
If it does, then maintaining the object as an abstract data type become
difficult. For example, we may have two attributes x and y and we want
to ensure a certain relationship always holds between them. Obviously,
if any old user can come along and assign something to x without
changing y, we have problems. Thus, in many languages that allow
assignment to x.f, books are written advising that all such f's be
hidden (private) and that a function x.f() be supplied to the users
instead. Then one worries about inlining, etc.
In Python, the situation is more complicated. There is no "private". So,
a convention has developed that if an attribute has a name beginning
with an underscore, then the warranty on the object is void if the
client assigns to it. This is a reasonable compromise in that the usual
strategy of having an f() and a set_f(value) may suffer some performance
penalty since inlining isn't going to be done, and anyway there is no
private data anyway.
Next, Python allows one to make it appear that f is an attribute even if
it is not, so that while x.f "works" there really isn't such a data
member and x.f = something would not be setting it (and may even fail,
depending on the way the class was written). However, learning about
this trick implies a level of sophistication beyond the level to be
expected in the user of an application in which Python is used as the
scripting language. Such a user is too likely to try to change the shape
of the matrix by assignment.
So, my conclusion is that if x.shape in the matrix class is an
attribute, and we don't want any user to do x.shape=something, we should
call it _shape, and supply an x.shape() for normal access. I think this
argument applies to the level intended for use by the general public.
They need a simple, consistent set of rules to live by.
--
Paul F. Dubois, L-472 (510)-422-5426
Lawrence Livermore National Laboratory FAX (510)-423-9969
Livermore, CA 94550 dubois1@llnl.gov
Consulting: PaulDubois@aol.com
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Fri Jan 19 19:53:55 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Fri, 19 Jan 96 14:53:55 EST
Subject: [PYTHON MATRIX-SIG] Attributes vs. methods
In-Reply-To: <30FFF100.7203@llnl.gov> (dubois1@llnl.gov)
Message-ID: <9601191953.AA15133@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Your last line made me realize that there is essentially a bug in the
current implementation of the matrix class (I just wasn't thinking
clearly) and this might be coloring the analysis.
> So, my conclusion is that if x.shape in the matrix class is an
> attribute, and we don't want any user to do x.shape=something, we should
> call it _shape, and supply an x.shape() for normal access.
For types written in C (as the array type is) access times between
attributes and function calls are really indistinguishable, so the
issue is whether to use x.shape() to return the current shape and
x.reshape(new_shape) to set a new one, or to use x.shape and
x.shape=new_shape instead (or maybe both). The fact that you can't do
x.shape=new_shape in the current implementation is more an accident
than a matter of design. It would be very easy to make this work
properly (using the set_attribute facilities of python).
Here's what I propose (after thinking about things one more time):
a.typecode, a.itemsize, a.contiguous should all become methods,
because setting these values makes no sense.
a.shape, a.real, and a.imaginary should be implemented as pseudo-data in
a self-consistent manner. This means that I'll fix the code so that
a.shape = new_shape will work properly.
Does this sound reasonable?
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Fri Jan 19 20:01:56 1996
From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey)
Date: Fri, 19 Jan 1996 15:01:56 -0500
Subject: [PYTHON MATRIX-SIG] Attributes vs. methods
In-Reply-To: "Paul. Dubois" <dubois1@llnl.gov>
"[PYTHON MATRIX-SIG] Attributes vs. methods" (Jan 19, 11:29am)
References: <9601191639.AA13724@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
<9601191158.ZM12402@dsdbqvarsa.er.usgs.gov>
<30FFF100.7203@llnl.gov>
Message-ID: <9601191501.ZM13147@dsdbqvarsa.er.usgs.gov>
On Jan 19, 11:29am, Paul. Dubois wrote:
> Subject: [PYTHON MATRIX-SIG] Attributes vs. methods
> Pardon in advance if this gets too pedantic; I figured the range of OOP
> experience varies so I go somewhat slow. The question before the SIG is
> whether, for example, it should be x.shape or x.shape().
>
> First, speaking in abstract OOP terms, a class can have two kinds of
> members, attributes and methods. The former is data, the latter is
> executable. In some languages (but not all) you can tell which kind of
> access is being made just by looking at it:
> x.f -- f is data
> x.f() -- f is a method
>
> (Aside: this is not a GoodThing. It forces the implementor of a class to
> commit to a representation for f, ie. to decide once and for all how
> clients will refer to f. For example, if f represents a certain
> quantity, say pressure, a decision is made immediately as to whether f
> is to be a primary state variable or whether to calculate it from
> something else, like the temperature. It is better if a data attribute
> is indistinguishable from the client's point of view from a function
> that takes no arguments. Eiffel has this right.)
>
> That said, one then has to ask whether a certain language allows
> statements of the form
>
> x.f = something
>
> If it does, then maintaining the object as an abstract data type become
> difficult. For example, we may have two attributes x and y and we want
> to ensure a certain relationship always holds between them. Obviously,
> if any old user can come along and assign something to x without
> changing y, we have problems. Thus, in many languages that allow
> assignment to x.f, books are written advising that all such f's be
> hidden (private) and that a function x.f() be supplied to the users
> instead. Then one worries about inlining, etc.
>
> In Python, the situation is more complicated. There is no "private".
All data in built-in types is private. Data in class instances can be made
semi-private, with some effort. Actually, I hope that in the future there will
be alternate class types that support features such as private attributes.
> So,
> a convention has developed that if an attribute has a name beginning
> with an underscore, then the warranty on the object is void if the
> client assigns to it. This is a reasonable compromise in that the usual
> strategy of having an f() and a set_f(value) may suffer some performance
> penalty since inlining isn't going to be done, and anyway there is no
> private data anyway.
>
> Next, Python allows one to make it appear that f is an attribute even if
> it is not, so that while x.f "works" there really isn't such a data
> member and x.f = something would not be setting it (and may even fail,
> depending on the way the class was written). However, learning about
> this trick implies a level of sophistication beyond the level to be
> expected in the user of an application in which Python is used as the
> scripting language. Such a user is too likely to try to change the shape
> of the matrix by assignment.
This argument assumes that the unsophisticated user is sophisticated enough to
worry about the implementation in the first place.
> So, my conclusion is that if x.shape in the matrix class is an
> attribute, and we don't want any user to do x.shape=something, we should
I'd be inclined to think about this a little differently. I think it is
reasonable for an interface to expose properties, where properties represent
information that you can query and set for the object, without regard to
implementation.
I'd be inclined to agree that if you don't want to allow:
x.shape = something
then shape should probably not be modeled as an attribute, however,
these same argument might be used to say that if you can say:
"spam"[0]
you should also able to say:
"spam"[0]="S"
> call it _shape, and supply an x.shape() for normal access. I think this
> argument applies to the level intended for use by the general public.
> They need a simple, consistent set of rules to live by.
I'm not really disagreeing with you, but I find the idea of abstract attributes
to be interesting.
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Fri Jan 19 19:53:55 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Fri, 19 Jan 96 14:53:55 EST
Subject: [PYTHON MATRIX-SIG] Attributes vs. methods
In-Reply-To: <30FFF100.7203@llnl.gov> (dubois1@llnl.gov)
Message-ID: <9601191953.AA15133@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Your last line made me realize that there is essentially a bug in the
current implementation of the matrix class (I just wasn't thinking
clearly) and this might be coloring the analysis.
> So, my conclusion is that if x.shape in the matrix class is an
> attribute, and we don't want any user to do x.shape=something, we should
> call it _shape, and supply an x.shape() for normal access.
For types written in C (as the array type is) access times between
attributes and function calls are really indistinguishable, so the
issue is whether to use x.shape() to return the current shape and
x.reshape(new_shape) to set a new one, or to use x.shape and
x.shape=new_shape instead (or maybe both). The fact that you can't do
x.shape=new_shape in the current implementation is more an accident
than a matter of design. It would be very easy to make this work
properly (using the set_attribute facilities of python).
Here's what I propose (after thinking about things one more time):
a.typecode, a.itemsize, a.contiguous should all become methods,
because setting these values makes no sense.
a.shape, a.real, and a.imaginary should be implemented as pseudo-data in
a self-consistent manner. This means that I'll fix the code so that
a.shape = new_shape will work properly.
Does this sound reasonable?
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Fri Jan 19 20:01:56 1996
From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey)
Date: Fri, 19 Jan 1996 15:01:56 -0500
Subject: [PYTHON MATRIX-SIG] Attributes vs. methods
In-Reply-To: "Paul. Dubois" <dubois1@llnl.gov>
"[PYTHON MATRIX-SIG] Attributes vs. methods" (Jan 19, 11:29am)
References: <9601191639.AA13724@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
<9601191158.ZM12402@dsdbqvarsa.er.usgs.gov>
<30FFF100.7203@llnl.gov>
Message-ID: <9601191501.ZM13147@dsdbqvarsa.er.usgs.gov>
On Jan 19, 11:29am, Paul. Dubois wrote:
> Subject: [PYTHON MATRIX-SIG] Attributes vs. methods
> Pardon in advance if this gets too pedantic; I figured the range of OOP
> experience varies so I go somewhat slow. The question before the SIG is
> whether, for example, it should be x.shape or x.shape().
>
> First, speaking in abstract OOP terms, a class can have two kinds of
> members, attributes and methods. The former is data, the latter is
> executable. In some languages (but not all) you can tell which kind of
> access is being made just by looking at it:
> x.f -- f is data
> x.f() -- f is a method
>
> (Aside: this is not a GoodThing. It forces the implementor of a class to
> commit to a representation for f, ie. to decide once and for all how
> clients will refer to f. For example, if f represents a certain
> quantity, say pressure, a decision is made immediately as to whether f
> is to be a primary state variable or whether to calculate it from
> something else, like the temperature. It is better if a data attribute
> is indistinguishable from the client's point of view from a function
> that takes no arguments. Eiffel has this right.)
>
> That said, one then has to ask whether a certain language allows
> statements of the form
>
> x.f = something
>
> If it does, then maintaining the object as an abstract data type become
> difficult. For example, we may have two attributes x and y and we want
> to ensure a certain relationship always holds between them. Obviously,
> if any old user can come along and assign something to x without
> changing y, we have problems. Thus, in many languages that allow
> assignment to x.f, books are written advising that all such f's be
> hidden (private) and that a function x.f() be supplied to the users
> instead. Then one worries about inlining, etc.
>
> In Python, the situation is more complicated. There is no "private".
All data in built-in types is private. Data in class instances can be made
semi-private, with some effort. Actually, I hope that in the future there will
be alternate class types that support features such as private attributes.
> So,
> a convention has developed that if an attribute has a name beginning
> with an underscore, then the warranty on the object is void if the
> client assigns to it. This is a reasonable compromise in that the usual
> strategy of having an f() and a set_f(value) may suffer some performance
> penalty since inlining isn't going to be done, and anyway there is no
> private data anyway.
>
> Next, Python allows one to make it appear that f is an attribute even if
> it is not, so that while x.f "works" there really isn't such a data
> member and x.f = something would not be setting it (and may even fail,
> depending on the way the class was written). However, learning about
> this trick implies a level of sophistication beyond the level to be
> expected in the user of an application in which Python is used as the
> scripting language. Such a user is too likely to try to change the shape
> of the matrix by assignment.
This argument assumes that the unsophisticated user is sophisticated enough to
worry about the implementation in the first place.
> So, my conclusion is that if x.shape in the matrix class is an
> attribute, and we don't want any user to do x.shape=something, we should
I'd be inclined to think about this a little differently. I think it is
reasonable for an interface to expose properties, where properties represent
information that you can query and set for the object, without regard to
implementation.
I'd be inclined to agree that if you don't want to allow:
x.shape = something
then shape should probably not be modeled as an attribute, however,
these same argument might be used to say that if you can say:
"spam"[0]
you should also able to say:
"spam"[0]="S"
> call it _shape, and supply an x.shape() for normal access. I think this
> argument applies to the level intended for use by the general public.
> They need a simple, consistent set of rules to live by.
I'm not really disagreeing with you, but I find the idea of abstract attributes
to be interesting.
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Fri Jan 19 20:36:12 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Fri, 19 Jan 1996 15:36:12 -0500
Subject: [PYTHON MATRIX-SIG] Attributes vs. methods
In-Reply-To: <9601191953.AA15133@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199601192036.PAA25872@cyclone.ERE.UMontreal.CA>
a.typecode, a.itemsize, a.contiguous should all become methods,
because setting these values makes no sense.
They could also be read-only attributes. BTW, setting a.typecode
does make sense, as a way of casting an array to another type.
Of course this is not the recommended style.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Fri Jan 19 21:20:44 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Fri, 19 Jan 96 16:20:44 EST
Subject: [PYTHON MATRIX-SIG] Attributes vs. methods
In-Reply-To: <199601192036.PAA25872@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA)
Message-ID: <9601192120.AA15839@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Date: Fri, 19 Jan 1996 15:36:12 -0500
From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad)
Cc: dubois1@llnl.gov, matrix-sig@python.org
a.typecode, a.itemsize, a.contiguous should all become methods,
because setting these values makes no sense.
They could also be read-only attributes. BTW, setting a.typecode
does make sense, as a way of casting an array to another type.
Of course this is not the recommended style.
Unfortunately, this can not be done with the current implementation of
array objects. Because it is possible to have arrays that are
references to the memory of another array, it is essential that
casting produce a new array rather than modify the existing one.
Right now they are read-only attributes, but I think I've come to
agree with Paul that read-only attributes are a little bit confusing.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Fri Jan 19 21:25:24 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Fri, 19 Jan 1996 16:25:24 -0500
Subject: [PYTHON MATRIX-SIG] Evaluation release of expression compiler
Message-ID: <199601192125.QAA28242@cyclone.ERE.UMontreal.CA>
For those of you who still don't know what to do during the weekend, I
offer a nice toy. In the next message, I will send the source code for
the very first incomplete, non-optimized version of my numerical
expression compiler. It is a single C extension module called
"numexprmodule.c". For now, both this module and the module "cmath"
must be statically linked with the interpreter, since numexpr calls
functions from cmath.
Here's an example for an application:
--------------------------------------------------
from numexpr import *
from omath import *
def f(x):
return sin(x*x)+0.5*sqrt(x/7)*cos(2/x)*log(x+exp(-x))-sqrt(2)*tanh(x)
f_compiled = f(FloatVariable())
print f(5)
print f_compiled(5)
--------------------------------------------------
The module "numexpr" exports three functions: IntegerVariable,
FloatVariable, and ComplexVariable. Each of these takes an
optional argument (default is zero) that indicates
the position of this variable in the argument list of the
compiled expression. So if you want a compiled expression
for "x+y", write
f = FloatVariable(0)+FloatVariable(1)
The range of mathematical functions is the same as for cmath. All
arithmetic operators are supported, with exception of exponentiation,
which with the current coercion scheme cannot be implemented
efficiently, so I decided to wait a bit. With that restriction, all
functions that contain no conditional or loop statements can be
compiled.
Speed: First tests have shown that the asymptotic speed (i.e. for
infinitely long functions) relative to uncompiled functions is about
40. For one-line expressions such as the one in the example above,
expect a factor of 7 (which shows that the overhead if the interpreter
is still substantial). Note that the expressions are not optimized
in any way. If you compile something like
temp = sin(x+y)
f = temp*temp,
then sin(x+y) will actually be calculated twice at execution time. So
elimination of common subexpressions is the very least amount of
optimization that will have to be done.
To do:
- Control structures
- Optimization
- Array expressions
I'd appreciate any comments whatsoever.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Fri Jan 19 21:25:48 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Fri, 19 Jan 1996 16:25:48 -0500
Subject: [PYTHON MATRIX-SIG] numexprmodule.c
Message-ID: <199601192125.QAA28264@cyclone.ERE.UMontreal.CA>
/* Numexpr objects */
#include "allobjects.h"
#include <math.h>
/* Type of stack entries */
typedef union { long i; double f; complex c; } stack_entry;
/* Error object */
static PyObject *ErrorObject;
/* Object definition */
typedef struct {
PyObject_HEAD
int nargs; /* number of parameters */
char *argtypes; /* types of parameters */
stack_entry *args; /* array to store actual arguments */
int nconstants; /* number of constants */
stack_entry *constants; /* constant array */
int codesize; /* number of code words */
int *code; /* compiled expression */
int stacksize; /* stack size needed for evaluation */
stack_entry *stack; /* execution stack */
char type; /* return type of the expression */
} PyNumExprObject;
staticforward PyTypeObject PyNumExpr_Type;
#define PyNumExpr_Check(v) ((v)->ob_type == &PyNumExpr_Type)
/* Type codes */
#define INTEGER 1
#define FLOAT 2
#define COMPLEX 3
#define NTYPES 4
/* Opcodes used in compiled code */
#define GET_ARG 1
#define GET_CONSTANT 2
#define ADD 10
#define ADD_INT 11
#define ADD_FLOAT 12
#define ADD_COMPLEX 13
#define SUB 15
#define SUB_INT 16
#define SUB_FLOAT 17
#define SUB_COMPLEX 18
#define MUL 20
#define MUL_INT 21
#define MUL_FLOAT 22
#define MUL_COMPLEX 23
#define DIV 25
#define DIV_INT 26
#define DIV_FLOAT 27
#define DIV_COMPLEX 28
#define POW 30
#define NEG 40
#define NEG_INT 41
#define NEG_FLOAT 42
#define NEG_COMPLEX 43
#define ABS 45
#define ABS_INT 46
#define ABS_FLOAT 47
#define ABS_COMPLEX 48
#define TYPE_CAST 50
#define INTEGER_FLOAT 59
#define INTEGER_COMPLEX 63
#define FLOAT_INTEGER 56
#define FLOAT_COMPLEX 64
#define COMPLEX_INTEGER 57
#define COMPLEX_FLOAT 61
#define ACOS_FLOAT 100
#define ACOS_COMPLEX 101
#define ACOSH_FLOAT 102
#define ACOSH_COMPLEX 103
#define ASIN_FLOAT 104
#define ASIN_COMPLEX 105
#define ASINH_FLOAT 106
#define ASINH_COMPLEX 107
#define ATAN_FLOAT 108
#define ATAN_COMPLEX 109
#define ATANH_FLOAT 110
#define ATANH_COMPLEX 111
#define COS_FLOAT 112
#define COS_COMPLEX 113
#define COSH_FLOAT 114
#define COSH_COMPLEX 115
#define EXP_FLOAT 116
#define EXP_COMPLEX 117
#define LOG_FLOAT 118
#define LOG_COMPLEX 119
#define LOG10_FLOAT 120
#define LOG10_COMPLEX 121
#define SIN_FLOAT 122
#define SIN_COMPLEX 123
#define SINH_FLOAT 124
#define SINH_COMPLEX 125
#define SQRT_FLOAT 126
#define SQRT_COMPLEX 127
#define TAN_FLOAT 128
#define TAN_COMPLEX 129
#define TANH_FLOAT 130
#define TANH_COMPLEX 131
/* Utility functions */
#define max(a,b) (((a) > (b)) ? (a) : (b))
#define min(a,b) (((a) < (b)) ? (a) : (b))
/* Allocation and deallocation of numexpr objects */
static PyNumExprObject *
new_numexprobject()
{
PyNumExprObject *self;
self = NEWOBJ(PyNumExprObject, &PyNumExpr_Type);
if (self == NULL)
return NULL;
self->nargs = 0;
self->argtypes = NULL;
self->args = NULL;
self->nconstants = 0;
self->constants = NULL;
self->codesize = 0;
self->code = NULL;
self->stacksize = 0;
self->stack = NULL;
self->type = 0;
return self;
}
static void
numexpr_dealloc(self)
PyNumExprObject *self;
{
PyMem_XDEL(self->argtypes);
PyMem_XDEL(self->args);
PyMem_XDEL(self->constants);
PyMem_XDEL(self->code);
PyMem_XDEL(self->stack);
PyMem_DEL(self);
}
/* Attribute access. The only attributes are methods. */
struct PyMethodDef numexpr_methods[];
static PyObject *
numexpr_getattr(self, name)
PyNumExprObject *self;
char *name;
{
return Py_FindMethod(numexpr_methods, (PyObject *)self, name);
}
/* Printed representation */
static PyObject *
numexpr_repr(self)
PyNumExprObject *self;
{
char buf[100];
sprintf(buf, "<compiled expression (%s)>",
self->argtypes);
return PyString_FromString(buf);
}
/* Evaluation of a compiled expression */
extern complex c_sqrt();
extern complex c_acos();
extern complex c_acosh();
extern complex c_asin();
extern complex c_asinh();
extern complex c_atan();
extern complex c_atanh();
extern complex c_cos();
extern complex c_cosh();
extern complex c_exp();
extern complex c_log();
extern complex c_log10();
extern complex c_sin();
extern complex c_sinh();
extern complex c_sqrt();
extern complex c_tan();
extern complex c_tanh();
static PyObject *
numexpr_call(self, args)
PyNumExprObject *self;
PyObject *args;
{
stack_entry *stack = self->stack;
int *cp, *end;
char text[50];
stack_entry *sp;
PyObject *arg;
char argtype[2];
PyObject *result;
int i;
if (PyObject_Length(args) != self->nargs) {
PyErr_SetString(TypeError,
"wrong number of arguments in compiled expression");
return NULL;
}
argtype[1] = '\0';
for (i = 0; i < self->nargs; i++)
{
arg = PySequence_GetItem(args, i);
argtype[0] = self->argtypes[i];
if (argtype[0] == ' ')
PyArg_Parse(arg, "O", &arg);
else
PyArg_Parse(arg, argtype, self->args+i);
}
end = self->code + self->codesize;
cp = self->code;
sp = stack-1;
while (cp < end) {
switch (*cp) {
case GET_ARG:
cp++;
*++sp = self->args[*cp];
break;
case GET_CONSTANT:
cp++;
*++sp = self->constants[*cp];
break;
case ADD_INT:
sp[-1].i += sp[0].i;
sp--;
break;
case ADD_FLOAT:
sp[-1].f += sp[0].f;
sp--;
break;
case ADD_COMPLEX:
sp[-1].c.real += sp[0].c.real;
sp[-1].c.imag += sp[0].c.imag;
sp--;
break;
case SUB_INT:
sp[-1].i -= sp[0].i;
sp--;
break;
case SUB_FLOAT:
sp[-1].f -= sp[0].f;
sp--;
break;
case SUB_COMPLEX:
sp[-1].c.real -= sp[0].c.real;
sp[-1].c.imag -= sp[0].c.imag;
sp--;
break;
case MUL_INT:
sp[-1].i *= sp[0].i;
sp--;
break;
case MUL_FLOAT:
sp[-1].f *= sp[0].f;
sp--;
break;
case MUL_COMPLEX:
sp[-1].c = c_prod(sp[-1].c, sp[0].c);
sp--;
break;
case DIV_INT:
sp[-1].i /= sp[0].i;
sp--;
break;
case DIV_FLOAT:
sp[-1].f /= sp[0].f;
sp--;
break;
case DIV_COMPLEX:
sp[-1].c = c_quot(sp[-1].c, sp[0].c);
sp--;
break;
case INTEGER_FLOAT:
sp->f = sp->i;
break;
case INTEGER_COMPLEX:
sp->c.real = sp->i;
sp->c.imag = 0.;
break;
case FLOAT_INTEGER:
sp->i = sp->f;
break;
case FLOAT_COMPLEX:
sp->c.real = sp->f;
sp->c.imag = 0.;
break;
case COMPLEX_INTEGER:
sp->i = sp->c.real;
break;
case COMPLEX_FLOAT:
sp->f = sp->c.real;
break;
case NEG_INT:
sp->i = -sp->i;
break;
case NEG_FLOAT:
sp->f = -sp->f;
break;
case NEG_COMPLEX:
sp->c.real = -sp->c.real;
sp->c.imag = -sp->c.imag;
break;
case ABS_INT:
if (sp->i < 0)
sp->i = -sp->i;
break;
case ABS_FLOAT:
if (sp->f < 0)
sp->f = -sp->f;
break;
case ABS_COMPLEX:
sp->f = hypot(sp->c.real, sp->c.imag);
break;
case ACOS_FLOAT:
sp->f = acos(sp->f);
break;
case ACOS_COMPLEX:
sp->c = c_acos(sp->c);
break;
case ACOSH_FLOAT:
sp->f = acosh(sp->f);
break;
case ACOSH_COMPLEX:
sp->c = c_acosh(sp->c);
break;
case ASIN_FLOAT:
sp->f = asin(sp->f);
break;
case ASIN_COMPLEX:
sp->c = c_asin(sp->c);
break;
case ASINH_FLOAT:
sp->f = asinh(sp->f);
break;
case ASINH_COMPLEX:
sp->c = c_asinh(sp->c);
break;
case ATAN_FLOAT:
sp->f = atan(sp->f);
break;
case ATAN_COMPLEX:
sp->c = c_atan(sp->c);
break;
case ATANH_FLOAT:
sp->f = atanh(sp->f);
break;
case ATANH_COMPLEX:
sp->c = c_atanh(sp->c);
break;
case COS_FLOAT:
sp->f = cos(sp->f);
break;
case COS_COMPLEX:
sp->c = c_cos(sp->c);
break;
case COSH_FLOAT:
sp->f = cosh(sp->f);
break;
case COSH_COMPLEX:
sp->c = c_cosh(sp->c);
break;
case EXP_FLOAT:
sp->f = exp(sp->f);
break;
case EXP_COMPLEX:
sp->c = c_exp(sp->c);
break;
case LOG_FLOAT:
sp->f = log(sp->f);
break;
case LOG_COMPLEX:
sp->c = c_log(sp->c);
break;
case LOG10_FLOAT:
sp->f = log10(sp->f);
break;
case LOG10_COMPLEX:
sp->c = c_log10(sp->c);
break;
case SIN_FLOAT:
sp->f = sin(sp->f);
break;
case SIN_COMPLEX:
sp->c = c_sin(sp->c);
break;
case SINH_FLOAT:
sp->f = sinh(sp->f);
break;
case SINH_COMPLEX:
sp->c = c_sinh(sp->c);
break;
case SQRT_FLOAT:
sp->f = sqrt(sp->f);
break;
case SQRT_COMPLEX:
sp->c = c_sqrt(sp->c);
break;
case TAN_FLOAT:
sp->f = tan(sp->f);
break;
case TAN_COMPLEX:
sp->c = c_tan(sp->c);
break;
case TANH_FLOAT:
sp->f = tanh(sp->f);
break;
case TANH_COMPLEX:
sp->c = c_tanh(sp->c);
break;
default:
sprintf(text, "unknown opcode %d", *cp);
PyErr_SetString(ErrorObject, text);
return NULL;
break;
}
cp++;
}
if (sp != stack) {
PyErr_SetString(ErrorObject, "stack error");
return NULL;
}
switch(self->type) {
case INTEGER:
result = (PyObject *)PyInt_FromLong(sp->i);
break;
case FLOAT:
result = (PyObject *)PyFloat_FromDouble(sp->f);
break;
case COMPLEX:
result = (PyObject *)PyComplex_FromCComplex(sp->c);
break;
default:
Py_INCREF(None);
result = None;
}
return result;
}
/* Arithmetic */
static char
numexpr_commontype(type1, type2)
char type1;
char type2;
{
return max(type1, type2);
}
static PyNumExprObject *
numexpr_binary_op(a, b, type, op)
PyNumExprObject *a;
PyNumExprObject *b;
char type;
int op;
{
PyNumExprObject *r = new_numexprobject();
int error = 0;
int i, j;
int *cp;
if (r != NULL) {
r->type = type;
r->nargs = max(a->nargs, b->nargs);
r->nconstants = a->nconstants + b->nconstants;
r->stacksize = max(a->stacksize, b->stacksize) + 1;
r->codesize = a->codesize + b->codesize
+ (a->type != type) + (b->type != type) + 1;
if ((r->argtypes = PyMem_NEW(char, r->nargs+1)) != NULL) {
for (i = 0; i < min(a->nargs, b->nargs); i++) {
if (a->argtypes[i] == ' ')
r->argtypes[i] = b->argtypes[i];
else if (b->argtypes[i] == ' ')
r->argtypes[i] = a->argtypes[i];
else if (a->argtypes[i] == b->argtypes[i])
r->argtypes[i] = a->argtypes[i];
else {
PyErr_SetString(ErrorObject,
"inconsistent argument lists");
error = 1;
}
}
for (; i < r->nargs; i++)
if (i < a->nargs)
r->argtypes[i] = a->argtypes[i];
else
r->argtypes[i] = b->argtypes[i];
r->argtypes[i] = '\0';
}
else
error = 1;
if ((r->args = PyMem_NEW(stack_entry, r->nargs)) == NULL)
error = 1;
if ((r->constants = PyMem_NEW(stack_entry, r->nconstants))
!= NULL) {
for (i = 0; i < a->nconstants; i++)
r->constants[i] = a->constants[i];
for (j = 0; j < b->nconstants; i++, j++)
r->constants[i] = b->constants[j];
}
else
error = 1;
if ((r->code = PyMem_NEW(int, r->codesize)) != NULL) {
cp = r->code;
for (i = 0; i < a->codesize; i++)
*cp++ = a->code[i];
if (a->type != type)
*cp++ = TYPE_CAST + a->type + NTYPES*type;
for (i = 0; i < b->codesize;) {
*cp++ = j = b->code[i++];
if (j == GET_CONSTANT) {
*cp++ = b->code[i++] + a->nconstants;
}
else if (j == GET_ARG) {
*cp++ = b->code[i++];
}
}
if (b->type != type)
*cp++ = TYPE_CAST + b->type + NTYPES*type;
*cp = op;
}
else
error = 1;
if ((r->stack = PyMem_NEW(stack_entry, r->stacksize)) == NULL)
error = 1;
if (error) {
numexpr_dealloc(r);
r = NULL;
}
}
return r;
}
static PyNumExprObject *
numexpr_unary_op(a, op)
PyNumExprObject *a;
int op;
{
PyNumExprObject *r = new_numexprobject();
int error = 0;
int i;
int *cp;
if (r != NULL) {
r->type = a->type;
r->nargs = a->nargs;
r->nconstants = a->nconstants;
r->stacksize = a->stacksize;
r->codesize = a->codesize + 1;
if ((r->argtypes = PyMem_NEW(char, r->nargs+1)) != NULL)
strcpy(r->argtypes, a->argtypes);
else
error = 1;
if ((r->args = PyMem_NEW(stack_entry, r->nargs)) == NULL)
error = 1;
if ((r->constants = PyMem_NEW(stack_entry, r->nconstants))
!= NULL)
for (i = 0; i < a->nconstants; i++)
r->constants[i] = a->constants[i];
else
error = 1;
if ((r->code = PyMem_NEW(int, r->codesize)) != NULL) {
cp = r->code;
for (i = 0; i < a->codesize; i++)
*cp++ = a->code[i];
*cp = op;
}
if ((r->stack = PyMem_NEW(stack_entry, r->stacksize)) == NULL)
error = 1;
if (error) {
numexpr_dealloc(r);
r = NULL;
}
}
return r;
}
static PyObject *
numexpr_add(a, b)
PyNumExprObject *a;
PyNumExprObject *b;
{
char type = numexpr_commontype(a->type, b->type);
return (PyObject *)numexpr_binary_op(a, b, type, ADD+type);
}
static PyObject *
numexpr_sub(a, b)
PyNumExprObject *a;
PyNumExprObject *b;
{
char type = numexpr_commontype(a->type, b->type);
return (PyObject *)numexpr_binary_op(a, b, type, SUB+type);
}
static PyObject *
numexpr_mul(a, b)
PyNumExprObject *a;
PyNumExprObject *b;
{
char type = numexpr_commontype(a->type, b->type);
return (PyObject *)numexpr_binary_op(a, b, type, MUL+type);
}
static PyObject *
numexpr_div(a, b)
PyNumExprObject *a;
PyNumExprObject *b;
{
char type = numexpr_commontype(a->type, b->type);
return (PyObject *)numexpr_binary_op(a, b, type, DIV+type);
}
static PyObject *
numexpr_pow(a, b)
PyNumExprObject *a;
PyNumExprObject *b;
{
char type = numexpr_commontype(a->type, b->type);
return (PyObject *)numexpr_binary_op(a, b, type, POW+type);
}
static PyObject *
numexpr_neg(a)
PyNumExprObject *a;
{
return (PyObject *)numexpr_unary_op(a, NEG+a->type);
}
static PyObject *
numexpr_pos(a)
PyNumExprObject *a;
{
Py_INCREF(a);
return (PyObject *)a;
}
static PyObject *
numexpr_abs(a)
PyNumExprObject *a;
{
PyNumExprObject *r = numexpr_unary_op(a, ABS+a->type);
if (r != NULL && r->type == COMPLEX)
r->type = FLOAT;
return (PyObject *)r;
}
static int
numexpr_nonzero(a)
PyNumExprObject *a;
{
return 1;
}
/* Type conversion */
static void
numexpr_convert(value, type_in, type_out)
stack_entry *value;
int type_in;
int type_out;
{
switch (type_in) {
case INTEGER:
if (type_out == FLOAT)
value->f = value->i;
else
value->c.real = value->i;
value->c.imag = 0;
break;
case FLOAT:
if (type_out == INTEGER)
value->i = value->f;
else
value->c.real = value->f;
value->c.imag = 0;
break;
case COMPLEX:
if (type_out == INTEGER)
value->i = value->c.real;
else
value->f = value->c.real;
break;
}
}
/* Type coercion: Numbers are converted to constant expressions */
static PyObject *
numexpr_const(value, type_in, type_out)
stack_entry value;
int type_in, type_out;
{
PyNumExprObject *rv;
int error = 0;
rv = new_numexprobject();
if (rv != NULL) {
rv->nargs = 0;
rv->nconstants = 1;
rv->codesize = 2;
rv->stacksize = 1;
rv->type = type_out;
if (type_in != type_out)
numexpr_convert(&value, type_in, type_out);
if ((rv->constants = PyMem_NEW(stack_entry, rv->nconstants))
!= NULL)
*rv->constants = value;
else
error = 1;
if ((rv->code = PyMem_NEW(int, rv->codesize)) != NULL) {
rv->code[0] = GET_CONSTANT;
rv->code[1] = 0;
}
else
error = 1;
if ((rv->stack = PyMem_NEW(stack_entry, rv->stacksize)) == NULL)
error = 1;
if (error) {
numexpr_dealloc(rv);
rv = NULL;
}
}
return (PyObject *)rv;
}
static void
numexpr_coercenumber(value, type, number, numexpr)
stack_entry value;
int type;
PyNumExprObject **number;
PyNumExprObject **numexpr;
{
int result_type = numexpr_commontype(type, (*numexpr)->type);
*number = (PyNumExprObject *)numexpr_const(value, type, result_type);
if ((*numexpr)->type != result_type)
*numexpr = numexpr_unary_op(*numexpr,
TYPE_CAST + (*numexpr)->type + NTYPES*result_type);
else
Py_INCREF(*numexpr);
}
static int
numexpr_coerce(a, b)
PyObject **a;
PyObject **b;
{
PyObject **number;
PyNumExprObject **numexpr;
stack_entry value;
if (PyNumExpr_Check(*a)) {
numexpr = (PyNumExprObject **)a;
number = b;
}
else {
numexpr = (PyNumExprObject **)b;
number = a;
}
if (PyInt_Check(*number)) {
value.i = PyInt_AsLong(*number);
numexpr_coercenumber(value, INTEGER, number, numexpr);
return 0;
}
else if (PyFloat_Check(*number)) {
value.f = PyFloat_AsDouble(*number);
numexpr_coercenumber(value, FLOAT, number, numexpr);
return 0;
}
else if (PyComplex_Check(*number)) {
value.c = PyComplex_AsCComplex(*number);
numexpr_coercenumber(value, COMPLEX, number, numexpr);
return 0;
}
else
return 1;
}
/* Module documentation string */
static char PyNumExpr_Type__doc__[] =
"Compiled numerical expressions";
/* Type definition */
static PyNumberMethods numexpr_as_number = {
(binaryfunc)numexpr_add, /*nb_add*/
(binaryfunc)numexpr_sub, /*nb_subtract*/
(binaryfunc)numexpr_mul, /*nb_multiply*/
(binaryfunc)numexpr_div, /*nb_divide*/
0, /*nb_remainder*/
0, /*nb_divmod*/
(ternaryfunc)numexpr_pow, /*nb_power*/
(unaryfunc)numexpr_neg, /*nb_negative*/
(unaryfunc)numexpr_pos, /*nb_positive*/
(unaryfunc)numexpr_abs, /*nb_absolute*/
(inquiry)numexpr_nonzero, /*nb_nonzero*/
0, /*nb_invert*/
0, /*nb_lshift*/
0, /*nb_rshift*/
0, /*nb_and*/
0, /*nb_xor*/
0, /*nb_or*/
(coercion)numexpr_coerce, /*nb_coerce*/
0, /*nb_int*/
0, /*nb_long*/
0, /*nb_float*/
0, /*nb_oct*/
0, /*nb_hex*/
};
static PyTypeObject PyNumExpr_Type = {
PyObject_HEAD_INIT(&PyType_Type)
0, /*ob_size*/
"numexpr", /*tp_name*/
sizeof(PyNumExprObject), /*tp_basicsize*/
0, /*tp_itemsize*/
/* methods */
(destructor)numexpr_dealloc, /*tp_dealloc*/
0, /*tp_print*/
(getattrfunc)numexpr_getattr, /*tp_getattr*/
0, /*tp_setattr*/
0, /*tp_compare*/
(reprfunc)numexpr_repr, /*tp_repr*/
&numexpr_as_number, /*tp_as_number*/
0, /*tp_as_sequence*/
0, /*tp_as_mapping*/
0, /*tp_hash*/
(ternaryfunc)numexpr_call, /*tp_call*/
(reprfunc)numexpr_repr, /*tp_str*/
/* Space for future expansion */
0L,0L,0L,0L,
PyNumExpr_Type__doc__ /* Documentation string */
};
/* --------------------------------------------------------------------- */
/* Return a new variable object. The (integer) argument
indicates the position of the variable in the argument
list of the compiled expression; it defaults to zero. */
static PyNumExprObject *
new_variable(args)
PyObject *args;
{
PyNumExprObject *rv;
int argnum = 0;
int error = 0;
int i;
PyArg_ParseTuple(args, "|i:numexpr variable", &argnum);
rv = new_numexprobject();
if (rv != NULL) {
rv->nargs = argnum + 1;
rv->stacksize = 1;
rv->codesize = 2;
if ((rv->argtypes = PyMem_NEW(char, rv->nargs+1)) != NULL) {
for (i = 0; i < rv->nargs; i++)
rv->argtypes[i] = ' ';
rv->argtypes[i] = '\0';
}
else
error = 1;
if ((rv->args = PyMem_NEW(stack_entry, rv->nargs)) == NULL)
error = 1;
if ((rv->code = PyMem_NEW(int, rv->codesize)) != NULL) {
rv->code[0] = GET_ARG;
rv->code[1] = argnum;
}
else
error = 1;
if ((rv->stack = PyMem_NEW(stack_entry, rv->stacksize)) == NULL)
error = 1;
if (error) {
numexpr_dealloc(rv);
rv = NULL;
}
}
return rv;
}
static PyObject *
numexpr_intvar(self, args)
PyObject *self; /* Not used */
PyObject *args;
{
PyNumExprObject *rv = new_variable(args);
if (rv != NULL) {
rv->argtypes[rv->nargs-1] = 'l';
rv->type = INTEGER;
}
return (PyObject *)rv;
}
static PyObject *
numexpr_floatvar(self, args)
PyObject *self; /* Not used */
PyObject *args;
{
PyNumExprObject *rv = new_variable(args);
if (rv != NULL) {
rv->argtypes[rv->nargs-1] = 'd';
rv->type = FLOAT;
}
return (PyObject *)rv;
}
static PyObject *
numexpr_complexvar(self, args)
PyObject *self; /* Not used */
PyObject *args;
{
PyNumExprObject *rv = new_variable(args);
if (rv != NULL) {
rv->argtypes[rv->nargs-1] = 'D';
rv->type = COMPLEX;
}
return (PyObject *)rv;
}
/* List of functions defined in the module */
static struct PyMethodDef numexpr_functions[] = {
{"FloatVariable", numexpr_floatvar, 1},
{"IntegerVariable", numexpr_intvar, 1},
{"ComplexVariable", numexpr_complexvar, 1},
{NULL, NULL} /* sentinel */
};
/* Various math functions */
#if 0
static PyObject *
numexpr_sqrt(self)
PyNumExprObject *self;
{
PyNumExprObject *temp = self;
PyNumExprObject *r;
if (self->type == INTEGER) {
temp = numexpr_unary_op(self, INTEGER_FLOAT);
temp->type = FLOAT;
}
if (temp->type == COMPLEX)
r = numexpr_unary_op(temp, SQRT_COMPLEX);
else
r = numexpr_unary_op(temp, SQRT_FLOAT);
if (temp != self)
numexpr_dealloc(temp);
return (PyObject *)r;
}
#endif
static PyObject *
numexpr_mathfunc(arg, float_op, complex_op)
PyNumExprObject *arg;
int float_op;
int complex_op;
{
PyNumExprObject *temp = arg;
PyNumExprObject *r;
if (arg->type == INTEGER) {
temp = numexpr_unary_op(arg, INTEGER_FLOAT);
temp->type = FLOAT;
}
if (temp->type == COMPLEX)
r = numexpr_unary_op(temp, complex_op);
else
r = numexpr_unary_op(temp, float_op);
if (temp != arg)
numexpr_dealloc(temp);
return (PyObject *)r;
}
#define add_mathfunc(func_name, float_op, complex_op) \
static PyObject * \
func_name(self) \
PyNumExprObject *self; \
{ \
return numexpr_mathfunc(self, float_op, complex_op); \
}
add_mathfunc(numexpr_acos, ACOS_FLOAT, ACOS_COMPLEX)
add_mathfunc(numexpr_acosh, ACOSH_FLOAT, ACOSH_COMPLEX)
add_mathfunc(numexpr_asin, ASIN_FLOAT, ASIN_COMPLEX)
add_mathfunc(numexpr_asinh, ASINH_FLOAT, ASINH_COMPLEX)
add_mathfunc(numexpr_atan, ATAN_FLOAT, ATAN_COMPLEX)
add_mathfunc(numexpr_atanh, ATANH_FLOAT, ATANH_COMPLEX)
add_mathfunc(numexpr_cos, COS_FLOAT, COS_COMPLEX)
add_mathfunc(numexpr_cosh, COSH_FLOAT, COSH_COMPLEX)
add_mathfunc(numexpr_exp, EXP_FLOAT, EXP_COMPLEX)
add_mathfunc(numexpr_log, LOG_FLOAT, LOG_COMPLEX)
add_mathfunc(numexpr_log10, LOG10_FLOAT, LOG10_COMPLEX)
add_mathfunc(numexpr_sin, SIN_FLOAT, SIN_COMPLEX)
add_mathfunc(numexpr_sinh, SINH_FLOAT, SINH_COMPLEX)
add_mathfunc(numexpr_sqrt, SQRT_FLOAT, SQRT_COMPLEX)
add_mathfunc(numexpr_tan, TAN_FLOAT, TAN_COMPLEX)
add_mathfunc(numexpr_tanh, TANH_FLOAT, TANH_COMPLEX)
/* List of methods of type "numexpr" */
static struct PyMethodDef numexpr_methods[] = {
{"acos", (PyCFunction)numexpr_acos, 1},
{"acosh", (PyCFunction)numexpr_acosh, 1},
{"asin", (PyCFunction)numexpr_asin, 1},
{"asinh", (PyCFunction)numexpr_asinh, 1},
{"atan", (PyCFunction)numexpr_atan, 1},
{"atanh", (PyCFunction)numexpr_atanh, 1},
{"cos", (PyCFunction)numexpr_cos, 1},
{"cosh", (PyCFunction)numexpr_cosh, 1},
{"exp", (PyCFunction)numexpr_exp, 1},
{"log", (PyCFunction)numexpr_log, 1},
{"log10", (PyCFunction)numexpr_log10, 1},
{"sin", (PyCFunction)numexpr_sin, 1},
{"sinh", (PyCFunction)numexpr_sinh, 1},
{"sqrt", (PyCFunction)numexpr_sqrt, 1},
{"tan", (PyCFunction)numexpr_tan, 1},
{"tanh", (PyCFunction)numexpr_tanh, 1},
{NULL, NULL} /* sentinel */
};
/* Initialization function for the module (*must* be called initnumexpr) */
void
initnumexpr()
{
PyObject *m, *d;
/* Create the module and add the functions */
m = Py_InitModule("numexpr", numexpr_functions);
/* Add some symbolic constants to the module */
d = PyModule_GetDict(m);
ErrorObject = PyString_FromString("NumExprError");
PyDict_SetItemString(d, "error", ErrorObject);
/* Check for errors */
if (PyErr_Occurred())
Py_FatalError("can't initialize module numexpr");
}
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Tue Jan 23 18:00:16 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Tue, 23 Jan 96 13:00:16 EST
Subject: [PYTHON MATRIX-SIG] floating point equalities
Message-ID: <9601231800.AA29105@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
In preparation for the 0.30 release (still on for Friday) I've been
cleaning up the array math functions.
I'm now working on improving the floating point equality operator. My
understanding is that doing a == b is unreasonable for floating point
numbers, and instead the test should be something like
abs(a-b) < TOLERANCE*abs(a)
Rather than have me try and rederive what this should look like from
my vague memories of the IEEE floating point standard, I was wondering
if anybody out there has a good chunk of code for doing this
comparision?
Thanks, Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From guido@CNRI.Reston.VA.US Tue Jan 23 18:26:24 1996
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Tue, 23 Jan 1996 13:26:24 -0500
Subject: [PYTHON MATRIX-SIG] floating point equalities
In-Reply-To: Your message of "Tue, 23 Jan 1996 13:00:16 EST."
<9601231800.AA29105@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
References: <9601231800.AA29105@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <199601231826.NAA15424@monty>
> I'm now working on improving the floating point equality operator. My
> understanding is that doing a == b is unreasonable for floating point
> numbers, and instead the test should be something like
>
> abs(a-b) < TOLERANCE*abs(a)
Hm. If you put this in the array module it would be a reasonable
expectation that it would also work for ordinary Python floats, which
it doesn't. Also, the tolerance may be application dependent or
dependent on the part of the application that's doing this, so it may
need to be specified in the operation.
I'd advise to continue using C's '==' operator for floats (which is
what ordinary Python floats do). Users who know enough to write
numerical code should also know not to compare for equality.
(What you could do is provide a method or function for comparisons
where you can pass the tolerance in as an argument.)
--Guido van Rossum <guido@CNRI.Reston.VA.US>
URL: <http://www.python.org/~guido/>
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Tue Jan 23 18:48:21 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Tue, 23 Jan 96 13:48:21 EST
Subject: [PYTHON MATRIX-SIG] floating point equalities
In-Reply-To: <199601231826.NAA15424@monty> (message from Guido van Rossum on Tue, 23 Jan 1996 13:26:24 -0500)
Message-ID: <9601231848.AA29834@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: Guido van Rossum <guido@CNRI.Reston.VA.US>
> I'm now working on improving the floating point equality operator. My
> understanding is that doing a == b is unreasonable for floating point
> numbers, and instead the test should be something like
>
> abs(a-b) < TOLERANCE*abs(a)
Hm. If you put this in the array module it would be a reasonable
expectation that it would also work for ordinary Python floats, which
it doesn't. Also, the tolerance may be application dependent or
dependent on the part of the application that's doing this, so it may
need to be specified in the operation.
I'd advise to continue using C's '==' operator for floats (which is
what ordinary Python floats do). Users who know enough to write
numerical code should also know not to compare for equality.
(What you could do is provide a method or function for comparisons
where you can pass the tolerance in as an argument.)
I'm still curious to see what the more numerically inclined folks have
to say on this, but you're obviously right that this should be
provided as an additional function, not the default comparision
behavior.
I'd like to have a method approxEqual (or some similar name) so that
a.approxEqual(b) will "do the right thing". I had believed that there
was some sort of consensus in the numerical programming community on
what the right thing is for floating point numbers, if I'm wrong on
this, I'd like to know that too.
Note: part of why this came up is the fact that on my development
system (1j)**2 == (-1+1.22460635382e-16j). I'd really like to be able
to have some way to say approxEqual((1j)**2, -1) and have it come out
true (OK, I have an ad hoc solution that works, what I want is the
"right" solution if there is one out there).
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Jan 23 19:05:49 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Tue, 23 Jan 1996 14:05:49 -0500
Subject: [PYTHON MATRIX-SIG] floating point equalities
In-Reply-To: <9601231800.AA29105@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199601231905.OAA01142@cyclone.ERE.UMontreal.CA>
I'm now working on improving the floating point equality operator. My
understanding is that doing a == b is unreasonable for floating point
numbers, and instead the test should be something like
abs(a-b) < TOLERANCE*abs(a)
I don't agree. First, there are cases where testing for equality
makes sense (e.g. if you know that the number represents an integer,
or if you want to detect an underflow by comparing to zero).
Second, standard Python floats use normal equality, so arrays
should behave identically.
On the other hand, having a test with tolerance would be very
useful and make code more readable. APL has such a feature
(using a global variable for the tolerance, which for Python
is not such a great idea), which I like a lot. And since
the comparison "operators" for arrays are methods anyway,
it is easy to have both, with an optional second argument
for the tolerance. I.e. you would write
a.equal(b)
for a normal equality test and
a.equal(b, 1e-6)
for a test with a given tolerance. As for the form of a test with
tolerance, I am not sure whether there is a generally accepted best
one. I have seen (and used) the form you have given above, but
also the more symmetric form
abs(a-b) < TOLERANCE*(abs(a)+abs(b))
(although I wonder whether this is ever necessary). Hopefully
the mathematicians among us can say something more definite.
And now for something completely different...
While rereading the discussion about the beta release, I noticed that
there was no mention at all of how to deal with functions of finite
rank. Although there are none in the current package, they will
appear in the application-specific packages to be written later
(e.g. linear algebra), so there should be a general mechanism for
dealing with them.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From strow@umbc.edu Tue Jan 23 19:40:48 1996
From: strow@umbc.edu (L. Larrabee Strow)
Date: Tue, 23 Jan 1996 14:40:48 -0500
Subject: [PYTHON MATRIX-SIG] floating point equalities
Message-ID: <v02130506ad2ae9c06db8@[130.85.140.14]>
>In preparation for the 0.30 release (still on for Friday) I've been
>cleaning up the array math functions.
>
>I'm now working on improving the floating point equality operator. My
>understanding is that doing a == b is unreasonable for floating point
>numbers, and instead the test should be something like
>
>abs(a-b) < TOLERANCE*abs(a)
>
>Rather than have me try and rederive what this should look like from
>my vague memories of the IEEE floating point standard, I was wondering
>if anybody out there has a good chunk of code for doing this
>comparision?
>
>Thanks, Jim
>
Let the user decide how to compare for equality. We are used to avoiding
the a==b problem. It would probably be a mistake to "help" us too much,
since we might get lazy and make even more subtle mistakes.
--
L. Larrabee Strow E-mail: strow@umbc.edu
Department of Physics FAX : (410) 455-1072
University of Maryland Baltimore County Phone : (410) 455-2528
5401 Wilkens Avenue
Baltimore, MD 21228-5398
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From guido@CNRI.Reston.VA.US Tue Jan 23 19:37:35 1996
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Tue, 23 Jan 1996 14:37:35 -0500
Subject: [PYTHON MATRIX-SIG] floating point equalities
In-Reply-To: Your message of "Tue, 23 Jan 1996 13:48:21 EST."
<9601231848.AA29834@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
References: <9601231848.AA29834@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <199601231937.OAA15633@monty>
> I'd like to have a method approxEqual (or some similar name) so that
> a.approxEqual(b) will "do the right thing". I had believed that there
> was some sort of consensus in the numerical programming community on
> what the right thing is for floating point numbers, if I'm wrong on
> this, I'd like to know that too.
Cool.
> Note: part of why this came up is the fact that on my development
> system (1j)**2 == (-1+1.22460635382e-16j). I'd really like to be able
> to have some way to say approxEqual((1j)**2, -1) and have it come out
> true (OK, I have an ad hoc solution that works, what I want is the
> "right" solution if there is one out there).
I bet this is because in x**y, Python currently coerces y, meaning you
were really calculating 1.0j ** 2.0j...
(Note that the (by now pathetically incomplete) Python test suite has
a similar problem and solution when testing the pow() function.)
--Guido van Rossum <guido@CNRI.Reston.VA.US>
URL: <http://www.python.org/~guido/>
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Jan 23 20:29:31 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Tue, 23 Jan 1996 15:29:31 -0500
Subject: [PYTHON MATRIX-SIG] floating point equalities
In-Reply-To: <9601231848.AA29834@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199601232029.PAA07104@cyclone.ERE.UMontreal.CA>
Note: part of why this came up is the fact that on my development
system (1j)**2 == (-1+1.22460635382e-16j). I'd really like to be able
to have some way to say approxEqual((1j)**2, -1) and have it come out
Are you sure that you have installed the latest version of my complex
object? It should give 1j**2 == -1., exactly. The reason is that
complex_power() detects that the exponent is actually an integer (no
fractional or imaginary part), and then calls c_powi(), which in turn
calls c_powu() because the exponent is positive and less than
100. c_powu() reduces the power calculation to an optimized sequence
of multiplications. Therefore 1j**2 is exactly equivalent to
1j*1j. (And yes, I have traced this example with a debugger right now
because I was a bit worried).
What does 1j*1j give on your system? It ought to be exactly -1., since
there are no round-off errors in this calculation.
I realize that this has nothing to do with doing comparisons in
general, of course...
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Tue Jan 23 20:39:21 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Tue, 23 Jan 96 15:39:21 EST
Subject: [PYTHON MATRIX-SIG] floating point equalities
In-Reply-To: <199601232029.PAA07104@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca)
Message-ID: <9601232039.AA01063@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Note: part of why this came up is the fact that on my development
system (1j)**2 == (-1+1.22460635382e-16j). I'd really like to be able
to have some way to say approxEqual((1j)**2, -1) and have it come out
Are you sure that you have installed the latest version of my complex
object? It should give 1j**2 == -1., exactly. The reason is that
complex_power() detects that the exponent is actually an integer (no
fractional or imaginary part), and then calls c_powi(), which in turn
calls c_powu() because the exponent is positive and less than
100. c_powu() reduces the power calculation to an optimized sequence
of multiplications. Therefore 1j**2 is exactly equivalent to
1j*1j. (And yes, I have traced this example with a debugger right now
because I was a bit worried).
What does 1j*1j give on your system? It ought to be exactly -1., since
there are no round-off errors in this calculation.
I realize that this has nothing to do with doing comparisons in
general, of course...
My fault, I'm still running with an old version of your patch.
I'll install the new one before I do much more testing. (Still, this
problem comes up in many other places).
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Jan 23 20:39:57 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Tue, 23 Jan 1996 15:39:57 -0500
Subject: [PYTHON MATRIX-SIG] floating point equalities
In-Reply-To: <199601231937.OAA15633@monty> (message from Guido van Rossum on Tue, 23 Jan 1996 14:37:35 -0500)
Message-ID: <199601232039.PAA07827@cyclone.ERE.UMontreal.CA>
I bet this is because in x**y, Python currently coerces y, meaning you
were really calculating 1.0j ** 2.0j...
(Note that the (by now pathetically incomplete) Python test suite has
a similar problem and solution when testing the pow() function.)
For complex numbers, 1j**2.0 is calculated as 1j**2. And with my
set of patches, the same happens for real numbers. You still don't
get the full efficiency of an integer exponentiation, but at least
the accuracy.
That is of course no argument for not fixing the pow() coercion
problem...
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dubois1@llnl.gov Tue Jan 23 21:59:47 1996
From: dubois1@llnl.gov (Paul. Dubois)
Date: Tue, 23 Jan 1996 13:59:47 -0800
Subject: [PYTHON MATRIX-SIG] floating point equalities
References: <9601231848.AA29834@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <31055A53.739E@llnl.gov>
Guido is clearly right.
As for how to do approxEqual(a,b), there is no one right answer.
There are two ways of measuring error, relative and absolute.
Relative is usually defined as abs(a-b)/max(abs(a),abs(b)).
Absolute is just abs(a-b).
Suppose the two numbers are 10**-15 and 10**-16. Depending on what you
are doing, the difference between these two is either (a) a factor of
10, which sounds "large", or (b) about 10**-15, which sounds small.
Many people use a "mixed" critera, considering two numbers equal if
their relative difference is small or their absolute difference is
(perhaps a different size of) small. You see tests like:
abs(a-b) < rel_tol * abs(a) + abs_tol
But small is still hard to be precise about, since it really depends on
context. If you get a relative error of 1.e-8 solving a linear system it
might be expected, but that same number might be a sign of trouble in
other contexts where errors more like 1.e-14 are normal.
One solution is say a class RealNumberComparer, which contains
attributes maximum_relative_error, maximum_absolute_error, routines to
set these (with some reasonable defaults set on creation) and an
equals(a,b) to carry out the test.
--
Paul F. Dubois, L-472 (510)-422-5426
Lawrence Livermore National Laboratory FAX (510)-423-9969
Livermore, CA 94550 dubois1@llnl.gov
Consulting: PaulDubois@aol.com
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Wed Jan 24 16:18:19 1996
From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey)
Date: Wed, 24 Jan 1996 11:18:19 -0500
Subject: [PYTHON MATRIX-SIG] Summary for main list?
Message-ID: <9601241118.ZM24858@dsdbqvarsa.er.usgs.gov>
A summary of our activities to the Python list is probably long overdue. Jim
or others, do you already have anything that would be good to post. Do we have
a complete summary of the work? Somethink like the original proposal that has
been updated to reflect decisions made?
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Thu Jan 25 12:34:25 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Thu, 25 Jan 96 07:34:25 EST
Subject: [PYTHON MATRIX-SIG] Summary for main list?
In-Reply-To: <9601241118.ZM24858@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov)
Message-ID: <9601251234.AA06874@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
There is no complete summary of the work anywhere. At this point, the
best summary is going to probably be the documentation for the
NumericPython extension package (completed in time for the first beta
release).
I'm still on the schedule I recently included in my proposals for a
final set of naming conventions, etc. This includes a general beta
release (advertised in the main newsgroup) on 2/12.
Personally, I'd just as soon wait until then to bring the main python
group "up to date".
If somebody wants to post a generic message reminding people of the
existence of the matrix-sig and its purpose, that might be useful.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Thu Jan 25 13:13:02 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Thu, 25 Jan 96 08:13:02 EST
Subject: [PYTHON MATRIX-SIG] type coercion one more time
Message-ID: <9601251313.AA07030@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
I've been working to design a stable version of automatic type
coercion that everybody (including myself) can be happy with. The
following is what I have. Much of this is in line with the C rules
for type coercion (at least according to my old K&R). I will use the
standard C type names (double means a python float).
Only the following three automatic coercions are allowed:
1) Any integer type can be coerced to a float, a double, a complex float or
a complex double.
2) A float can only be coerced to a complex float.
3) A double can only be coerced to a complex double.
Explanation:
The following type hierarchy is assumed:
INTEGER < FLOAT < COMPLEX
Within each level there are a number of possible precisions. No
automatic coercion will take place between these precisions.
This remains completely consistent with the existing python automatic
type coercion.
Examples:
a_t = array([x,y,z,...], 't') # Where t is a legal typecode
1 - signed byte
l - long
f - float
d - double
F - complex float
D - complex double
a_l * a_d = a_d
a_f * a_l = a_f
a_f * a_d = TypeError Exception raised
a_d * a_D = a_D
a_1 * a_l = TypeError Exception raised
Comments? - Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Thu Jan 25 14:52:01 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Thu, 25 Jan 1996 09:52:01 -0500
Subject: [PYTHON MATRIX-SIG] type coercion one more time
In-Reply-To: <9601251313.AA07030@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199601251452.JAA02955@cyclone.ERE.UMontreal.CA>
Only the following three automatic coercions are allowed:
1) Any integer type can be coerced to a float, a double, a complex float or
a complex double.
2) A float can only be coerced to a complex float.
3) A double can only be coerced to a complex double.
Sounds OK for me.
This remains completely consistent with the existing python automatic
type coercion.
Which is all I really care about...
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From guido@CNRI.Reston.VA.US Thu Jan 25 14:57:56 1996
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Thu, 25 Jan 1996 09:57:56 -0500
Subject: [PYTHON MATRIX-SIG] type coercion one more time
In-Reply-To: Your message of "Thu, 25 Jan 1996 08:13:02 EST."
<9601251313.AA07030@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
References: <9601251313.AA07030@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <199601251457.JAA21034@monty>
> No automatic coercion will take place between these precisions.
This surprises me (especially since C does support float -> double
coercions). Would you care to explain it?
--Guido van Rossum <guido@CNRI.Reston.VA.US>
URL: <http://www.python.org/~guido/>
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Thu Jan 25 15:45:58 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Thu, 25 Jan 96 10:45:58 EST
Subject: [PYTHON MATRIX-SIG] type coercion one more time
In-Reply-To: <199601251457.JAA21034@monty> (message from Guido van Rossum on Thu, 25 Jan 1996 09:57:56 -0500)
Message-ID: <9601251545.AA08514@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: Guido van Rossum <guido@CNRI.Reston.VA.US>
> No automatic coercion will take place between these precisions.
This surprises me (especially since C does support float -> double
coercions). Would you care to explain it?
Naive users of the system are only expected to use arrays of longs,
doubles, and complex doubles. I wanted to make sure that all of the
automatic coercion rules would work for these most frequently used
types so that they'll work just like the corresponding python scalars.
On the other hand, I see the other types as being used for cases where
the programmer really wants to use a particular precision, ie. 2-byte
integers for dealing with 16-bit sound data. In a case like this it
seems decidely unfriendly to automatically coerce to another
precision.
One can imagine dividing each sample in a sound by 2, and then trying
to play it back and hearing garbage because it got automatically
converted to an array of longs during the division.
I do have one other possible set of rules that I would be happy with.
These are basically the same as the last post, except:
4) Coercion to a higher precision is allowed within the main
groupings.
5) The precision of a python scalar is allowed to change to match the
other arguments (3.14 can be either a float or a double depending on
which one is "desired").
I don't like this rule, but without it I think that it is too easy to
get bitten by python scalars when writing "real" code.
Does this make sense?
Do people prefer this revised scheme?
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From stoll@atr-sw.atr.co.jp Thu Jan 25 16:09:53 1996
From: stoll@atr-sw.atr.co.jp (Perry A. Stoll)
Date: Fri, 26 Jan 1996 01:09:53 +0900
Subject: [PYTHON MATRIX-SIG] type coercion one more time
In-Reply-To: Your message of "Thu, 25 Jan 1996 08:13:02 JST"
References: <9601251313.AA07030@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <199601251609.BAA02646@ciris21.atr-sw.atr.co.jp>
>>>>> "JH" == James Hugunin <jjh@Goldilocks.LCS.MIT.EDU> writes:
JH> I've been working to design a stable version of automatic type
JH> coercion that everybody (including myself) can be happy with.
Sounds good. I offer my support for continuing something similar to
the current scheme.
JH> Only the following three automatic coercions are allowed:
JH> 1) Any integer type can be coerced to a float, a double, a
JH> complex float or a complex double.
Does 'any integer' type mean byte,short,integer? If so, OK.
JH> 2) A float can only be coerced to a complex float.
JH> 3) A double can only be coerced to a complex double.
JH> Explanation:
JH> The following type hierarchy is assumed:
JH> INTEGER < FLOAT < COMPLEX
JH> Within each level there are a number of possible precisions.
True, and it would be unrealistic to do arbitrary coercions.
JH> No automatic coercion will take place between these
JH> precisions.
But there is no reason not to do coercion to a greater precision. See
below.
JH> This remains completely consistent with the existing python
JH> automatic type coercion.
To some extent. But there is no access in Python to most underlying C
numeric types, for good or bad. Good in most cases, bad in C/Fortran
interfacing cases. It seems like we're trying to add these types to
Python - this could be difficult.
JH> a_l * a_d = a_d
JH> a_f * a_l = a_f
JH> a_f * a_d = TypeError Exception raised
JH> a_d * a_D = a_D
JH> a_1 * a_l = TypeError Exception raised
Raising exceptions in these circumstances seems a bit arbitrary.
I think the source of your problem is the difference between Python's
floats (i.e. C doubles) and C floats. I don't know any way to specify
a C float in Python - does anyone? It seems the idealistic approach of
having only one floating point representation in Python is at odds
with peoples' desire to access C floats.
Also, I don't see how the scheme you outline above will solve your problem
Array_f([1.0,2.0,3.0]) * 2.0 = Array_f([2.0,4.0,6.0])
since '2.0' is interpreted as a Python 'float', i.e. a C double. From
the scheme you propose above, this would raise an exception. Assuming
it is too late to add '2.0f' to Python (or a better type specification
scheme as outlined earlier on this list), I can't think of a way to
get around this without explicitly doing an ungainly Matrix_f(2.0). Of
course, if it matters this much, you could always do M_f = Matrix_f
and have M_f(2.0) scattered through your code.
The more I think about this (and since I'm thinking as I'm writing,
this is at the expense of conciseness - sorry!), the more I think your
situation is outside the Python norm and should really be viewed as
such.
Although it may not solve your problem of C floats, I think it makes
more sense to allow conversion to any "higher" type, where "higher"
means higher in the list of types, i.e.
b < s < i < l < f < d
Also: (b,s,i,l,f) <= C
(b,s,i,l,f,d,C) <= D
With the exception of the last bit, isn't this how the present (0.2)
release works?
Also, I don't recall anyone presenting a list of 'nice' names for
the Matrix functions as Konrad has been advocating. I think having
these is a good idea; the type code that we have now has always seemed
somehow counter to (my view of) the Python 'do-it-the-Right-Way'
attitude. If I actually list them all out here, maybe this will force
the issue and start some discussion. Comments on the following?
Array_Char, Array_UChar
Array_Short, Array_UShort
Array_Integer,Array_UInteger,
Array_Float, Array_Double,
Array_FloatComplex, Array_DoubleComplex
'Unsigned' instead of 'U'? XArray instead of Array_X? BTW, is the C
multiarray object going to support all builtin numeric C types? Right
now, there is supported for both signed and unsigned bytes, but not
for unsigned shorts, ints, or longs - is that good enough?
-Perry Stoll
Research Associate, CSRL Advanced Telecommunications Research
2-2 Hikaridai, Seika-cho Soraku-gun, Kyoto 619-02 JAPAN
FINGER: stoll@yoshino.atr.co.jp EMAIL: stoll@atr-sw.atr.co.jp
PGP 2.6 Key fingerprint = AF 56 5C D8 5F 78 BA FD 21 6E 2A 68 C4 55 9E B0
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From stoll@atr-sw.atr.co.jp Thu Jan 25 16:33:59 1996
From: stoll@atr-sw.atr.co.jp (Perry A. Stoll)
Date: Fri, 26 Jan 1996 01:33:59 +0900
Subject: [PYTHON MATRIX-SIG] type coercion one more time
In-Reply-To: Your message of "Thu, 25 Jan 1996 10:45:58 JST"
References: <9601251545.AA08514@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <199601251633.BAA02667@ciris21.atr-sw.atr.co.jp>
>>>>> "JH" == James Hugunin <jjh@Goldilocks.LCS.MIT.EDU> writes:
JH> From: Guido van Rossum <guido@CNRI.Reston.VA.US>
>> No automatic coercion will take place between these precisions.
JH> This surprises me (especially since C does support float ->
JH> double coercions). Would you care to explain it?
JH> Naive users of the system are only expected to use arrays of
JH> longs, doubles, and complex doubles. I wanted to make sure
JH> that all of the automatic coercion rules would work for these
JH> most frequently used types so that they'll work just like the
JH> corresponding python scalars.
JH> On the other hand, I see the other types as being used for
JH> cases where the programmer really wants to use a particular
JH> precision, ie. 2-byte integers for dealing with 16-bit sound
JH> data. In a case like this it seems decidely unfriendly to
JH> automatically coerce to another precision.
JH> One can imagine dividing each sample in a sound by 2, and then
JH> trying to play it back and hearing garbage because it got
JH> automatically converted to an array of longs during the
JH> division.
JH> I do have one other possible set of rules that I would be
JH> happy with. These are basically the same as the last post,
JH> except:
JH> 4) Coercion to a higher precision is allowed within the main
JH> groupings.
JH> 5) The precision of a python scalar is allowed to change to
JH> match the other arguments (3.14 can be either a float or a
JH> double depending on which one is "desired").
JH> Do people prefer this revised scheme?
So you try to coerce a scalar to the type of the matrix, using
the same idea as in 4)? Is the following right?
Array_SomeIntType * AnyIntType = Array_SomeIntType
Array_SomeIntType * SomeFloatType =? Array_SomeFloatType
Array_SomeFloatType * AnyIntType =? Array_SomeFloatType
Array_SomeFloatType * OtherFloatType = Array_SomeFloatType
That seems pretty reasonable. Do the '=?' cases sound ok? It should
preserve coercions with the normal Python types. Users of specialized
C type multiarrays must exercise a bit of caution that they don't
cause overflow, but that isn't unreasonable.
-Perry
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From guido@CNRI.Reston.VA.US Thu Jan 25 16:27:05 1996
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Thu, 25 Jan 1996 11:27:05 -0500
Subject: [PYTHON MATRIX-SIG] type coercion one more time
In-Reply-To: Your message of "Thu, 25 Jan 1996 10:45:58 EST."
<9601251545.AA08514@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
References: <9601251545.AA08514@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <199601251627.LAA21529@monty>
> > No automatic coercion will take place between these precisions.
>
> This surprises me (especially since C does support float -> double
> coercions). Would you care to explain it?
>
> Naive users of the system are only expected to use arrays of longs,
> doubles, and complex doubles. I wanted to make sure that all of the
> automatic coercion rules would work for these most frequently used
> types so that they'll work just like the corresponding python scalars.
>
> On the other hand, I see the other types as being used for cases where
> the programmer really wants to use a particular precision, ie. 2-byte
> integers for dealing with 16-bit sound data. In a case like this it
> seems decidely unfriendly to automatically coerce to another
> precision.
>
> One can imagine dividing each sample in a sound by 2, and then trying
> to play it back and hearing garbage because it got automatically
> converted to an array of longs during the division.
I see your point, but I am uneasy with the solution. Shouldn't there
be a "divide in place" operation for this case that coerces constants
the other way? (I haven't followed all the details -- you may have it
already or you may have a good reason for not having it.)
I guess that you also don't check for overflow on such operations --
suppose I multiply each sample by 2 and it doesn't fit... This is
also different than Python's own numeric operations, but again I can
see the reason why (and I don't object in this case).
I guess I won't complain too loudly as long as you also have a way to
coerce an array of shorts into an array of longs.
(BTW, on 64-bit machines, there are three relevant int sizes. Do you
support this?)
> I do have one other possible set of rules that I would be happy with.
> These are basically the same as the last post, except:
>
> 4) Coercion to a higher precision is allowed within the main
> groupings.
>
> 5) The precision of a python scalar is allowed to change to match the
> other arguments (3.14 can be either a float or a double depending on
> which one is "desired").
Some examples, please? Where else is coercion applied? It does sound
like it could be a good compromise.
> I don't like this rule, but without it I think that it is too easy to
> get bitten by python scalars when writing "real" code.
Agreed.
> Does this make sense?
>
> Do people prefer this revised scheme?
I'm not sure. More input, please...
--Guido van Rossum <guido@CNRI.Reston.VA.US>
URL: <http://www.python.org/~guido/>
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Thu Jan 25 16:40:49 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Thu, 25 Jan 96 11:40:49 EST
Subject: [PYTHON MATRIX-SIG] type coercion one more time
In-Reply-To: <199601251633.BAA02667@ciris21.atr-sw.atr.co.jp> (stoll@atr-sw.atr.co.jp)
Message-ID: <9601251640.AA09236@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: "Perry A. Stoll" <stoll@atr-sw.atr.co.jp>
JH> I do have one other possible set of rules that I would be
JH> happy with. These are basically the same as the last post,
JH> except:
JH> 4) Coercion to a higher precision is allowed within the main
JH> groupings.
JH> 5) The precision of a python scalar is allowed to change to
JH> match the other arguments (3.14 can be either a float or a
JH> double depending on which one is "desired").
JH> Do people prefer this revised scheme?
So you try to coerce a scalar to the type of the matrix, using
the same idea as in 4)? Is the following right?
Array_SomeIntType * AnyIntType = Array_SomeIntType
Array_SomeIntType * SomeFloatType =? Array_SomeFloatType
Array_SomeFloatType * AnyIntType =? Array_SomeFloatType
Array_SomeFloatType * OtherFloatType = Array_SomeFloatType
That seems pretty reasonable. Do the '=?' cases sound ok? It should
preserve coercions with the normal Python types. Users of specialized
C type multiarrays must exercise a bit of caution that they don't
cause overflow, but that isn't unreasonable.
-Perry
This is exactly what I'm proposing (change the '=?' to '='). The fact
that it seems to make sense to you is a good sign that it might be
reasonable.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From guido@CNRI.Reston.VA.US Thu Jan 25 16:46:14 1996
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Thu, 25 Jan 1996 11:46:14 -0500
Subject: [PYTHON MATRIX-SIG] type coercion one more time
In-Reply-To: Your message of "Thu, 25 Jan 1996 11:40:49 EST."
<9601251640.AA09236@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
References: <9601251640.AA09236@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <199601251646.LAA21663@monty>
OK, I'm convinced too.
--Guido van Rossum <guido@CNRI.Reston.VA.US>
URL: <http://www.python.org/~guido/>
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Thu Jan 25 16:54:06 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Thu, 25 Jan 96 11:54:06 EST
Subject: [PYTHON MATRIX-SIG] type coercion one more time
In-Reply-To: <199601251627.LAA21529@monty> (message from Guido van Rossum on Thu, 25 Jan 1996 11:27:05 -0500)
Message-ID: <9601251654.AA09280@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: Guido van Rossum <guido@CNRI.Reston.VA.US>
> One can imagine dividing each sample in a sound by 2, and then trying
> to play it back and hearing garbage because it got automatically
> converted to an array of longs during the division.
I see your point, but I am uneasy with the solution. Shouldn't there
be a "divide in place" operation for this case that coerces constants
the other way? (I haven't followed all the details -- you may have it
already or you may have a good reason for not having it.)
There should be a divide in place method, I just never bothered to add
it. I'll try and remedy this before the beta release.
I guess that you also don't check for overflow on such operations --
suppose I multiply each sample by 2 and it doesn't fit... This is
also different than Python's own numeric operations, but again I can
see the reason why (and I don't object in this case).
There are in fact two different modules which contain array math
operations. umath and fast_umath. These modules will actually define
the code used for things like a*4. fast_umath will never check for
overflow for the obvious speed reasons. umath is supposed to work as
much like regular python math as possible. It checks for
divide-by-zero and for floating point overflows. At the moment it
doesn't check for integer overflows, but it SHOULD, and it will as
soon as I get around to it. I think having both of these modules is a
nice way to get "the best of both worlds".
I guess I won't complain too loudly as long as you also have a way to
coerce an array of shorts into an array of longs.
Anything can be cast to anything else, including shoving an array of
doubles into an array of bytes. Of course, some casts are a better
idea than others...
(BTW, on 64-bit machines, there are three relevant int sizes. Do you
support this?)
four actually, byte, short (2-byte), int (4-byte), and long (8-byte).
They're all there.
> I do have one other possible set of rules that I would be happy with.
> These are basically the same as the last post, except:
>
> 4) Coercion to a higher precision is allowed within the main
> groupings.
>
> 5) The precision of a python scalar is allowed to change to match the
> other arguments (3.14 can be either a float or a double depending on
> which one is "desired").
Some examples, please? Where else is coercion applied? It does sound
like it could be a good compromise.
See Perry's recent post for a good explanation of this proposal.
Examples:
array([1,2,3], 'f') * 2. -> array([1,2,3], 'f') # 2. is treated as a float here
array([1,2,3], 'l') * 2. -> array([1,2,3], 'l') # 2. is treated as a double here
> I don't like this rule, but without it I think that it is too easy to
> get bitten by python scalars when writing "real" code.
Agreed.
> Does this make sense?
>
> Do people prefer this revised scheme?
I'm not sure. More input, please...
Well, I just saw your latest reply, so I don't think more input's
needed. I'll send this anyway because the earlier comments should be
useful.
I think I'll give this thing another hour to settle down and then I'll
code up the new typecasting rules (I really love how quickly
discussion moves on this list).
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Thu Jan 25 17:09:17 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Thu, 25 Jan 1996 12:09:17 -0500
Subject: [PYTHON MATRIX-SIG] type coercion one more time
In-Reply-To: <9601251545.AA08514@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199601251709.MAA13702@cyclone.ERE.UMontreal.CA>
On the other hand, I see the other types as being used for cases where
the programmer really wants to use a particular precision, ie. 2-byte
integers for dealing with 16-bit sound data. In a case like this it
seems decidely unfriendly to automatically coerce to another
precision.
I wouldn't call it "decidedly unfriendly" (you can always avoid
coercion by making sure that all variables have the same type), but it
makes sense. It should also be noted that one of the main reasons to
have float->double coercion in C is to make it possible to call
functions written for double with float arguments. This problem does
not exist in Python.
4) Coercion to a higher precision is allowed within the main
groupings.
What are "main groupings"?
5) The precision of a python scalar is allowed to change to match the
other arguments (3.14 can be either a float or a double depending on
which one is "desired").
I don't like this rule for two reasons:
1) You can lose precision without noticing it.
2) It creates a subtle distinction between scalars and arrays of rank 0,
which can only confuse users.
On the other hand, I do see the problem you want to solve. It doesn't
make any code easier to read if you have to convert every constant to
a rank-0 float array. If I get an ingenious idea for solving this
problem, I'll let you know immediately...
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Thu Jan 25 17:15:06 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Thu, 25 Jan 1996 12:15:06 -0500
Subject: [PYTHON MATRIX-SIG] type coercion one more time
In-Reply-To: <199601251609.BAA02646@ciris21.atr-sw.atr.co.jp> (stoll@atr-sw.atr.co.jp)
Message-ID: <199601251715.MAA14003@cyclone.ERE.UMontreal.CA>
Also, I don't recall anyone presenting a list of 'nice' names for
the Matrix functions as Konrad has been advocating. I think having
these is a good idea; the type code that we have now has always seemed
somehow counter to (my view of) the Python 'do-it-the-Right-Way'
attitude. If I actually list them all out here, maybe this will force
the issue and start some discussion. Comments on the following?
Array_Char, Array_UChar
Array_Short, Array_UShort
Array_Integer,Array_UInteger,
Array_Float, Array_Double,
Array_FloatComplex, Array_DoubleComplex
'Unsigned' instead of 'U'? XArray instead of Array_X? BTW, is the C
Do we have unsigned types? I proposed a similar list of names a
while ago, using "XArray" instead of "Array_X". I prefer that form
because it is closer to the real-life expressions "integer array",
"float array" etc. But the main point is to get rid of these
silly type codes!
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Thu Jan 25 17:28:10 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Thu, 25 Jan 96 12:28:10 EST
Subject: [PYTHON MATRIX-SIG] type coercion one more time
In-Reply-To: <199601251709.MAA13702@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA)
Message-ID: <9601251728.AA09628@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad)
On the other hand, I see the other types as being used for cases where
the programmer really wants to use a particular precision, ie. 2-byte
integers for dealing with 16-bit sound data. In a case like this it
seems decidely unfriendly to automatically coerce to another
precision.
I wouldn't call it "decidedly unfriendly" (you can always avoid
coercion by making sure that all variables have the same type), but it
makes sense. It should also be noted that one of the main reasons to
have float->double coercion in C is to make it possible to call
functions written for double with float arguments. This problem does
not exist in Python.
This all is irrelevant if we can agree on the part below.
4) Coercion to a higher precision is allowed within the main
groupings.
What are "main groupings"?
int, float, complex (actually, string is also a grouping, but it
doesn't really pertain).
5) The precision of a python scalar is allowed to change to match the
other arguments (3.14 can be either a float or a double depending on
which one is "desired").
I don't like this rule for two reasons:
1) You can lose precision without noticing it.
You can also lose precision by coercing a long to a double, but
nobody's complaining about that. Also, the only place you can lose
precision is in a python scalar so I'm not really worried about this.
Admittedly, array([1,2,3], 'f')*3.1415926535 might cause you to lose
some precision in pi, but I assume that this is what most people would
want.
2) It creates a subtle distinction between scalars and arrays of rank 0,
which can only confuse users.
This is my own objection to the idea, but I think I have a way to make
this cleaner:
Add three more names, "GenericInt", "GenericFloat", "GenericComplex"
(or whatever).
Now I can say a = array(1, "GenericFloat") and this means that in
general a will behave as a double, but it can be coerced downwards to
a float if required.
Now, 3.14 -> array(3.14, "GenericFloat").
Does this seem conceptually clean?
On the other hand, I do see the problem you want to solve. It doesn't
make any code easier to read if you have to convert every constant to
a rank-0 float array. If I get an ingenious idea for solving this
problem, I'll let you know immediately...
I'm finally starting to feel happy about this proposal, but ingenious
ideas are always welcome.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Thu Jan 25 17:30:24 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Thu, 25 Jan 96 12:30:24 EST
Subject: [PYTHON MATRIX-SIG] type coercion one more time
In-Reply-To: <199601251715.MAA14003@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA)
Message-ID: <9601251730.AA09638@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad)
Also, I don't recall anyone presenting a list of 'nice' names for
the Matrix functions as Konrad has been advocating. I think having
these is a good idea; the type code that we have now has always seemed
somehow counter to (my view of) the Python 'do-it-the-Right-Way'
attitude. If I actually list them all out here, maybe this will force
the issue and start some discussion. Comments on the following?
Array_Char, Array_UChar
Array_Short, Array_UShort
Array_Integer,Array_UInteger,
Array_Float, Array_Double,
Array_FloatComplex, Array_DoubleComplex
'Unsigned' instead of 'U'? XArray instead of Array_X? BTW, is the C
Do we have unsigned types? I proposed a similar list of names a
while ago, using "XArray" instead of "Array_X". I prefer that form
because it is closer to the real-life expressions "integer array",
"float array" etc. But the main point is to get rid of these
silly type codes!
Currently the types are:
character
unsigned byte
signed byte
signed short
signed int
signed long
float
double
complex float
complex double
If you really want to add unsigned shorts, longs, and ints, I'd
obviously be willing, but the whole type collection already feels a
little bit large, so I'd rather wait and see if anybody ever NEEDS
them.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Thu Jan 25 18:46:05 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Thu, 25 Jan 1996 13:46:05 -0500
Subject: [PYTHON MATRIX-SIG] type coercion one more time
In-Reply-To: <9601251728.AA09628@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199601251846.NAA20452@cyclone.ERE.UMontreal.CA>
You can also lose precision by coercing a long to a double, but
nobody's complaining about that. Also, the only place you can lose
That's not exactly the same problem. More importantly, it is
a much less frequent problem.
precision is in a python scalar so I'm not really worried about this.
Admittedly, array([1,2,3], 'f')*3.1415926535 might cause you to lose
some precision in pi, but I assume that this is what most people would
want.
In this case, with an explicit constant, I agree that most people
would probably want a float result. But suppose you are using a
library function that does float calculations and returns a float
array as a result. You call this function and multiply its result with
a double that you have obtained from some complicated high-precision
calculation. Wouldn't you be both surprised and annoyed if the
result were a float array?
Add three more names, "GenericInt", "GenericFloat", "GenericComplex"
(or whatever).
Now I can say a = array(1, "GenericFloat") and this means that in
general a will behave as a double, but it can be coerced downwards to
a float if required.
Why not make it float from the beginning? If you can't rely on the
extra precision being conserved, you could just as well forget about
it. I don't see much use for "unknown precision" numbers.
Now, 3.14 -> array(3.14, "GenericFloat").
Does this seem conceptually clean?
Conceptually clean, yes. But it is a somewhat weird concept.
Let me start another approach to the problem: do you see a need for
"downcasting" anything else than constants? In that case one could
think about some special treatment for them.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Thu Jan 25 18:48:52 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Thu, 25 Jan 1996 13:48:52 -0500
Subject: [PYTHON MATRIX-SIG] type coercion one more time
In-Reply-To: <9601251730.AA09638@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199601251848.NAA20609@cyclone.ERE.UMontreal.CA>
If you really want to add unsigned shorts, longs, and ints, I'd
I didn't mean to suggest that we need more unsigned types!
I just wanted an inventory of existing types for the purpose
of finding names.
obviously be willing, but the whole type collection already feels a
little bit large, so I'd rather wait and see if anybody ever NEEDS
them.
Definitely!
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Thu Jan 25 19:01:34 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Thu, 25 Jan 96 14:01:34 EST
Subject: [PYTHON MATRIX-SIG] type coercion one more time
In-Reply-To: <199601251846.NAA20452@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca)
Message-ID: <9601251901.AA10739@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
precision is in a python scalar so I'm not really worried about this.
Admittedly, array([1,2,3], 'f')*3.1415926535 might cause you to lose
some precision in pi, but I assume that this is what most people would
want.
In this case, with an explicit constant, I agree that most people
would probably want a float result. But suppose you are using a
library function that does float calculations and returns a float
array as a result. You call this function and multiply its result with
a double that you have obtained from some complicated high-precision
calculation. Wouldn't you be both surprised and annoyed if the
result were a float array?
I'd be tempted to say that if you have a high-precision double that
you're really concerned about then you'll just have to say something
like pi = array(3.1415926535, 'd'). In general, library functions
should be expected to only return longs, doubles and complex doubles
unless they are somehow "told otherwise". This is just my opinion.
Add three more names, "GenericInt", "GenericFloat", "GenericComplex"
(or whatever).
Now I can say a = array(1, "GenericFloat") and this means that in
general a will behave as a double, but it can be coerced downwards to
a float if required.
Why not make it float from the beginning? If you can't rely on the
extra precision being conserved, you could just as well forget about
it. I don't see much use for "unknown precision" numbers.
The reason to not just make it float is that 3. * Array_l should
produce an array of doubles, not of floats. Also, if a function can
accept either floats or doubles, 3. should be treated as a double.
Now, 3.14 -> array(3.14, "GenericFloat").
Does this seem conceptually clean?
Conceptually clean, yes. But it is a somewhat weird concept.
Weird I can agree with.
Let me start another approach to the problem: do you see a need for
"downcasting" anything else than constants? In that case one could
think about some special treatment for them.
I don't really see much difference between python scalars and
constants. The problem with both of them is that they have double
precision no matter what. I think that it's important to allow these
types to be downcast to a lower precision. (Or don't allow any
automatic coercion of precision, but that original proposal seems to
be an extremely unpopular idea based on some private responses I've
gotten).
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dubois1@llnl.gov Thu Jan 25 19:43:31 1996
From: dubois1@llnl.gov (Paul. Dubois)
Date: Thu, 25 Jan 1996 11:43:31 -0800
Subject: [PYTHON MATRIX-SIG] type coercion one more time
References: <9601251759.AA10036@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <3107DD63.7CAB@llnl.gov>
I have been busy giving talks and attending meetings, etc., and had a
hard time plowing through all this carefully. If I understand the latest
idea with respect to precision, I think I can live with it but I don't
like it. It would violate the principle of least surprise.
It essentially changes the precision of the scalars in Python iff they
get involved with doing something with Array_f's. I don't like
exceptions in languages. Exceptions lead to weird behavior.
For example, we all agree it is a Good Thing that x[i] is a scalar not
an array of size 1. But in fact this means that
(1./3.) * x[i] is not ((1./3.) * x)[i] if x is Array_f.
I agree it is nice to go fast but let's not go too fast.
I thought the "generic" stuff was off track. Everybody who writes
anything that handles Arrays has enough trouble without one more type to
look for.
Jim, what you need to do in your own work can be pretty much solved with
a little literal function:
def lit(d):
return Array_f(d)
so that you just always say lit(2.) rather than 2. You have to train
yourself to do this in the places where it matters, true, but otherwise
you are just going to make the rest of us miserable.
--
Paul F. Dubois, L-472 (510)-422-5426
Lawrence Livermore National Laboratory FAX (510)-423-9969
Livermore, CA 94550 dubois1@llnl.gov
Consulting: PaulDubois@aol.com
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Thu Jan 25 21:26:37 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Thu, 25 Jan 96 16:26:37 EST
Subject: [PYTHON MATRIX-SIG] type coercion one more time
In-Reply-To: <3107DD63.7CAB@llnl.gov> (dubois1@llnl.gov)
Message-ID: <9601252126.AA12494@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Paul's and Konrad's arguments make me think that this whole type
coercion issue needs to be reevaluated before mistakenly commiting to
a bad strategy. I still want to release a 0.30 version tommorrow,
hopefully with stable naming conventions and API. The only things I
have left to do are packaging and tests on our alphas and pentiums for
some semblance of portability. Unfortunately, this release will have
an interim API that is highly unlikely to be the final solution, oh
well.
Sender: dubois@llnl.gov
I have been busy giving talks and attending meetings, etc., and had a
hard time plowing through all this carefully. If I understand the latest
idea with respect to precision, I think I can live with it but I don't
like it. It would violate the principle of least surprise.
I agree with this completely. This is why I proposed the awkward
Array_f*3.14 -> Exception solution. For me having this go to an
Array_d is not what I am expecting.
It essentially changes the precision of the scalars in Python iff they
get involved with doing something with Array_f's. I don't like
exceptions in languages. Exceptions lead to weird behavior.
For example, we all agree it is a Good Thing that x[i] is a scalar not
an array of size 1. But in fact this means that
(1./3.) * x[i] is not ((1./3.) * x)[i] if x is Array_f.
This is a great example of where my newer proposal has problems.
I agree it is nice to go fast but let's not go too fast.
I thought the "generic" stuff was off track. Everybody who writes
anything that handles Arrays has enough trouble without one more type to
look for.
I don't think this will actually impact people much beyond the sort of
problems you mention above. It's mainly just to keep things
reasonably clean conceptually.
Jim, what you need to do in your own work can be pretty much solved with
a little literal function:
def lit(d):
return Array_f(d)
so that you just always say lit(2.) rather than 2. You have to train
yourself to do this in the places where it matters, true, but otherwise
you are just going to make the rest of us miserable.
Well, the last thing I want to do is to make anybody miserable, but I
still think that I have a reasonable point here.
With the current (0.20) type coercion semantics, python scalars are
completely useless in an equation unless you want to work with longs,
doubles, or complex doubles. The new proposal, while slightly
strange, at least gives a reasonable meaning to python scalars for
other types.
Just to make things concrete, I picked a reasonable simple function
that I use a lot of the time. Do you find this readable?
def hamming(x):
return lit(0.54)-lit(0.46)*cos(lit(2*pi)*x/lit(len(x)-1))
vs.
def hamming(x):
return 0.54-0.46*cos(2*pi*x/(len(x)-1))
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Thu Jan 25 19:17:58 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Thu, 25 Jan 1996 14:17:58 -0500
Subject: [PYTHON MATRIX-SIG] type coercion one more time
In-Reply-To: <9601251901.AA10739@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199601251917.OAA22980@cyclone.ERE.UMontreal.CA>
like pi = array(3.1415926535, 'd'). In general, library functions
should be expected to only return longs, doubles and complex doubles
unless they are somehow "told otherwise". This is just my opinion.
Library routines might need floats for the same reason they were
introduced: efficiency. And converting the result to double just
to conform to a specification is a bad idea if the caller wants
to go on using floats.
The reason to not just make it float is that 3. * Array_l should
produce an array of doubles, not of floats. Also, if a function can
accept either floats or doubles, 3. should be treated as a double.
I understand that, but don't see the point. Either I want double
precision, and then I want it under circumstances, or I am happy
with float, and then I start with float. For what application
might I possible want a number of unknown precision that is as
expensive as double, but can't be trusted to be more than float?
That's the worst of both types.
I don't really see much difference between python scalars and
constants. The problem with both of them is that they have double
Not in implementation, but in use. Most constants occurring in real
programs can be represented by a float, but that is not true for an
arbitrary Python scalar that might be the result of a calculation. So
suppose (in principle; I don't want to do this!) that we add a type
"float" to Python and make all constants that do not exceed the
precision of "float" float constants (with automatic coercion to
double if the need arises), would you say that your problem is solved?
In fact, now that I think of it, this idea is not completely
unrealistic. A new type "float" could be seen as an implementation
alternative to "double" and wouldn't even have to be visible to the
programmer. I'll have to think a bit about this.
types to be downcast to a lower precision. (Or don't allow any
automatic coercion of precision, but that original proposal seems to
be an extremely unpopular idea based on some private responses I've
gotten).
Given the choice between the two proposals, I very much prefer the
first one. An occasional exception (whis is easy to fix) is a lot
better than a loss in accuracy without warning.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From stoll@atr-sw.atr.co.jp Fri Jan 26 07:37:38 1996
From: stoll@atr-sw.atr.co.jp (Perry A. Stoll)
Date: Fri, 26 Jan 1996 16:37:38 +0900
Subject: [PYTHON MATRIX-SIG] Type coercion two more times
Message-ID: <199601260737.QAA03317@ciris21.atr-sw.atr.co.jp>
I'm about 12 hours behind all of you in North America, so I come to
work in the morning and read all the activity of the previous
day. Most of the debate is over by the time I read it, so I have to
spend the day working instead ;-)
Here's the situation now:
- There is support for three scalar numerical types in Python:
PyInteger ( C long )
PyFloat ( C double )
PyComplex ( C double complex )
- The multiarray object is introducing (providing access to)
operations for many more C numerical types.
The question is how do we handle operations that mix C numerical types
not in Python (i.e. C float, C short) with Python supported C types (C
long,C double)?
In the following, I'm going to use Jim's term of "3 Classes" of
numerical types ( Integer, Float, Complex) to mean the set of
corresponding C types:
Integer = any of the C byte,short,int, or long
Float = either C float or C double
Complex = either C float complex or C double complex
--------------------------------------------------
The Proposal:
We define an optional attribute for multiarrays, say, oh, I don't
know:
array.__maintain_ctype__, or
array.__avoid_coercion__, or
array.__do_no_coerce_me__ (?!?)
- Coercions within a Class (Integer<->Integer,Float<->Float) are
governed by this attribute, namely,
If this attribute exists and is TRUE:
then the result will be the same type as the multiarray.
If the attribute does not exists or is FALSE,
then NORMAL Python coercion rules apply, and
the result will always be a PyInteger, PyFloat, or PyComplex.
- Coercions to a higher Class (i.e. Integer-> Float -> Complex) always
occur if necessary. The result will be the normal Python type for that
Class, namely PyInteger, PyFloat, or PyComplex.
- If both operands have this attribute set but the array types
disagree, raise an exception. Operations involving two multiarrays can
be dealt with this way.
- Correspondingly, add a keyword to the Array creation function, i.e.
>>> myshortarray = Array.array([1,2,3,4],'s',nocoercion=1)
--------------------------------------------------
The Explanation
Normally, this attribute would not be set, so operations will follow
normal Python rules. That makes it consistant with the rest of Python
for new and old users.
But when Jim reads in a C float file, he sets the
"__do_not_coerce_me__" attribute, letting the C multiarray code know
he expects operations involving this array and other Floats to result
in C float arrays.
--------------------------------------------------
Comments?
-Perry
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Fri Jan 26 14:54:08 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Fri, 26 Jan 1996 09:54:08 -0500
Subject: [PYTHON MATRIX-SIG] Type coercion two more times
In-Reply-To: <199601260737.QAA03317@ciris21.atr-sw.atr.co.jp> (stoll@atr-sw.atr.co.jp)
Message-ID: <199601261454.JAA23121@cyclone.ERE.UMontreal.CA>
I'm about 12 hours behind all of you in North America, so I come to
work in the morning and read all the activity of the previous
On the contrary, you are 14 hours ahead of us (us on the East coast
at least), so you could complete a whole discussion ready for
us every morning ;-)
- Coercions within a Class (Integer<->Integer,Float<->Float) are
governed by this attribute, namely,
If this attribute exists and is TRUE:
then the result will be the same type as the multiarray.
If the attribute does not exists or is FALSE,
then NORMAL Python coercion rules apply, and
the result will always be a PyInteger, PyFloat, or PyComplex.
Basically that means having two array types with different behaviour,
and hiding that distinction in an attribute. Which means most users
won't be aware of its existence and will be surprised the first time
they get something else than they expect.
Besides, it doesn't solve Jim's problem of mixing C float arrays with
Python float scalars. If you consider scalars to be of "coercible"
type, then you run into the same problem as with Jim's second
proposal. If not, then the problem is the same as with full automatic
coercion.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From stoll@atr-sw.atr.co.jp Fri Jan 26 15:35:12 1996
From: stoll@atr-sw.atr.co.jp (Perry A. Stoll)
Date: Sat, 27 Jan 1996 00:35:12 +0900
Subject: [PYTHON MATRIX-SIG] Type coercion two more times
In-Reply-To: Your message of "Fri, 26 Jan 1996 09:54:08 JST"
References: <199601261454.JAA23121@cyclone.ERE.UMontreal.CA>
Message-ID: <199601261535.AAA03838@ciris21.atr-sw.atr.co.jp>
>>>>> "HK" == Hinsen Konrad <hinsenk@ERE.UMontreal.CA> writes:
HK> On the contrary, you are 14 hours ahead of us (us on the East
Yes, I realized that this afternoon as I left work for the
weekend. BTW, have a good day at work/school today :-) But don't ask
why I'm logged in after midnight to read this mail :-)
HK> Basically that means having two array types with different
HK> behaviour, and hiding that distinction in an attribute. Which
Maybe, but it seems like we're trying to satisfy two conflicting
constraints, namely,
1) providing a nice, consistant array library usable for the
mathematically oriented (potential) Python user, and
2) providing easy access to custom, relatively low level number
crunching and control of C types, i.e. 16 bit speech data, 8 bit
images, etc.
HK> means most users won't be aware of its existence and will be
HK> surprised the first time they get something else than they
HK> expect.
Most users won't care about its existance unless they fall into group
2) above. Then they must care about data type conversion and this
provides some degree of control. I'm not saying it's good; this is a
real problem.
HK> Besides, it doesn't solve Jim's problem of mixing C float
HK> arrays with Python float scalars.
I thought it did - can you explain this a bit more? I'm probably
missing something.
HK> If you consider scalars to be of "coercible" type, then you
HK> run into the same problem as with Jim's second proposal.
I see. Yes, the question of whether or not to to convert scalars is
pivotal. If someone is using a type other than a Python numeric
type, then doesn't that mean they are willing to accept the loss of
precision? If you have an array_f, do you expect calculations with
PyFloats to occur at C double precision? I say by using an array_f,
you say "I want C float precision!".
-Perry
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Fri Jan 26 15:51:15 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Fri, 26 Jan 1996 10:51:15 -0500
Subject: [PYTHON MATRIX-SIG] Type coercion two more times
In-Reply-To: <199601261535.AAA03838@ciris21.atr-sw.atr.co.jp> (stoll@atr-sw.atr.co.jp)
Message-ID: <199601261551.KAA28001@cyclone.ERE.UMontreal.CA>
Maybe, but it seems like we're trying to satisfy two conflicting
constraints, namely,
1) providing a nice, consistant array library usable for the
mathematically oriented (potential) Python user, and
2) providing easy access to custom, relatively low level number
crunching and control of C types, i.e. 16 bit speech data, 8 bit
images, etc.
You seem to imply that the additional types will be used only by
people in group 2. But I expect that C floats will also be used by
ordinary mathematically oriented Python users, whenever memory
or CPU time are more important than accuracy. Most of my numerical
programs (mostly in Fortran) use both precisions, so why should
Python code be different?
I see. Yes, the question of whether or not to to convert scalars is
pivotal. If someone is using a type other than a Python numeric
type, then doesn't that mean they are willing to accept the loss of
precision? If you have an array_f, do you expect calculations with
Sometimes. There are two situations where I expect problems:
1) Someone uses a library written by someone else, and this
library uses C floats. The user of this library may not be
aware of this, or might not care. Until he experiences a
mysterious loss of precision in his code...
2) Someone wants to use both precisions, and either forgets
about the coercion problems, or expects them to be like in
all other languages (which coerce float->double, but not
the reverse).
Therefore I consider Jim's first proposal (no coercion between
float and double) still the best: those who want both precisions
and do everything correctly get what they want, and those who
make a mistake or have misunderstood the rules get an exception.
I'd rather be surprised by an exception than by loss of precision.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From stoll@atr-sw.atr.co.jp Fri Jan 26 16:21:57 1996
From: stoll@atr-sw.atr.co.jp (Perry A. Stoll)
Date: Sat, 27 Jan 1996 01:21:57 +0900
Subject: [PYTHON MATRIX-SIG] Type coercion two more times
In-Reply-To: Your message of "Fri, 26 Jan 1996 10:51:15 JST"
References: <199601261551.KAA28001@cyclone.ERE.UMontreal.CA>
Message-ID: <199601261621.BAA03904@ciris21.atr-sw.atr.co.jp>
You seem to imply that the additional types will be used only by
people in group 2.
I had been implicitly assuming that. I was basing this on the
precendent set by Python numerical types.
Therefore I consider Jim's first proposal (no coercion between
float and double) still the best: those who want both precisions
and do everything correctly get what they want, and those who
make a mistake or have misunderstood the rules get an exception.
Ok, I'm starting to be convinced. My last hurdle to saying yes is
having to explain the following to potential Python array users:
>>> pi = omath.arctan(1.0)*4.0
>>> m = Array_f([1.0,2.0,3.0])
>>> m * pi
Traceback (innermost last):
File "<stdin>", line 1, in ?
TypeError: inconsistant floating point types
-Perry
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Fri Jan 26 16:28:26 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Fri, 26 Jan 96 11:28:26 EST
Subject: [PYTHON MATRIX-SIG] Type naming conventions
Message-ID: <9601261628.AA00464@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Returning one last time to the issue of naming conventions. (I have
some time on my hands until a type coercion strategy is decided on).
I'd like to propose adding the following functions to Numeric.py.
Character
Integer
Float
Complex
On a typical system, Integer(16) would return "s" (the typecode for an
int).
Integer(64) would raise an exception on most systems, but would return
"l" on an alpha.
Thus you could write things like array([1,2,3], Integer(32)).
Things like Integer(minimumSize=16) would also be possible if we
agreed they made sense. You'd also want to have Integer() just return
the typecode used by python ints.
This could be integrated with the standard array print function to
also have more pleasing outputs.
I know that Paul DuBois had some good comments on this previously in
regards to FORTRAN 90. I'd be curious to know if this new proposal is
at all useful or reasonable.
Opinions?
-Jim
PS - Thanks to Mike McLay for the note that made me think of this again.
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Fri Jan 26 16:26:38 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Fri, 26 Jan 1996 11:26:38 -0500
Subject: [PYTHON MATRIX-SIG] Type coercion two more times
In-Reply-To: <199601261621.BAA03904@ciris21.atr-sw.atr.co.jp> (stoll@atr-sw.atr.co.jp)
Message-ID: <199601261626.LAA01120@cyclone.ERE.UMontreal.CA>
Ok, I'm starting to be convinced. My last hurdle to saying yes is
having to explain the following to potential Python array users:
>>> pi = omath.arctan(1.0)*4.0
>>> m = Array_f([1.0,2.0,3.0])
>>> m * pi
Traceback (innermost last):
File "<stdin>", line 1, in ?
TypeError: inconsistant floating point types
It could be made a bit easier by changing the text of the
error message:
TypeError: no automatic coercion between precisions
Yet I agree, this is not a desirable feature. It is just the most
acceptable among the evils we have to choose from.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Fri Jan 26 16:35:47 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Fri, 26 Jan 96 11:35:47 EST
Subject: [PYTHON MATRIX-SIG] Type coercion two more times
In-Reply-To: <199601261626.LAA01120@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca)
Message-ID: <9601261635.AA00493@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Ok, I'm starting to be convinced. My last hurdle to saying yes is
having to explain the following to potential Python array users:
>>> pi = omath.arctan(1.0)*4.0
>>> m = Array_f([1.0,2.0,3.0])
>>> m * pi
Traceback (innermost last):
File "<stdin>", line 1, in ?
TypeError: inconsistant floating point types
It could be made a bit easier by changing the text of the
error message:
TypeError: no automatic coercion between precisions
Yet I agree, this is not a desirable feature. It is just the most
acceptable among the evils we have to choose from.
An important thing to remember here is that a naive user should never
write m = array([1.0,2.0,3.0], 'f') (the new syntax). They should
just write m = array([1.0,2.0,3.0]) which will produce an array of
doubles and should allow everybody to live happily ever after.
Not that I particularly like the type error either...
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From guido@CNRI.Reston.VA.US Fri Jan 26 16:32:45 1996
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Fri, 26 Jan 1996 11:32:45 -0500
Subject: [PYTHON MATRIX-SIG] Type coercion two more times
In-Reply-To: Your message of "Sat, 27 Jan 1996 01:21:57 +0900."
<199601261621.BAA03904@ciris21.atr-sw.atr.co.jp>
References: <199601261551.KAA28001@cyclone.ERE.UMontreal.CA>
<199601261621.BAA03904@ciris21.atr-sw.atr.co.jp>
Message-ID: <199601261632.LAA25356@monty>
Re confusing Python users: how confusing is Python's use of the words
"float" for "double" and "int" for "long"? Is it more important to be
consistent with C or with Python? This may affect the names chosen
for the array types (Array_float, etc.)...
--Guido van Rossum <guido@CNRI.Reston.VA.US>
URL: <http://www.python.org/~guido/>
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From da@maigret.cog.brown.edu Fri Jan 26 17:03:29 1996
From: da@maigret.cog.brown.edu (David Ascher)
Date: Fri, 26 Jan 1996 12:03:29 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] Type coercion two more times
In-Reply-To: <9601261635.AA00493@baribal.LCS.MIT.EDU.LCS.MIT.EDU> from "James Hugunin" at Jan 26, 96 11:35:47 am
Message-ID: <199601261703.MAA03624@maigret>
> having to explain the following to potential Python array users:
> >>> pi = omath.arctan(1.0)*4.0
> >>> m = Array_f([1.0,2.0,3.0])
> >>> m * pi
> Traceback (innermost last):
> File "<stdin>", line 1, in ?
> TypeError: inconsistant floating point types
>
> It could be made a bit easier by changing the text of the
> error message:
> TypeError: no automatic coercion between precisions
I have no idea whether this is possible or not, but one area where I
think Python needs help is the detail of the exceptions. For example,
would it be possible at all to have at least the two uncoercable types
listed ("first argument is a single precision float, second argument is
a double precision float")?
--david
PS: This doesn't have to be in the next release, just something to keep
in mind. A certain set of users at least will want to use the array
tool in an interactive context, where these things matter a lot.
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Fri Jan 26 17:08:12 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Fri, 26 Jan 1996 12:08:12 -0500
Subject: [PYTHON MATRIX-SIG] Type coercion two more times
In-Reply-To: <199601261632.LAA25356@monty> (message from Guido van Rossum on Fri, 26 Jan 1996 11:32:45 -0500)
Message-ID: <199601261708.MAA05505@cyclone.ERE.UMontreal.CA>
Re confusing Python users: how confusing is Python's use of the words
"float" for "double" and "int" for "long"? Is it more important to be
consistent with C or with Python? This may affect the names chosen
for the array types (Array_float, etc.)...
Internal consistency is more important. I don't think of Python as a
dialect of C, and many potential numerical users won't know C at all.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Fri Jan 26 17:05:45 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Fri, 26 Jan 1996 12:05:45 -0500
Subject: [PYTHON MATRIX-SIG] Type naming conventions
In-Reply-To: <9601261628.AA00464@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199601261705.MAA05321@cyclone.ERE.UMontreal.CA>
I'd like to propose adding the following functions to Numeric.py.
Character
Integer
Float
Complex
I like the idea, but would prefer names like CharacterType, IntType etc.
"Integer" looks too much like a cast to integer type.
Things like Integer(minimumSize=16) would also be possible if we
agreed they made sense. You'd also want to have Integer() just return
the typecode used by python ints.
The latter makes more sense to me.
To be precise, I'd like these functions to return a typecode for a
type that is at least as big as indicated, not necessarily exactly as
big. For integers this is not so important (the sizes are always
8/16/32/64 on modern machines), but there is more variation for float
types. In fact the FloatType() function should perhaps have two
arguments, one specifying precision and the other range. Then
FloatType(10, 50) would specify a type with an accuracy of 10 or more
decimal digits and a range of at least 10**50 (which would be "double"
on most machines, but "float" on a Cray).
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Fri Jan 26 23:39:56 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Fri, 26 Jan 96 18:39:56 EST
Subject: [PYTHON MATRIX-SIG] Final pre-beta (0.30) of NumericalPython is available!
Message-ID: <9601262339.AA04290@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Release 0.30 of the numerical extension to python is available.
Now that the design appears to be final, I'd like as many people as
possible to install this system and play with it. Once it is released
as 1.0BETA1, any changes to the fundamental design will be very
unlikely to happen, so if you want to have a final system that you'll
be happy with, now's the time to speak up.
You can find the code at:
ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericalPython-0.30.tar.gz
If somebody (Tom are you there?) would like to place this on a site in
Europe and post the location, I'd appreciate it. It's frustrating
just how hard it can be to cross the Atlantic these days.
The easiest way to install it is to untar the file in your top level
python directory and follow the instructions in INSTALL.NumPy. Please
do read this file as it contains important information (unlike the
currently vacuous README).
The naming conventions and C API of this version should be final. If
this release goes well, the next release will be 1.0BETA1 to the
general python community. I intend to produce a new bug-fixed release
every week until I manage to go a week without any bug reports, and
then I'll release 1.0BETA, so PLEASE report any bugs you find so that
they can be squashed quickly.
The code is running reasonably stably right now, and I've compilied it
without a problem under the following configurations:
alpha DECOS cc
Sun SUNOS4 gcc
P90 NextStep3 cc
If you do grab the code and try to compile it, please send me mail
(hugunin@mit.edu) telling me the machine and OS you used, and whether
or not you could get it to compile, and whether or not you could run
the TestSuite (ArrayTest/TestSuite.py). I'd also like to know the
time reported by the Benchmark in the test suite just out of general
curiosity.
You should also check out TUTORIAL.NumPy for a VERY brief summary of
how to use the object.
Enjoy - Jim
Things that probably will change before the BETA release:
-1 The print function sucks! There is a function in Numeric.py called
print_array that is the function that gets invoked whenever an array
is printed. Currently it displays a very ugly representation. I
really hope that somebody will write a nice pretty printing function
to go here (you get to code it in python instead of C!).
-2 The exact contents of Numeric.py, I'm very interested in comments
-3 The type coercion scheme. If you only work with longs, doubles,
and complex doubles (the basic python types) you can consider the
coercion scheme fixed and stable. If you use other types, this is the
current scheme:
Only the following three automatic coercions are allowed:
1) Any integer type can be coerced to a float, a double, a complex float or
a complex double.
2) A float can only be coerced to a complex float.
3) A double can only be coerced to a complex double.
Explanation:
The following type hierarchy is assumed:
INTEGER < FLOAT < COMPLEX
Within each level there are a number of possible precisions. No
automatic coercion will take place between these precisions.
This remains completely consistent with the existing python automatic
type coercion.
This scheme seems to make the most people the least unhappy, but I'm
still interested in better ideas.
-4 The Precision.py stuff I just coded in the last half hour, so
obviously it might change in the near future. I am convinced that
something like this is the right approach.
-5 If Jim Fulton implements a nice dynamically loadable version using
CObjects and I can easily install it on my SUNs (I'm usually clueless
about dynamic loading issues) then this will be used.
-6 Now that the design seems stable its time to write documentation.
I'm not enough of a fool to widely release something without decent
documentation. I've heard rumors that Paul DuBois might be helping
out with this. Anybody else who'd like to contribute?
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From stoll@atr-sw.atr.co.jp Sat Jan 27 05:09:12 1996
From: stoll@atr-sw.atr.co.jp (Perry A. Stoll)
Date: Sat, 27 Jan 1996 14:09:12 +0900
Subject: [PYTHON MATRIX-SIG] Final pre-beta (0.30) of NumericalPython is
available!
In-Reply-To: Your message of "Fri, 26 Jan 1996 18:39:56 JST"
References: <9601262339.AA04290@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <199601270509.OAA04528@ciris21.atr-sw.atr.co.jp>
Being 14 hours ahead may have given me the honor of being the first to
download and try the code. As usual, being on the cutting edge means
you can get cut, to wit:
the file Modules/arraytypes.c is NOT IN THE DISTRIBUTION!!!
I'm trying to use the old matrixtypes.c (converting matrix ->
array). It compiles, but I'll let you know if it works or not...
-Perry
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Sat Jan 27 16:49:14 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Sat, 27 Jan 96 11:49:14 EST
Subject: [PYTHON MATRIX-SIG] Oops! Problems with 0.30 distribution
Message-ID: <9601271649.AA17704@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
My apologies to all (and thanks to Perry and Konrad for being the guinea
pigs), but I left a file out of the distribution.
This has been fixed now, and the file NumericalPython-0.30.tar.gz should
now contain all of the needed files for a successful compile.
If you've already downloaded the distribution, you can just get the
file arraytypes.c and put it in your Modules directory. This is the file
I left out.
I hope this solves the problem - Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Sun Jan 28 18:21:22 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Sun, 28 Jan 96 13:21:22 EST
Subject: [PYTHON MATRIX-SIG] New release of NumericPython 0.31
Message-ID: <9601281821.AA06247@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
There turned out to be a small bug in the test suite pertaining to
complex numbers, so I produced a new release that fixes this.
When untarring the NumericPython release, as well as patches.tar (Konrad's)
patches to the python core), you should use tar -xvfm. The m tells
tar to set the modification times on the files to the current time, and
this will ensure that a new build will not use old object files.
It was forgetting to do this myself that led to my latest build not including
Konrad's latest patch version, and the bug in the test suite.
For those who don't want to download a complete new version, you just need
to pick up TestCreation.py and put it in ArrayTest.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Mon Jan 29 02:37:00 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Sun, 28 Jan 1996 21:37:00 -0500
Subject: [PYTHON MATRIX-SIG] New release of NumericPython 0.31
In-Reply-To: <9601281821.AA06247@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199601290237.VAA05601@cyclone.ERE.UMontreal.CA>
When untarring the NumericPython release, as well as patches.tar (Konrad's)
patches to the python core), you should use tar -xvfm. The m tells
tar to set the modification times on the files to the current time, and
this will ensure that a new build will not use old object files.
Also make sure that you remove the old files Lib/Matrix.py and
Lib/MLab.py, otherwise they will be loaded instead of their new
replacements in the directory "numeric". And don't worry if
"TestPrecision" reports bad results; that's just a bug in the test
suite. I got the whole test suite running except for the final speed
benchmark, which uses a function "sum" that no longer exists
(should it?).
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From stoll@atr-sw.atr.co.jp Mon Jan 29 03:03:10 1996
From: stoll@atr-sw.atr.co.jp (Perry A. Stoll)
Date: Mon, 29 Jan 1996 12:03:10 +0900
Subject: [PYTHON MATRIX-SIG] New release of NumericPython 0.31
In-Reply-To: Your message of "Sun, 28 Jan 1996 21:37:00 JST"
References: <199601290237.VAA05601@cyclone.ERE.UMontreal.CA>
Message-ID: <199601290303.MAA08416@ciris21.atr-sw.atr.co.jp>
> I got the whole test suite running except for the final speed
> benchmark, which uses a function "sum" that no longer exists
> (should it?)
Yes, it exists! It's in MLab.py. You have to add 'from MLab import
*' to the top of the Benchmark.py file.
This means Jim has removed the MLab support from the default Numeric
library. Is this ok with people? I don't mind, but we're going to have
to let people
Can I make a suggestions: can we avoid the 'from X import *'
construction in the numeric package where it's not speed critical
(??)- it makes finding the source of definitions a pain.
-Perry
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Mon Jan 29 13:23:36 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Mon, 29 Jan 1996 08:23:36 -0500
Subject: [PYTHON MATRIX-SIG] New release of NumericPython 0.31
In-Reply-To: <199601290303.MAA08416@ciris21.atr-sw.atr.co.jp> (stoll@atr-sw.atr.co.jp)
Message-ID: <199601291323.IAA00117@cyclone.ERE.UMontreal.CA>
Yes, it exists! It's in MLab.py. You have to add 'from MLab import
*' to the top of the Benchmark.py file.
Right. I also had to add "from fast_umath import *" to MLab.py,
otherwise it wouldn't know how add.reduce. Note that "from umath import *"
wouldn't work with the benchmark because it produces an overflow:
Traceback (innermost last):
File "Benchmark.py", line 226, in ?
test2(10)
File "Benchmark.py", line 46, in test2
esc= escout(zt, akap, srcfun)
File "Benchmark.py", line 115, in escout
exf= exp(outer(multiply,-1./mu,tau))
OverflowError: math range error
This is of course something the benchmark author should worry about.
This means Jim has removed the MLab support from the default Numeric
library. Is this ok with people? I don't mind, but we're going to have
to let people
I'd have done the same. I see the Matlab compatibility module as
something distinct from the standard array/matrix function, because
the differ in some conventions. Mixing this up would only generate
confusion.
Can I make a suggestions: can we avoid the 'from X import *'
construction in the numeric package where it's not speed critical
(??)- it makes finding the source of definitions a pain.
This is not a matter of speed; the import takes hardly any time. It
is a matter of typing convenience. It might indeed be a good idea to
to replace this with an explicit list (e.g. "from umath import acos,
asin, atan, ...") in the standard modules, but that should be done
only in the final version, to avoid updating problems.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From friedric@rose.rsoc.rockwell.com Mon Jan 29 14:43:45 1996
From: friedric@rose.rsoc.rockwell.com (Robin Friedrich)
Date: Mon, 29 Jan 1996 08:43:45 -0600
Subject: [PYTHON MATRIX-SIG] New release of NumericPython 0.31
Message-ID: <199601291443.IAA02517@darwin.rsoc.rockwell.com>
Having a few problems with the Numeric release.
I built 0.30 with tar xvfm as instructed. (I'm not using .31 yet but I don't
think it relates to this problem.)
>From the tutorial:
1) First problem is with the types.py file. For some reason when it's imported
vars(__builtins__) is an illegal argument while it works fine from the prompt.
Traceback (innermost last):
File "<stdin>", line 1, in ?
File "/local2/tools/lib/python/numeric/Numeric.py", line 9, in ?
import string, types, math
File "/local2/tools/lib/python/types.py", line 13, in ?
if vars(__builtins__).has_key('complex'):
TypeError: vars() argument must have __dict__ attribute
2) The following basic function is worrisome as well...
>>> m = Matrix([[1,2,3], [4,5,6], [7,8,9]])
>>> m*m
Traceback (innermost last):
File "<stdin>", line 1, in ?
File "/local2/tools/lib/python/numeric/Matrix.py", line 74, in __mul__
NameError: MathError
>>>
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From stoll@atr-sw.atr.co.jp Mon Jan 29 15:27:44 1996
From: stoll@atr-sw.atr.co.jp (Perry A. Stoll)
Date: Tue, 30 Jan 1996 00:27:44 +0900
Subject: [PYTHON MATRIX-SIG] New release of NumericPython 0.31
In-Reply-To: Your message of "Mon, 29 Jan 1996 08:43:45 JST"
References: <199601291443.IAA02517@darwin.rsoc.rockwell.com>
Message-ID: <199601291527.AAA10895@ciris21.atr-sw.atr.co.jp>
1) Problem with: if vars(__builtins__).has_key('complex'):
TypeError: vars() argument must have __dict__ attribute
yep, I had the same problem. The easiest way to get around this and
yet have the same functionality is to comment out the following lines
in $PYLIB/types.py:
#if vars(__builtins__).has_key('complex'):
# ComplexType = type(complex(0,1))
And add:
try:
ComplexType = type(complex(0,1))
except:
pass
I know how __builtins__ is becoming a dictioary instead of a module
(in Python/ceval.c), but i don't know why. Anyone? Guido?
I don't know about problem 2), although I just tried the same thing ,
namely a matrix multiply, and had got errors. Hey, what would a
prebeta be without a few some problems? :-)
-Perry
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Mon Jan 29 15:53:34 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Mon, 29 Jan 1996 10:53:34 -0500
Subject: [PYTHON MATRIX-SIG] New release of NumericPython 0.31
In-Reply-To: <199601291443.IAA02517@darwin.rsoc.rockwell.com> (friedric@rose.rsoc.rockwell.com)
Message-ID: <199601291553.KAA09076@cyclone.ERE.UMontreal.CA>
>From the tutorial:
1) First problem is with the types.py file. For some reason when it's imported
vars(__builtins__) is an illegal argument while it works fine from the prompt.
You can solve this temporarily by removing the call to vars(). It
seems that inside a module definition, __builtins__ is not a module,
but a dictionary. I suspect this is a bug in Python, until someone
can explain me the reason...
2) The following basic function is worrisome as well...
>>> m = Matrix([[1,2,3], [4,5,6], [7,8,9]])
>>> m*m
Traceback (innermost last):
File "<stdin>", line 1, in ?
File "/local2/tools/lib/python/numeric/Matrix.py", line 74, in __mul__
NameError: MathError
>>>
When I try this here, it is a bit different:
Traceback (innermost last):
File "test.py", line 3, in ?
m*m
File "/usr/people/hinsenk/lib/python/numeric/Matrix.py", line 5, in __mul__
return self.__return_constructor__(self.array.matrixMultiply(other))
AttributeError: __int__
This indicates that some function can't deal with an int matrix.
What puzzles me in your error message is the indication "line 74".
Matrix.py is rather short!
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Mon Jan 29 16:21:27 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Mon, 29 Jan 96 11:21:27 EST
Subject: [PYTHON MATRIX-SIG] And onto release 0.32
In-Reply-To: <199601291553.KAA09076@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca)
Message-ID: <9601291621.AA18040@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
The good news is that most of the bugs found so far are extremely
minor, the bad news is that so many small bugs slipped by me on
Friday.
I have a new release that fixes all of the bugs reported so far
Numeric.py lacked some definitions
UserArray and Matrix had a serious bug
small warning in ofuncobject
small bugs in TestSuite
the bug in types.py (which strangely doesn't show up on my system)
ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericPython-0.32.tar.gz
I'd also like to remind people of Konrad's warning to remove any
previous versions of Matrix.py that might be in your python library
path.
Similarly, anybody interested in using the new Matrix.py (for linear
algebra style matrices) should take a good long look at Matrix.py. At
the moment it does almost nothing. I have no intentions to use this
class personally (a.matrixMultiply(b) works fine for me). If somebody
sends me a better version of this class I'd be happy to add it to the
distribution.
I know that people probably don't want to keep downloading and
installing a completely new version of the distribution each time some
simple bugs are fixed (the diff's between 0.30 and 0.32 are about 100
lines). Embarassingly enough, I don't know an easy way to take these
two tar files and produce a diff between them that will be appropriate
for the patch program to be applied to the in-place distribution.
I've included the file "diffs-0.30-0.32" that I made between untarred
directories created from the distribution, if somebody knows how to
take this file and update from 0.30 to 0.32, please share your knowledge.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From chris.chase@jhuapl.edu Mon Jan 29 17:20:22 1996
From: chris.chase@jhuapl.edu (Chris Chase S1A)
Date: Mon, 29 Jan 1996 12:20:22 -0500
Subject: [PYTHON MATRIX-SIG] Final pre-beta (0.30) of NumericalPython is available!
In-Reply-To: <9601262339.AA04290@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
References: <9601262339.AA04290@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <199601291714.MAA16713@python.org>
>>>>> "JH" == James Hugunin <jjh@Goldilocks.LCS.MIT.EDU> writes:
JH> Release 0.30 of the numerical extension to python is available.
I got it and I am checking it out. Jim, you announced with the 2.0
release that you would be adding the slice notation patches, i.e. in
the 2.0 release post:
JH> Things I plan to fix ASAP
JH> 1) Integrate Chris Chase's grammar patches to replace
JH> Slice(None,None,2) with ::2.
JH> 2) Make matrixMultiply work on matrices larger than 2d (this
JH> requires a notion of rank).
The ":" patch is missing.
I haven't checked yet for extension of matrix multiplication to larger
dimensional arrays.
Are these to be added?
Chris
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Mon Jan 29 17:34:47 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Mon, 29 Jan 96 12:34:47 EST
Subject: [PYTHON MATRIX-SIG] Slice notation and function ranks
In-Reply-To: <9601291713.AA25467@goldilocks> (message from Chris Chase S1A on Mon, 29 Jan 1996 12:20:22 -0500)
Message-ID: <9601291734.AA18476@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: Chris Chase S1A <chris.chase@jhuapl.edu>
JH> Things I plan to fix ASAP
JH> 1) Integrate Chris Chase's grammar patches to replace
JH> Slice(None,None,2) with ::2.
JH> 2) Make matrixMultiply work on matrices larger than 2d (this
JH> requires a notion of rank).
The ":" patch is missing.
I haven't checked yet for extension of matrix multiplication to larger
dimensional arrays.
Are these to be added?
I'd still like to do these ASAP, but that "as possible" part is really
getting in my way.
I'm eager to produce a stable, robust, useful version of the array
object that Guido will be happy to include as part of the standard
distribution as quickly as possible. I think that it would be a very
GOOD THING to have these array objects as a standard module in
Python-1.4. In the interests of moving this along, I'm unfortunately
neglecting two things that I'd also really like to see done.
1) Better multidimensional indexing (a[:-1, ::2] as well as a[1, .., 2])
Why not? I don't feel like I have the energy to do the necessary
lobbying to convince the python community/Guido that these changes to
the python core are needed. I'm using Konrad's patches because he
packaged them up nicely and negociated with Guido for their inclusion
in the next release of the language core. If somebody (Chris?) can
produce a similar set of patches for slices (okay you've done this
part) AND get Guido to agree to add them to the python core, then I'd
love to incorporate them with the NumericPython package.
2) A solid notion of rank for both bounded and unbounded rank
functions
Why not? Personally I have no pressing need for this capability (my
own needs for ranks are satisfied by pseudo-indices), and I figure it
would probably take me two-weeks of serious work to do this right and
make sure it's bug free.
Also, this particular feature is an extension to the basic array
system, so it can always be added later without causing any
incompatible changes (I took reasonable care designing the ofuncobject
with such an extension in mind). This makes me feel more comfortable with
waiting on it until after a solid release 1.0 is out.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dredish@CS.cmu.edu Mon Jan 29 18:00:13 1996
From: dredish@CS.cmu.edu (David Redish)
Date: Mon, 29 Jan 1996 13:00:13 -0500
Subject: [PYTHON MATRIX-SIG] Documentation
Message-ID: <19160.822938413@GS151.SP.CS.CMU.EDU>
Is there some documentation available on the Matrix implementation?
Although it seems fast and complete, I've had trouble implementing
even relatively basic operations in v0.2 (such as dot product). In
v0.3 there is a file TUTORIAL.NumPy but it is rather meager (to say
the least).
Thanks
David Redish Computer Science Department CMU
graduate student Neural Processes in Cognition Training Program
Center for the Neural Basis of Cognition (CNBC)
http://www.cs.cmu.edu/Web/People/dredish/home.html
------------------------------------------------------------
maintainer, CNBC website:
http://www.cs.cmu.edu/Web/Groups/CNBC
maintainer, Cog-Neuro Sites on the Internet, courtesy CNBC:
http://www.cs.cmu.edu/Web/Groups/CNBC/other
maintainer, NIPS*96 website:
http://www.cs.cmu.edu/Web/Groups/NIPS
------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Mon Jan 29 18:20:43 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Mon, 29 Jan 96 13:20:43 EST
Subject: [PYTHON MATRIX-SIG] Documentation
In-Reply-To: <19160.822938413@GS151.SP.CS.CMU.EDU> (message from David Redish on Mon, 29 Jan 1996 13:00:13 -0500)
Message-ID: <9601291820.AA18632@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Short answer: Not yet.
If you are confused about how to do something with the array object,
this is the perfect place to ask questions. I'll try and collect the
answers to these questions into the Tutorial (and then when the BETA
release comes out those folks will have a real tutorial).
There are two ways to take the dot product of two vectors a and b.
The first is close to the standard form for writing a dot product:
> add.reduce(a*b)
add.reduce(m) is equivalent to the python form reduce(add, m), but it
will run several orders of magnitude faster for large arrays.
The module Numeric.py also defines the function sum, which is
equivalent to add.reduce, so this could also be written as:
> sum(a*b)
The second approach is to realize that matrixMultiply when applied to
two vectors is equivalent to taking their dot product, so the
following will also work (and will run the fastest):
a.matrixMultiply(b)
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From chris.chase@jhuapl.edu Mon Jan 29 19:02:42 1996
From: chris.chase@jhuapl.edu (Chris Chase S1A)
Date: Mon, 29 Jan 1996 14:02:42 -0500
Subject: [PYTHON MATRIX-SIG] Problem compiling ofuncobject.c
In-Reply-To: <199601291553.KAA09076@cyclone.ERE.UMontreal.CA>
References: <199601291443.IAA02517@darwin.rsoc.rockwell.com>
<199601291553.KAA09076@cyclone.ERE.UMontreal.CA>
Message-ID: <199601291857.NAA17704@python.org>
While compiling the 0.32 release of the NumericPython I ran into an
error.
Compiling on SGI with IRIX 5.2 and gcc 2.6.0.
I received the following output:
gcc -O -I./../Include -I.. -DHAVE_CONFIG_H -c ./ofuncobject.c
./ofuncobject.c: In function `PyOFunc_GenericReduction':
./ofuncobject.c:490: wrong type argument to unary exclamation mark
./ofuncobject.c: In function `PyOFunc_GenericReduceAt':
./ofuncobject.c:590: wrong type argument to unary exclamation mark
*** Error code 1 (bu21)
In ofuncobject.c, lines 490 and 590 have the following:
if (self->check_return) TRY(check_array(ret));
With TRY defined as:
#define TRY(E) if(! (E)) return NULL
The problem is check_array() has return type void.
I took out the TRY() on those two lines as no action is taken on the
result.
The rest of the compile was successful.
Chris
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dubois1@llnl.gov Mon Jan 29 19:00:22 1996
From: dubois1@llnl.gov (Paul. Dubois)
Date: Mon, 29 Jan 1996 11:00:22 -0800
Subject: [PYTHON MATRIX-SIG] Documentation
References: <9601291820.AA18632@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <310D1946.7FF5@llnl.gov>
I have a deadline of Feb. 15 for my article for Computers in Physics in
which I will give the basic documentation for the numerical extension.
It will exist, trust me. I've been a little late getting it started
because of my work connecting PDB to Python, which is a bottleneck here
for our local work and I just had to push it.
In recent developments, we used Python/Numeric combined with an
extension that can read PDB files and one that knows how to talk to a
fancy graphics package we got from our sister laboratory in France, and
were able to make 3d pictures of the data in the files. (They haven't
made a general release of this package so I won't bother to describe
it).
--
Paul F. Dubois, L-472 (510)-422-5426
Lawrence Livermore National Laboratory FAX (510)-423-9969
Livermore, CA 94550 dubois1@llnl.gov
Consulting: PaulDubois@aol.com
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dredish@CS.cmu.edu Mon Jan 29 19:06:51 1996
From: dredish@CS.cmu.edu (David Redish)
Date: Mon, 29 Jan 1996 14:06:51 -0500
Subject: [PYTHON MATRIX-SIG] Compiling on HPUX with gcc
Message-ID: <19315.822942411@GS151.SP.CS.CMU.EDU>
When trying to compile in HPUX with gcc, I get errors on lines 490 and
590 of ofuncobject.c. The problem is that check_array is of type
void, but TRY() tries to test whether it's 0 or not. By changing from
if (self->check_return) TRY(check_array(ret));
to
if (self->check_return) check_array(ret);
I can get it to compile.
PS. I see someone just posted this in reference to the SGI.
However, I get a ld error when I try to load it saying unsatisfied
symbol: "finite".
Am I missing a file?
I'm using NumericPython-0.32.tar.gz. (I also untarred patches.tar.)
Thanks
David Redish Computer Science Department CMU
graduate student Neural Processes in Cognition Training Program
Center for the Neural Basis of Cognition (CNBC)
http://www.cs.cmu.edu/Web/People/dredish/home.html
------------------------------------------------------------
maintainer, CNBC website:
http://www.cs.cmu.edu/Web/Groups/CNBC
maintainer, Cog-Neuro Sites on the Internet, courtesy CNBC:
http://www.cs.cmu.edu/Web/Groups/CNBC/other
maintainer, NIPS*96 website:
http://www.cs.cmu.edu/Web/Groups/NIPS
------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Mon Jan 29 19:07:37 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Mon, 29 Jan 96 14:07:37 EST
Subject: [PYTHON MATRIX-SIG] Another bug in release 0.32
In-Reply-To: <9601291856.AA26104@goldilocks> (message from Chris Chase S1A on Mon, 29 Jan 1996 14:02:42 -0500)
Message-ID: <9601291907.AA18816@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
I should have known better than to mess around with ofuncobject.c in
the middle of getting out a new release...
For those who haven't been bitten yet, the following patch needs to be
applied to Modules/ofuncobject.c
Basically, on lines 490, and 590, you should remove the TRY from
around the check_array function call. I have no idea why this didn't
show up as an error under gcc.
*** 487,493 ****
/* Cleanup the returned matrices so that scalars will be returned as python scalars */
! if (self->check_return) check_array(ret);
return PyArray_Return(ret);
}
--- 487,493 ----
/* Cleanup the returned matrices so that scalars will be returned as python scalars */
! if (self->check_return) TRY(check_array(ret));
return PyArray_Return(ret);
}
***************
*** 587,593 ****
Py_DECREF(indices);
/* Cleanup the returned matrices so that scalars will be returned as python scalars */
! if (self->check_return) check_array(ret);
return PyArray_Return(ret);
}
--- 587,593 ----
Py_DECREF(indices);
/* Cleanup the returned matrices so that scalars will be returned as python scalars */
! if (self->check_return) TRY(check_array(ret));
return PyArray_Return(ret);
}
***************
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Mon Jan 29 19:22:36 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Mon, 29 Jan 96 14:22:36 EST
Subject: [PYTHON MATRIX-SIG] Compiling on HPUX with gcc
In-Reply-To: <19315.822942411@GS151.SP.CS.CMU.EDU> (message from David Redish on Mon, 29 Jan 1996 14:06:51 -0500)
Message-ID: <9601291922.AA18868@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: David Redish <David_Redish@GS151.SP.CS.CMU.EDU>
<SNIP> stuff about embarassing error covered elsewhere <SNIP>
However, I get a ld error when I try to load it saying unsatisfied
symbol: "finite".
Am I missing a file?
You're not missing a file, you're running into one of the unfortunate
system dependent issues in IEEE libraries.
On most systems, the standard library includes the function "finite"
which returns true iff a floating point value is not +-Inf, and not
NaN.
This function is only referenced in ofuncobject.c in the macro CHECK
which is defined at the top of the file.
A quick fix is to just define CHECK(x) as nothing. This will mean
that the umathmodule will not check for floating point overflows in
calculations as it should.
A better solution would be for you to figure out how to get this
desired behavior on HPUX and send me the patches.
A good check to see if you have a working implementation is to do the
following:
>>> import umath
>>> umath.multiply(1.0e200, 1.0e200)
This should produce OverflowError
If instead it returns Infinity, you know that your routine is not
doing the needed checks.
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Mon Jan 29 19:49:21 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Mon, 29 Jan 1996 14:49:21 -0500
Subject: [PYTHON MATRIX-SIG] Bug in matrixMultiply, and once more names
Message-ID: <199601291949.OAA01699@cyclone.ERE.UMontreal.CA>
Here's a bug (or two) that I just found:
>>> from Numeric import *
>>> x = array([1,2,3.])
>>> add.reduce(x*x)
14.0
>>> x.matrixMultiply(x)
array(14.0, 'd')
>>> 2.*x.matrixMultiply(x)
Segmentation fault
The segmentation fault is the obvious bug, but I am also surprised
that x.matrixMultiply(x) returns an array of rank 0 instead of a
scalar.
This raises the question of how rank-0 arrays are treated in
general. Although it may seem reasonable to convert them to scalars,
it also makes sense to keep them as arrays if they have one of the new
types (e.g. C float).
About names:
>From the recent discussion I remember that there are supposed to be
two array types: a low-level type "array" and a high-level type
"Array". The latter may have become unnecessary now that "array"s have
automatic coercion again. But should the remaining one be called
"array" or "Array"? I don't really care, but certainly we should have
either "array" and "matrix" or "Array" and "Matrix", not "array" and
"Matrix" as it is now.
Also, I have some more objections against the name of the module
UserArray. I don't like the "UserXxx" names at all (because they don't
relate to the user any more than any other module), but that's not my
point now. UserList is a Python class that is equivalent to the
built-in list type, UserDict similarly wraps dictionaries. So one
should conclude that UserArray is an analogous wrapping of the old
array type, but that is not true. The new arrays have different
functionality, so they should have another name. One possibility would
be "NumericArray", since the main new feature is numerical
operations. On the other hand, there are no numerical operations on
e.g. character arrays. Maybe someone else finds a better name.
And now back to testing...
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Mon Jan 29 20:08:35 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Mon, 29 Jan 96 15:08:35 EST
Subject: [PYTHON MATRIX-SIG] Re: Bug in matrixMultiply, and once more names
In-Reply-To: <199601291949.OAA01699@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA)
Message-ID: <9601292008.AA19173@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad)
Here's a bug (or two) that I just found:
>>> from Numeric import *
>>> x = array([1,2,3.])
>>> add.reduce(x*x)
14.0
I assume this isn't a bug.
>>> x.matrixMultiply(x)
array(14.0, 'd')
This is a bug all right. The standing agreement is that all array
functions will return a python scalar instead of a 0d array. Perhaps
you're right and this should only happen for returned arrays of type
long, double and double complex. It's all in one function so this
would be an easy change, the final decision on the right thing to do
here depends critically on the decisions made regarding type
coercions. I just forgot to call this function for matrixMultiply.
>>> 2.*x.matrixMultiply(x)
Segmentation fault
I'll fix this too. (once I figure it out)
About names:
>From the recent discussion I remember that there are supposed to be
two array types: a low-level type "array" and a high-level type
"Array". The latter may have become unnecessary now that "array"s have
automatic coercion again. But should the remaining one be called
"array" or "Array"? I don't really care, but certainly we should have
either "array" and "matrix" or "Array" and "Matrix", not "array" and
"Matrix" as it is now.
I actually rather like the way that things stand now. Here's what we
have:
A low level type named array that is what I expect 90% of the people
who use this system to use. This is entirely implemented in C.
A wrapper class called UserArray that provides a wrapper around this
new built-in array type. The theory here is that so long as we don't
break any old code, we are free to ignore the old array type as
obsolete.
A subclass of UserArray called Matrix that implements matrixMultiply
instead of array multiply for the "*" operator. This is mainly
provided for people who do entirely linear algebra work. Personally I
think that using a.matrixMultiply(b) is a better solution for people
who want to work with both arrays and matrices.
Also, I have some more objections against the name of the module
UserArray. I don't like the "UserXxx" names at all (because they don't
relate to the user any more than any other module), but that's not my
point now. UserList is a Python class that is equivalent to the
built-in list type, UserDict similarly wraps dictionaries. So one
should conclude that UserArray is an analogous wrapping of the old
array type, but that is not true. The new arrays have different
functionality, so they should have another name. One possibility would
be "NumericArray", since the main new feature is numerical
operations. On the other hand, there are no numerical operations on
e.g. character arrays. Maybe someone else finds a better name.
To reiterate, UserArray is a wrapper class around the new C level
type of array. There is a little bit of baggage in this wrapper
involved in the fact that arrays are a fairly complicated class. In
fact, I was able to substitute UserArrays into Benchmark.py with
about 10 lines of code as a replacement for arrays, I used this for
testing the speed of the wrapper class.
And now back to testing...
Keep those bugs coming!
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From et@appl-math.tu-muenchen.de Mon Jan 29 23:30:20 1996
From: et@appl-math.tu-muenchen.de (Thomas Schwaller)
Date: Tue, 30 Jan 1996 00:30:20 +0100
Subject: [PYTHON MATRIX-SIG] NumericPython-0.32.tar.gz (Europe) + PDE Proposal
Message-ID: <9601300030.ZM28266@hamster.appl-math.tu-muenchen.de>
Yeah. I'm still there...
For people in Europe:
You can fetch the matrixmodule at
ftp://ftp.appl-math.tu-muenchen.de/pub/et/NumericPython-0.32.tar.gz
I was quite happy not to get it more than once (takes really very long).
By the way. Is FDL (or FIDL ?) working with the latest matrixmodule.
(This is rally a question for Jim Fulton ! ;-))
Or asked the other way: is there some work done binding the usual FORTRAN
Libraries (*PACK) to this extension? (To get the state of Matlab, SciLab,
Octave).
Well, let's look one step further. After several experiences with integrating
C++ classes in Python, I'm not that enthousiastic anymore about this approach.
One year earlier I would have tried to do it with Diffpack (C++ classes for
the solution of PDE, which ist one of the best public packages for that
purpose if you like C++). But some native Python Module for that purpose
would be much more efficient, now that we have a common base for matrix
extensions. What about a package for the abstract specification of PDE's?
1) High level commands for building stiffness matrices out of the week form
of a PDE. This is for expert users. I saw such an abstract matrix
language (I don't remember the name....) at the International
Congress of Applied Mathematics.
2) Linear Algebra stuff related to that (Multigrid, Mutlilevel, domain
decomposition)
3) For non expert users the possibility to solve nonlinear systems of PDE's
by combining predefinied equations of type 1) (Iteration, Linearization
and the like)
4) Using the C++ Visual Toolkit as a starting point for the visualisation
of the results (OpenGL). I had no time to rework my Python Binding for it,
which is very necessary because it has some bugs. Loading the C++ module
is very slow so I would prefer to have a native Python/C module using the
C++ Code (where is code reusing gone ??? :-( ) Unfortunately a lot of work
in this direction will be done for Java by SGI (read their announcments
http://www.sgi.com/Products/cosmo).
Some comments?
Anybody written some ODE stuff yet. I took a look at the matlab ODE suite
recently. What a hack... When we present such an example with the official
release of the matrix module, people will like it I think.
I'll polish some Delaunay-Triangulation modules for public release as soon as I
can. They work nice with the matrixmodule, but I have to pythonize the error
handling of the C-Code it uses. For me it's ok, but you will certainly agree
with me, that it's not nice to see the Python interpreter crashing with wrong
geometrical data.
Back to first tests with the new matrix version....
Tom
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Tue Jan 30 00:19:25 1996
From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey)
Date: Mon, 29 Jan 1996 19:19:25 -0500
Subject: [PYTHON MATRIX-SIG] NumericPython-0.32.tar.gz (Europe) + PDE Proposal
In-Reply-To: "Thomas Schwaller" <et@appl-math.tu-muenchen.de>
"[PYTHON MATRIX-SIG] NumericPython-0.32.tar.gz (Europe) + PDE Proposal" (Jan 30, 12:30am)
References: <9601300030.ZM28266@hamster.appl-math.tu-muenchen.de>
Message-ID: <9601291919.ZM3155@dsdbqvarsa.er.usgs.gov>
On Jan 30, 12:30am, Thomas Schwaller wrote:
> Subject: [PYTHON MATRIX-SIG] NumericPython-0.32.tar.gz (Europe) + PDE Prop
> Yeah. I'm still there...
> For people in Europe:
> You can fetch the matrixmodule at
> ftp://ftp.appl-math.tu-muenchen.de/pub/et/NumericPython-0.32.tar.gz
>
> I was quite happy not to get it more than once (takes really very long).
>
> By the way. Is FDL (or FIDL ?) working with the latest matrixmodule.
> (This is rally a question for Jim Fulton ! ;-))
Not yet. I've been waiting for the matrix implementation to settle down.
I still have to jump through some hoops before I can release FIDL.
> Or asked the other way: is there some work done binding the usual FORTRAN
> Libraries (*PACK) to this extension? (To get the state of Matlab, SciLab,
> Octave).
>
> Well, let's look one step further. After several experiences with integrating
> C++ classes in Python, I'm not that enthousiastic anymore about this
approach.
> One year earlier I would have tried to do it with Diffpack (C++ classes for
> the solution of PDE, which ist one of the best public packages for that
> purpose if you like C++). But some native Python Module for that purpose
> would be much more efficient, now that we have a common base for matrix
> extensions.
My a "native Python module", do you mean a module written in Python, using the
matrix extension? Or do you mean a custom built extension?
In what way would this approach be more efficient?
A tool like FIDL allows you to reuse existing libraries with minimal effort. A
FIDL spec for a routine typically only requires a line or two, not counting
comments.
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From chris.chase@jhuapl.edu Tue Jan 30 00:57:54 1996
From: chris.chase@jhuapl.edu (Chris Chase S1A)
Date: Mon, 29 Jan 1996 19:57:54 -0500
Subject: [PYTHON MATRIX-SIG] Compiling on HPUX with gcc
In-Reply-To: <9601291922.AA18868@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
References: <19315.822942411@GS151.SP.CS.CMU.EDU>
<9601291922.AA18868@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <199601300052.TAA20689@python.org>
From: David Redish <David_Redish@GS151.SP.CS.CMU.EDU>
<SNIP> stuff about embarassing error covered elsewhere <SNIP>
However, I get a ld error when I try to load it saying unsatisfied
symbol: "finite".
Am I missing a file?
HPUX series 700/800 at Release 9 contain two versions of the math
library. The precision architecture (PA) libm version 1.1 contains
the IEEE functions. It is located in /lib/pa1.1. By default gcc
links against version 1.0 (in /lib) whereas HPUX cc links against
version 1.1.
If you have the pa1.1 libraries then use the --with-libm=-L/lib/pa1.1
option to configure to get the pa1.1 version of libm when using gcc.
Using this I was able to compile successfully. An aside:
/usr/lib/pa1.1 for the library path when using FORTRAN or PASCAL or
the vector library. Of possible future interest, the vector library
on the HP series 700 systems is very fast and contains a number of
operations that Numeric module could use for many of its basic
operations.
Another note: I have been unable to get Python to compile with the HP
C compiler with or without ANSI mode.
Chris
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From stoll@atr-sw.atr.co.jp Tue Jan 30 02:57:23 1996
From: stoll@atr-sw.atr.co.jp (Perry A. Stoll)
Date: Tue, 30 Jan 1996 11:57:23 +0900
Subject: [PYTHON MATRIX-SIG] A fix for mrange
Message-ID: <199601300257.LAA12363@ciris21.atr-sw.atr.co.jp>
>>> Numeric.mrange(1,5,1,'i')
Traceback (innermost last):
File "<stdin>", line 1, in ?
File "/home/stoll/Python/1.3/lib/python/numeric/Numeric.py", line 64, in mrange
return m*step + start
TypeError: cannot perform this operation on these types
>>> Numeric.mrange(1,5,0.5,'f')
Traceback (innermost last):
File "<stdin>", line 1, in ?
File "/home/stoll/Python/1.3/lib/python/numeric/Numeric.py", line 64, in mrange
return m*step + start
TypeError: cannot perform this operation on these types
>>> Numeric.mrange(1,5,1.0,'l')
array([1.0,2.0,3.0,4.0], 'd')
Shouldn't mrange honor the typecode?
How about this as a replacement for mrange:
# ???default typecode to long int to match builtin range()???
def mrange(start, stop=None, step=1, typecode='l'):
"""Just like range() except it returns an array whose
type can be specfied by the keyword argument typecode.
"""
if (stop == None):
stop = start
start = 0
n = int(math.ceil(float(stop-start)/step))
m = array(range(n), 'l')*step + start
if (m.typecode() != typecode):
m = m.cast(typecode)
return m
Now I get this:
>>> Numeric.mrange(1,5,1,'i')
array([1,2,3,4], 'i')
>>> Numeric.mrange(1,5,1.0,'l')
array([1,2,3,4], 'l')
>>> Numeric.mrange(200,300,10,'b')
array([200,210,220,230,240,250,4,14,24,34], 'b')
>>> Numeric.mrange(1,5,0.25,'l')
array([1,1,1,1,2,2,2,2,3,3,3,3,4,4,4,4], 'l')
The last is a little weird, but mrange always honors the typecode,
which I think is very important. When I specify a type, I expect an
array of that type.
To coerce or not to coerce, that is the question...
-Perry
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Jan 30 13:39:04 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Tue, 30 Jan 1996 08:39:04 -0500
Subject: [PYTHON MATRIX-SIG] First experiences
Message-ID: <199601301339.IAA22977@cyclone.ERE.UMontreal.CA>
Yesterday I replaced a rather awkward combination of a shell script
and a Matlab program by a Python program that is shorter, easier to
read, and faster than the old solution. First the most important and
good news: it works, and it produces the same results as the Matlab
code.
Of course I also noticed a few problems/strange features/possible
improvements, and here they are for general discussion:
"mrange" should be called "arange" now. Or maybe something clearer,
e.g. "rangeArray"?
Why add.reduce(...) but outer(add, ...)? I have typed add.outer(...)
more than once, and that makes more sense to me.
There is a function sum() that is almost equivalent to add.reduce() -
almost because it doesn't allow a second argument to specify the
axis. Why?
The more I use it (and see it on my screen), the pseudo-index None
seems strange to me. As a test, I have shown my code to a colleague
who is an active Matlab user. I told him what x[2,3] means and asked
him what he expects x[All, None] to mean. His immediate answer was:
"everything along the first axis, nothing along the second axis." Then
after a pause: "but that doesn't make sense."
So once I again I propose to change the name to "NewAxis" or "New" or
anything else indicating that this index *creates* a new dimension.
A related observation: I have used the combination [All, None] very
often, but no other index expression involving All or None. This is of
course a feature of my application, but I wouldn't be surprised if
appending a new axis were the most frequent use of these
pseudo-indices. In that case it would be worthwhile providing an
equivalent convenience function that is shorter (i.e. less visually
obtrusive).
Currently Numeric.py imports from fast_umath; in my opinion the
default should be umath. Those who want speed can always overwrite the
definition by importing from fast_umath.
A minor cosmetic point: when I print a ufunc, is still says 'ofunc'.
Another cosmetic point: "Array.error" should be "ArrayError".
I don't know how much effort this would be, but I'd really like
reshape() to accept new shapes that lead to a different size. From my
APL/J experience, I found that such reshaping is one of the most
useful methods to construct non-standard matrices. Some examples:
- an NxN array with all elements 37:
reshape(37, (N, N))
- a checkerboard matrix with 0s and 1s:
reshape([1,0],(8,9))[All, 0..7]
We could then eliminate special case constructors like zeros().
Another important function that seems to be missing is appending,
i.e. the equivalent of + for lists. Of course for arrays it has
to work along any axis.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From da@maigret.cog.brown.edu Tue Jan 30 15:30:50 1996
From: da@maigret.cog.brown.edu (David Ascher)
Date: Tue, 30 Jan 1996 10:30:50 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] None, All, etc.
In-Reply-To: <199601301339.IAA22977@cyclone.ERE.UMontreal.CA> from "Hinsen Konrad" at Jan 30, 96 08:39:04 am
Message-ID: <199601301530.KAA18459@maigret>
After just reading Hinsen's post, and remembering Jim H's, comments
about 'this is the perfect place to ask', I'd like to ask one question
which I'm sure many novices will have. A good answer to this one should
be in the Tutorial. [Stealing from APL/J docs is a possible way to go]
What are these pseudo-indices (None, All, etc.) and how do they work?
--david
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Tue Jan 30 16:12:04 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Tue, 30 Jan 96 11:12:04 EST
Subject: [PYTHON MATRIX-SIG] Re: A fix for mrange
In-Reply-To: <199601300257.LAA12363@ciris21.atr-sw.atr.co.jp> (stoll@atr-sw.atr.co.jp)
Message-ID: <9601301612.AA03369@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Shouldn't mrange honor the typecode?
Obviously, I'll switch over to using your new version. However, I
think that it's important to make a small change for determining the
default typecode. If any of the arguments is a python float, than the
range returned should be an array of doubles, not longs.
ie. arange(1, 5, 1.0) should be array([1.,2.,3.,4.], 'd')
I'll add some checks to the code to get this behavior in the case of
no explicit typecode given.
BTW - the typecode argument is really intended to be used as a keyword
argument, as in arange(1, 5, 1.0, typecode=Integer()). This keeps
things uniform for the case of arange(10, typecode=Integer()) which
wouldn't work without the keyword argument.
The last is a little weird, but mrange always honors the typecode,
which I think is very important. When I specify a type, I expect an
array of that type.
Agreed!
To coerce or not to coerce, that is the question...
Now that question needs to be addressed seperately.
Thanks for the fix - Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Tue Jan 30 16:35:08 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Tue, 30 Jan 96 11:35:08 EST
Subject: [PYTHON MATRIX-SIG] First experiences
In-Reply-To: <199601301339.IAA22977@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca)
Message-ID: <9601301635.AA03443@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Yesterday I replaced a rather awkward combination of a shell script
and a Matlab program by a Python program that is shorter, easier to
read, and faster than the old solution. First the most important and
good news: it works, and it produces the same results as the Matlab
code.
Great news! This is of course one of the main purposes for this code.
Of course I also noticed a few problems/strange features/possible
improvements, and here they are for general discussion:
"mrange" should be called "arange" now. Or maybe something clearer,
e.g. "rangeArray"?
Definately "arange" (or something clearer but equally short). I use
arange far to much to be willing to put up with a nice clear name like
rangeArray.
Why add.reduce(...) but outer(add, ...)? I have typed add.outer(...)
more than once, and that makes more sense to me.
Writing outer product in python code was trivial, writing it in C
would have taken some time, so I cut some corners while I was still
unsure about the permanent status of pseudo-indices in the system.
I'll remove outer from the standard functions, and add a new method to
the ofuncs (ufuncs now?).
There is a function sum() that is almost equivalent to add.reduce() -
almost because it doesn't allow a second argument to specify the
axis. Why?
I don't know whether or not sum even belongs in the set of standard
functions. On the one hand I'd rather make people use add.reduce
because it's much more in the spirit of the array object. On the
other hand, sum is such a friendly little function. If it remains in
the system, then I agree it should get the second argument.
The more I use it (and see it on my screen), the pseudo-index None
seems strange to me. As a test, I have shown my code to a colleague
who is an active Matlab user. I told him what x[2,3] means and asked
him what he expects x[All, None] to mean. His immediate answer was:
"everything along the first axis, nothing along the second axis." Then
after a pause: "but that doesn't make sense."
So once I again I propose to change the name to "NewAxis" or "New" or
anything else indicating that this index *creates* a new dimension.
I agree with you here. Clearly there is another naming issue to be
resolved. See my next post.
A related observation: I have used the combination [All, None] very
often, but no other index expression involving All or None. This is of
course a feature of my application, but I wouldn't be surprised if
appending a new axis were the most frequent use of these
pseudo-indices. In that case it would be worthwhile providing an
equivalent convenience function that is shorter (i.e. less visually
obtrusive).
Well, I find that All is used a lot of the time, but I agree with you
that None is almost always use to append a new axis. I'll address
this also in my next post.
Currently Numeric.py imports from fast_umath; in my opinion the
default should be umath. Those who want speed can always overwrite the
definition by importing from fast_umath.
And vice versa. This is mainly a matter of personal taste, and since
my taste favors fast_umath, that's going to be the default version for
now. If I find that many people on this sig disagree with me then
I'll change it.
A minor cosmetic point: when I print a ufunc, is still says 'ofunc'.
That's because ufunc's are still called ofunc's. This is another one
of those great renaming sorts of things, and it should be done.
Another cosmetic point: "Array.error" should be "ArrayError".
Easy to fix.
I don't know how much effort this would be, but I'd really like
reshape() to accept new shapes that lead to a different size. From my
APL/J experience, I found that such reshaping is one of the most
useful methods to construct non-standard matrices. Some examples:
- an NxN array with all elements 37:
reshape(37, (N, N))
- a checkerboard matrix with 0s and 1s:
reshape([1,0],(8,9))[All, 0..7]
We could then eliminate special case constructors like zeros().
What you suggest would be very easy to code, but I don't want it
included in the behavior for reshape. I like having reshape error
check my dimensions for me, and I like the fact that reshape doesn't
copy the whole array, but just returns a new pointer to it.
However, I have no objections to adding another method which does what
you're asking for. What would you like it to be called?
Another important function that seems to be missing is appending,
i.e. the equivalent of + for lists. Of course for arrays it has
to work along any axis.
Try this:
a = array([1,2,3])
a.concat([4,5], [7,8])
-> array([1,2,3,4,5,7,8])
I assume that this is what you're looking for?
It should probably work along any axis, but I don't have a terribly
good way to specify the axis and still keep the nice feature of being
able to concat together an arbitrary number of arrays (very important
for efficiency). It would also be a bit of a hassle to write in C the
way you're suggesting, would it be enough to add in a standard python
function with the desired behavior?
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Tue Jan 30 17:42:49 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Tue, 30 Jan 96 12:42:49 EST
Subject: [PYTHON MATRIX-SIG] Pseudo Indices
In-Reply-To: <199601301530.KAA18459@maigret> (da@maigret.cog.brown.edu)
Message-ID: <9601301742.AA03714@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Well, here's a fairly long and slightly confusing response, but what
do you expect from a first draft?
The idea of pseudo indices comes from the language Yorick created by
Dave Munro at LLNL, not from APL/J.
Everybody expects this to work:
a = array([1,2,3])
a+2 -> array([3,4,5])
2 is 0-dimensional array with shape [], and a has shape [3]. The rule
says that any non-existent dimension is "broadcast" to match a higher
dimensional array as required.
The idea is to extend this notion of broadcasting to any dimension of
length one. So the following will also work:
a+array([2]) -> array([3,4,5])
array([2]) has shape [1], and a has shape [3]. The [1] is broadcast
to match the [3].
The classic example of where this is useful is in taking an
outer product.
b = array([10, 20])
a * b.reshape(2, 1) -> array([[11, 12, 13], [21, 22, 23]])
a has shape [3], and b.reshape(2,1) has shape [2, 1]. The result of a binary
operator applied to them has shape [2,3].
In order to make such code easier to read, Yorick introduces the
notion of a pseudo index that can be added to any multidimensional index
and means, add a new dimension here.
ie. b[2, PseudoIndex] <--> b.reshape(2, 1)
"All" is not in fact related to Yorick pseudo indices at all, but is
related to Slices.
c = array([[1,2,3], [11,12,13]])
I'd really like to be able to say c[:, 3] -> array([3,13]). Python
syntax doesn't currently allow this, so until somebody convinces Guido
to make the change, the following objects are used instead.
c[:, 1:2] --> c[Slice(), Slice(1,2)]
All = Slice()
One more piece is needed to handle multidimensional indexing properly.
Let's say that I want to extract the third element from the last
dimension of an array.
In Yorick that would look like a[.., 3]. Since Python doesn't yet
have this rubber index, I use the following:
a[RubberIndex, 3].
So, the following special symbols are needed to properly treat
multidimensional indexing:
Slice
PseudoIndex
RubberIndex
My current feeling on the best way to do this is to keep Slice's as
they are (until somebody lobbys Guido for a better system).
Define string constants
NewAxis = "NewAxis"
Ellipses = "Ellipses"
NewAxis is equivalent to the current "None" and indicates a
PseudoIndex
Ellipses is equivalent to the current "RubberIndex" and indicates
standard ellipses as in, fill in what should naturally go here.
Comments?
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Tue Jan 30 17:54:22 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Tue, 30 Jan 96 12:54:22 EST
Subject: [PYTHON MATRIX-SIG] First experiences
In-Reply-To: <199601301726.MAA16169@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA)
Message-ID: <9601301754.AA03750@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad)
I don't know whether or not sum even belongs in the set of standard
functions. On the one hand I'd rather make people use add.reduce
because it's much more in the spirit of the array object. On the
other hand, sum is such a friendly little function. If it remains in
the system, then I agree it should get the second argument.
In the spirit of "no needless redundancies", it would be better not to
have this function. Maybe we should add an (optional) module with
standard abbreviations, where we would simply write sum =
add.reduce. That would also be a good way to have clearer standard
names for things like arange() while still having short alternatives
that are available on every system.
I think I like this idea, I'll play with it and see what I come up with.
What you suggest would be very easy to code, but I don't want it
included in the behavior for reshape. I like having reshape error
check my dimensions for me, and I like the fact that reshape doesn't
copy the whole array, but just returns a new pointer to it.
It could still return a new pointer if the size is the same,
and make a copy only if the size changes. Personally I don't care
This I REALLY don't want. I firmly believe that whether a function
returns a pointer or a copy should not be dependent on its arguments.
about dimension checking; in all my APL code that has never been
a problem. If the size remains the same, then typically the
new dimensions are specified as an expression involving the old
dimensions. I have never had an error there.
Try this:
a = array([1,2,3])
a.concat([4,5], [7,8])
-> array([1,2,3,4,5,7,8])
I assume that this is what you're looking for?
Not really. First of all, in most cases I need a non-destructive
version, similar to sequence addition. Second, I really need
it for all dimensions. In my application, I have two 1d arrays
of equal length and have to construct a 2d array from them.
What I used is
c = array([tuple(a), tuple(b)])
but there ought to be a much cleaner and more efficient way.
Well, you should use c = array([a, b]) for that. This seems fairly
clean to me, and will run quite efficiently if a and b are already
arrays of the same type.
What do you mean by non-destructive? a.concat(b) does not modify a,
but instead returns a new array that is b concatenated with a. This
is almost exactly equivalent to a+b if a and b were lists.
BTW, we also ought to have a way to convert arrays to nested
lists and maybe tuples. Oddly enough, Python has no built-in
function list(), although it does have tuple().
I agree, I'll add this when I get the time. I'll probablyu call it
toList() to go along with toString and toFile.
It should probably work along any axis, but I don't have a terribly
good way to specify the axis and still keep the nice feature of being
able to concat together an arbitrary number of arrays (very important
One argument could be a tuple of all arrays to be concatenated.
This doesn't work nicely if you only have one array.
for efficiency). It would also be a bit of a hassle to write in C the
way you're suggesting, would it be enough to add in a standard python
function with the desired behavior?
That's just a matter of efficiency and has to be tried. In my APL code
this function is used a lot, and often on small arrays.
Again, what would you call this new method?
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dubois1@llnl.gov Tue Jan 30 18:05:22 1996
From: dubois1@llnl.gov (Paul. Dubois)
Date: Tue, 30 Jan 1996 10:05:22 -0800
Subject: [PYTHON MATRIX-SIG] Uniform random number generator v. 2.0
Message-ID: <310E5DE2.5E40@llnl.gov>
On ftp-icf.llnl.gov, directory /pub/basis, lies:
URNG-2.0.tar.gz
which contains a C extension and Python wrapper for a random number
generator equivalent to the one on Cray systems. This version works with
the Numeric extension 0.32. (It contains a feature for returning a
random sample or randomly-filled array of arbitrary shape, as well as a
more pedestrian "ranf()".)
URNG can create separate generators that act independently for use in
different modules.
Note to Jim Hugunin: I can't remember how I "delivered" this to your ftp
directory before...
--
Paul F. Dubois, L-472 (510)-422-5426
Lawrence Livermore National Laboratory FAX (510)-423-9969
Livermore, CA 94550 dubois1@llnl.gov
Consulting: PaulDubois@aol.com
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From guido@CNRI.Reston.VA.US Tue Jan 30 18:09:42 1996
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Tue, 30 Jan 1996 13:09:42 -0500
Subject: [PYTHON MATRIX-SIG] Pseudo Indices
In-Reply-To: Your message of "Tue, 30 Jan 1996 12:42:49 EST."
<9601301742.AA03714@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
References: <9601301742.AA03714@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <199601301809.NAA08602@monty>
> ie. b[2, PseudoIndex] <--> b.reshape(2, 1)
I'm not sure I understand reshape completely, but isn't this a rather
drastic reinterpretation of the index arguments when a pseudo index is
present? In general, if no pseudo index is present, single indices
take away that dimension, don't they? (While slice indices can reduce
the length in that dimension but otherwise leave it intact.) But in
your example, the '2' becomes a shorthand for a slice. I don't like
this very much...
> I'd really like to be able to say c[:, 3] -> array([3,13]). Python
> syntax doesn't currently allow this, so until somebody convinces Guido
> to make the change, the following objects are used instead.
Same comment -- I don't like the '3' index to turn into a slice by
context, if I understand this right. (Bear with me, I'm not even sure
that array([3, 13]) really is a 3x13 matrix.)
> c[:, 1:2] --> c[Slice(), Slice(1,2)]
This is fine (and what I thought you were proposing all the time). I
would consider a patch if the existing 1-dim sequence and mapping
types are not affected.
> a[RubberIndex, 3].
You could sneak this into the abovementioned patch as "a[::, 3]".
> (until somebody lobbys Guido for a better system).
Ain't I nice today? ;-)
--Guido van Rossum <guido@CNRI.Reston.VA.US>
URL: <http://www.python.org/~guido/>
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Tue Jan 30 18:13:06 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Tue, 30 Jan 96 13:13:06 EST
Subject: [PYTHON MATRIX-SIG] Uniform random number generator v. 2.0
In-Reply-To: <310E5DE2.5E40@llnl.gov> (dubois1@llnl.gov)
Message-ID: <9601301813.AA03881@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Note to Jim Hugunin: I can't remember how I "delivered" this to your ftp
directory before...
Our sysadmins plugged a security hole that had made it so convenient
for people to upload code like this to my ftp directory in the past.
Since I still like the idea of keeping things like this in one place,
I'll take it on myself to download code like this to
sls-ftp.lcs.mit.edu into directory /pub/jjh.
This is already done for Paul's URNG package.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Tue Jan 30 18:34:54 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Tue, 30 Jan 96 13:34:54 EST
Subject: [PYTHON MATRIX-SIG] Pseudo Indices
In-Reply-To: <199601301809.NAA08602@monty> (message from Guido van Rossum on Tue, 30 Jan 1996 13:09:42 -0500)
Message-ID: <9601301834.AA03955@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: Guido van Rossum <guido@CNRI.Reston.VA.US>
> ie. b[2, PseudoIndex] <--> b.reshape(2, 1)
I'm not sure I understand reshape completely, but isn't this a rather
drastic reinterpretation of the index arguments when a pseudo index is
present? In general, if no pseudo index is present, single indices
take away that dimension, don't they? (While slice indices can reduce
the length in that dimension but otherwise leave it intact.) But in
your example, the '2' becomes a shorthand for a slice. I don't like
this very much...
I'm sorry, that was a type-o on my part. What I should have said was
b[All, PseudoIndex] <--> b.reshape(2,1).
> I'd really like to be able to say c[:, 3] -> array([3,13]). Python
> syntax doesn't currently allow this, so until somebody convinces Guido
> to make the change, the following objects are used instead.
Same comment -- I don't like the '3' index to turn into a slice by
context, if I understand this right. (Bear with me, I'm not even sure
that array([3, 13]) really is a 3x13 matrix.)
Here you are simply not understanding my (admittedly poor)
description. array([3,13]) is a one dimensional array with a shape of
[2] which contains the numeric elements 3 and 13. So the 3 index is
not returning a slice by context. The returned array has a dimension
one less that the initial array because it has the index 3 instead of
a slice.
> c[:, 1:2] --> c[Slice(), Slice(1,2)]
This is fine (and what I thought you were proposing all the time). I
would consider a patch if the existing 1-dim sequence and mapping
types are not affected.
Great! I'll get on with the process of coordinating this patch.
Currently the best way we've come up to implement this patch is to add
a new object type to python called a sliceobject. c[:, 1:2] then gets
converted to c[ (slice(), slice(1,2)) ] in the interpreter and then
passed to the standard getitem mapping method. Special consideration
is payed to c[:] so as not to break existing code.
I'm a little bit troubled by the need to add a new object (admittedly
extremely small) just to get this behavior. Does this solution
seem reasonable to you?
The only other approach I can think of is to add a new protocol for
multidimensional arrays, on equal footing with number, mapping, and
sequence. This seems like it would be overkill.
> a[RubberIndex, 3].
You could sneak this into the abovementioned patch as "a[::, 3]".
Slices have an additional feature which is a third index specifying
a stride. ie. Slice(None,None,2) will take all the even elements
along a dimension. Slice(None,None,-1) will reverse a dimension.
This arrangement corresponds very nicely with the internals of array
objects.
Unfortunately, this means that things like ::-1 will be reasonably
common, and so using :: as a RubberIndex might be confusing.
However, I do like this syntax for the RubberIndex as it really
captures what it does which is to apply : to every unused dimension.
Maybe there's a similar proposal that will work.
> (until somebody lobbys Guido for a better system).
Ain't I nice today? ;-)
Gee, now that we've got Guido in such a friendly mood, what else
should we ask for? A new operator just for matrix multiplication? ;)
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Jan 30 19:21:11 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Tue, 30 Jan 1996 14:21:11 -0500
Subject: [PYTHON MATRIX-SIG] Re: Pseudo Indices
In-Reply-To: <9601301742.AA03714@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199601301921.OAA23872@cyclone.ERE.UMontreal.CA>
In order to make such code easier to read, Yorick introduces the
notion of a pseudo index that can be added to any multidimensional index
and means, add a new dimension here.
ie. b[2, PseudoIndex] <--> b.reshape(2, 1)
Not quite. "2" means "the third element".
"All" is not in fact related to Yorick pseudo indices at all, but is
related to Slices.
But "All" is the equivalent of the Yorick pseudoindex *, which of
course is also a special form of a slice.
c = array([[1,2,3], [11,12,13]])
I'd really like to be able to say c[:, 3] -> array([3,13]). Python
Small correction: c[:, 2].
In Yorick that would look like a[.., 3]. Since Python doesn't yet
have this rubber index, I use the following:
a[RubberIndex, 3].
That should also get a better name... I think I already made
some proposals, I just have to dig that message out again...
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Jan 30 19:33:41 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Tue, 30 Jan 1996 14:33:41 -0500
Subject: [PYTHON MATRIX-SIG] First experiences
In-Reply-To: <9601301754.AA03750@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199601301933.OAA24990@cyclone.ERE.UMontreal.CA>
It could still return a new pointer if the size is the same,
and make a copy only if the size changes. Personally I don't care
This I REALLY don't want. I firmly believe that whether a function
returns a pointer or a copy should not be dependent on its arguments.
You are right. I had forgotten that returning a pointer or returning
a copy makes a real difference here; I am probably too much infected
by J, in which arrays are immutable objects. And the more I think
about it, the more I like immutable arrays! But that's another point.
Well, you should use c = array([a, b]) for that. This seems fairly
clean to me, and will run quite efficiently if a and b are already
arrays of the same type.
I didn't know this was possible (of course I should have tried...).
But still that is a solution only if I want to create a new first
axis. What if I don't want a new axis, or if the new axis is not
the first one?
What do you mean by non-destructive? a.concat(b) does not modify a,
but instead returns a new array that is b concatenated with a. This
is almost exactly equivalent to a+b if a and b were lists.
OK, I misunderstood this.
One argument could be a tuple of all arrays to be concatenated.
This doesn't work nicely if you only have one array.
Not as long as it is a method. But concat() could be made a normal
function. Actually, that is another slightly unclear point in the
current array module. Some operations are functions (e.g. reshape()),
others are methods (like concat()), but that seems to be completely
arbitrary.
That's just a matter of efficiency and has to be tried. In my APL code
this function is used a lot, and often on small arrays.
Again, what would you call this new method?
Ehh... Which one? Concatenation? Well, I still hope to find
a solution that incorporates all applications into one, which
can then be called concat(). We should try to keep the number
of functions/methods to a minimum. So how about this:
function concat((a,b,c...), n=0)
concatenates arrays a,b,c,... along the nth axis and returns
the result. The shape along the other axes must agree.
This doesn't take care of new axes, but they can be added
easily with the pseudo index.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Tue Jan 30 19:50:41 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Tue, 30 Jan 96 14:50:41 EST
Subject: [PYTHON MATRIX-SIG] First experiences
In-Reply-To: <199601301933.OAA24990@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA)
Message-ID: <9601301950.AA04262@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad)
You are right. I had forgotten that returning a pointer or returning
a copy makes a real difference here; I am probably too much infected
by J, in which arrays are immutable objects. And the more I think
about it, the more I like immutable arrays! But that's another point.
I rather like immutable arrays myself, but I was convinced that
mutable arrays were the right choice sometime in the middle of this
project, and I'm unlikely to change again.
current array module. Some operations are functions (e.g. reshape()),
others are methods (like concat()), but that seems to be completely
arbitrary.
Actually, reshape is now an array method, and the old reshape function
is only around for "historical reasons". I should probably remove it.
That's just a matter of efficiency and has to be tried. In my APL code
this function is used a lot, and often on small arrays.
Again, what would you call this new method?
Ehh... Which one? Concatenation? Well, I still hope to find
a solution that incorporates all applications into one, which
can then be called concat(). We should try to keep the number
of functions/methods to a minimum. So how about this:
function concat((a,b,c...), n=0)
concatenates arrays a,b,c,... along the nth axis and returns
the result. The shape along the other axes must agree.
This doesn't take care of new axes, but they can be added
easily with the pseudo index.
This can easily be added to the standard set of functions and
implemented on top of the existing transpose and concat methods. You
might want to do this as an exercise ;).
I was actually asking for a new name for the method that will do a
reshape allowing you to make copies of the array in the process. I'm
fairly convinced that I don't want this to be the default behavior of
the reshape method.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Jan 30 20:00:59 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Tue, 30 Jan 1996 15:00:59 -0500
Subject: [PYTHON MATRIX-SIG] Pseudo Indices
In-Reply-To: <199601301809.NAA08602@monty> (message from Guido van Rossum on Tue, 30 Jan 1996 13:09:42 -0500)
Message-ID: <199601302000.PAA26746@cyclone.ERE.UMontreal.CA>
> c[:, 1:2] --> c[Slice(), Slice(1,2)]
This is fine (and what I thought you were proposing all the time). I
would consider a patch if the existing 1-dim sequence and mapping
types are not affected.
What do you mean by "not affected"? That their semantics stays
the same (no problem) or that their implementation needn't
be changed? I am not sure the latter is possible for C-types.
> a[RubberIndex, 3].
You could sneak this into the abovementioned patch as "a[::, 3]".
I like that idea...
If I had enough time, I'd look at this, but that might not happen
for a while. Any other volunteers?
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Tue Jan 30 20:26:30 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Tue, 30 Jan 96 15:26:30 EST
Subject: [PYTHON MATRIX-SIG] Pseudo Indices
In-Reply-To: <199601302000.PAA26746@cyclone.ERE.UMontreal.CA> (hinsenk@ERE.UMontreal.CA)
Message-ID: <9601302026.AA04461@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: hinsenk@ERE.UMontreal.CA (Hinsen Konrad)
> c[:, 1:2] --> c[Slice(), Slice(1,2)]
This is fine (and what I thought you were proposing all the time). I
would consider a patch if the existing 1-dim sequence and mapping
types are not affected.
What do you mean by "not affected"? That their semantics stays
the same (no problem) or that their implementation needn't
be changed? I am not sure the latter is possible for C-types.
Chris Chase actually has a prototype implementation that doesn't alter
the implementation for C types. Basically it special cases these calls.
> a[RubberIndex, 3].
You could sneak this into the abovementioned patch as "a[::, 3]".
I like that idea...
If I had enough time, I'd look at this, but that might not happen
for a while. Any other volunteers?
Again, Chris Chase has a good starting point for an implementation of
this. I'll probably polish it up myself (unless I can talk him into
spending some more time on it) and add it to the NumericPython
distribution for testing.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Jan 30 20:43:22 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Tue, 30 Jan 1996 15:43:22 -0500
Subject: [PYTHON MATRIX-SIG] First experiences
In-Reply-To: <9601301950.AA04262@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199601302043.PAA29453@cyclone.ERE.UMontreal.CA>
Actually, reshape is now an array method, and the old reshape function
is only around for "historical reasons". I should probably remove it.
One or the other. I am not sure that "everything as a method" is the
right strategy. For example, operations that have two symmetric
operands look a bit odd as methods. There's also a difference in
behaviour, since array functions accept nested lists instead of
arrays, but methods can be called only on previously existing array
objects. In the case of the "copying reshape", the array argument
would often be a constant and it would make perfect sense to allow
nested lists there.
A useful way to make use of the function/method distinction would be
to use methods for operations that modify an array and functions for
operations that don't. The modified array would then be the one on
which the method is called, and it makes sense to require that this
actually is an array. There are not too many in-place operations right
now, but there will be once topics like linear algebra are dealt
with. For example, it would be nice to have
m.invert()
for an in-place inversion and
inverse(m)
for a function that returns the inverse without changing m.
This can easily be added to the standard set of functions and
implemented on top of the existing transpose and concat methods. You
might want to do this as an exercise ;).
No problem, but that would certainly be slow!
I was actually asking for a new name for the method that will do a
reshape allowing you to make copies of the array in the process. I'm
I see. Now that's difficult. I have been calling this operation
"reshape" since my high school days, so it is not easy to come
up with something else! Give me some time...
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Jan 30 17:26:11 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Tue, 30 Jan 1996 12:26:11 -0500
Subject: [PYTHON MATRIX-SIG] First experiences
In-Reply-To: <9601301635.AA03443@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199601301726.MAA16169@cyclone.ERE.UMontreal.CA>
I don't know whether or not sum even belongs in the set of standard
functions. On the one hand I'd rather make people use add.reduce
because it's much more in the spirit of the array object. On the
other hand, sum is such a friendly little function. If it remains in
the system, then I agree it should get the second argument.
In the spirit of "no needless redundancies", it would be better not to
have this function. Maybe we should add an (optional) module with
standard abbreviations, where we would simply write sum =
add.reduce. That would also be a good way to have clearer standard
names for things like arange() while still having short alternatives
that are available on every system.
And vice versa. This is mainly a matter of personal taste, and since
my taste favors fast_umath, that's going to be the default version for
Its not personal taste, but consistency. I have replaced "fast_umath"
by "umath" in my installation, but then immediately added "from
fast_umath import *" to my only application. I expect to use
mostly the fast version for debugged code, but still I think that
the default should obey the principle of least surprise, and that
is maximum similarity to Python scalar behaviour.
What you suggest would be very easy to code, but I don't want it
included in the behavior for reshape. I like having reshape error
check my dimensions for me, and I like the fact that reshape doesn't
copy the whole array, but just returns a new pointer to it.
It could still return a new pointer if the size is the same,
and make a copy only if the size changes. Personally I don't care
about dimension checking; in all my APL code that has never been
a problem. If the size remains the same, then typically the
new dimensions are specified as an expression involving the old
dimensions. I have never had an error there.
Try this:
a = array([1,2,3])
a.concat([4,5], [7,8])
-> array([1,2,3,4,5,7,8])
I assume that this is what you're looking for?
Not really. First of all, in most cases I need a non-destructive
version, similar to sequence addition. Second, I really need
it for all dimensions. In my application, I have two 1d arrays
of equal length and have to construct a 2d array from them.
What I used is
c = array([tuple(a), tuple(b)])
but there ought to be a much cleaner and more efficient way.
BTW, we also ought to have a way to convert arrays to nested
lists and maybe tuples. Oddly enough, Python has no built-in
function list(), although it does have tuple().
It should probably work along any axis, but I don't have a terribly
good way to specify the axis and still keep the nice feature of being
able to concat together an arbitrary number of arrays (very important
One argument could be a tuple of all arrays to be concatenated.
for efficiency). It would also be a bit of a hassle to write in C the
way you're suggesting, would it be enough to add in a standard python
function with the desired behavior?
That's just a matter of efficiency and has to be tried. In my APL code
this function is used a lot, and often on small arrays.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From chris.chase@jhuapl.edu Tue Jan 30 22:53:56 1996
From: chris.chase@jhuapl.edu (Chris Chase S1A)
Date: Tue, 30 Jan 1996 17:53:56 -0500
Subject: [PYTHON MATRIX-SIG] Pseudo Indices
In-Reply-To: <199601301809.NAA08602@monty>
References: <9601301742.AA03714@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
<199601301809.NAA08602@monty>
Message-ID: <199601302248.RAA28764@python.org>
>> a[RubberIndex, 3].
Guido> You could sneak this into the abovementioned patch as "a[::, 3]".
Actually, All is equivalent to "::" and it corresponds to the nil
index syntax of Yorick (which is indicated by whitespace between the
commas) and the * index syntax of Matlab or IDL.
Rubber indexes are different. Yorick has two types of rubber indexes,
one has syntax "*" and the other is ".." and neither type has the same
behavior as "::". Both types act as if the entire length of the
missing intervening dimensions were selected. "*" collapses the
missing intervening dimensions into 1 dimension. ".." does not
collapse the intervening dimensions (it is like using All for the
missing dimensions).
Currently the Numeric module RubberIndex is like the Yorick ".."
rubber index. I imagine that the collapsing type rubber index could
easily be added by applying a shape change to the current RubberIndex
behavior. What would be a possible name for this type of rubber
index? CollapseIndex?
Chris
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Wed Jan 31 15:56:04 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Wed, 31 Jan 96 10:56:04 EST
Subject: [PYTHON MATRIX-SIG] Slices as :
In-Reply-To: <199601302248.RAA28764@python.org> (message from Chris Chase S1A on Tue, 30 Jan 1996 17:53:56 -0500)
Message-ID: <9601311556.AA19776@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
The following is mainly directed to Chris, but I thought that people
on the list might be interested, so I decided to include the CC.
Well, it looks like your slices patch is going to get itself added to
python after all. Guido turned out to be a softer touch than I'd
expected. If you'd been at the workshop when I started talking about
rubber indices, you'd better understand my fears.
I'd like to see a couple of changes to your slices patch to make my
life easier for adding it into the NumericPython distribution.
Could you prepare a version of your patches that include both
1::2 -> slice(1,None,2)
and
::: or .. (you pick, but it might get changed by popular vote)
-> Py_Ellipses (a special object just like Py_None)
I'd also prefer to see 1:2:3:4 -> Syntax Error rather than a runtime
error.
I know that a[:] is unchanged by your patches for backwards
compatibilty. It would be nice if a[::-1] would be turned into
a[slice(None,None,-1) so that it could be passed to the mapping
protocol rather than being a syntax error.
I don't think that pseudo indices should have special notation as
these are particular to the array object. Konrad's name of
Numeric.NewAxis strikes me as a good idea.
I also don't think that it is necessary to add special notation for
the compressing rubber index as this is done easily enough with
reshape when needed.
Does this all sound reasonable to you?
Could you prepare this as a patch on top of the 0.32 distribution (on
top of Konrad's existing patches)?
As soon as you get this patch to me I'll add it into my working
version and to the next distribution.
I understand other commitments, but the sooner you could get this to
me the better.
Thanks for your help - Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================