From da@maigret.cog.brown.edu Fri Feb 2 00:40:45 1996
From: da@maigret.cog.brown.edu (David Ascher)
Date: Thu, 1 Feb 1996 19:40:45 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] lists vs. tuples
Message-ID: <199602020040.TAA28070@maigret>
Just a thought.
I believe that most existing python code, when returning collections,
return tuples.
The matrix code as implemented returns lists (e.g. shape). Shape
strikes me as the kind of thing which might be better off being a tuple
than a list. That way people doing:
a = array(...)
s = a.shape()
s[0] = 3
would be stopped by the interpreter (I'm assuming this isn't working and
isn't likely to work).
Thoughts?
--david
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Fri Feb 2 01:00:29 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Thu, 1 Feb 1996 20:00:29 -0500
Subject: [PYTHON MATRIX-SIG] lists vs. tuples
In-Reply-To: <199602020040.TAA28070@maigret> (da@maigret.cog.brown.edu)
Message-ID: <199602020100.UAA08987@cyclone.ERE.UMontreal.CA>
I believe that most existing python code, when returning collections,
return tuples.
I'd say it's mostly lists, but that's just my personal experience.
The matrix code as implemented returns lists (e.g. shape). Shape
Well, my installation returns tuples for shapes! Are we talking about
the same thing?
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Fri Feb 2 15:31:26 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Fri, 2 Feb 96 10:31:26 EST
Subject: [PYTHON MATRIX-SIG] lists vs. tuples
In-Reply-To: <199602020040.TAA28070@maigret> (da@maigret.cog.brown.edu)
Message-ID: <9602021531.AA01424@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: da@maigret.cog.brown.edu (David Ascher)
Just a thought.
I believe that most existing python code, when returning collections,
return tuples.
The matrix code as implemented returns lists (e.g. shape). Shape
strikes me as the kind of thing which might be better off being a tuple
than a list. That way people doing:
a = array(...)
s = a.shape()
s[0] = 3
would be stopped by the interpreter (I'm assuming this isn't working and
isn't likely to work).
Your code fragment won't work for two reasons. First, a.shape is a
data member, not a function (this is because you can do a.shape =
new_shape). And the data member a.shape is a tuple. You are allowed
to assign to a.shape from any sequence type, but it can only be
accessed as a tuple.
Are you thinking of some other problem?
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From da@maigret.cog.brown.edu Fri Feb 2 15:42:07 1996
From: da@maigret.cog.brown.edu (David Ascher)
Date: Fri, 2 Feb 1996 10:42:07 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] lists vs. tuples
In-Reply-To: <9602021531.AA01424@baribal.LCS.MIT.EDU.LCS.MIT.EDU> from "James Hugunin" at Feb 2, 96 10:31:26 am
Message-ID: <199602021542.KAA29265@maigret>
> And the data member a.shape is a tuple. You are allowed
> to assign to a.shape from any sequence type, but it can only be
> accessed as a tuple.
>
>> The matrix code as implemented returns lists (e.g. shape). Shape
> Well, my installation returns tuples for shapes! Are we talking about
> the same thing?
Hmm. Sorry. I was remembering Jim's description of the pseudoindices
etc., where he says things like "array([3,13]) is a one dimensional array
with a shape of [2]".
Clearly the implemenation is doing the right thing.
<take foot out of mouth>
Sorry for decreasing the signal-to-noise ratio...
--david
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Fri Feb 2 18:03:31 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Fri, 2 Feb 96 13:03:31 EST
Subject: [PYTHON MATRIX-SIG] Bug-Fix Release 0.33 available
Message-ID: <9602021803.AA02697@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Bug-Fix Release 0.33 of the numerical extension to python is available.
New in this release:
Several small BUG fixes.
Hopefully this release will compile "out of the box".
Konrad contributed a first cut at a decent print function for arrays
An expanded TUTORIAL.NumPy
mrange is now arange and honors the typecode (thanks Perry)
the function reshape is gone, use the method instead
A new function copyAndReshape is available that does what APL folks
expect from reshape (I think). A better name would be appreciated.
You can try picking up
ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumPy.patch-0.32-0.33.gz
If you copy this to the root python directory and type
patch < patch-0.32-0.33
You might have managed to pick up the latest release. After remaking
your system, rerun the test suite, and if the matrices are printed in
a pretty fashion, you'll know that the patch worked.
This is the first time that I've tried distributing a patch, so
please let me know whether or not this works for you.
If you're having problems, or don't trust patch, the complete tar file
is also available.
ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericalPython-0.33.tar.gz
Enjoy - Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Fri Feb 2 19:51:08 1996
From: hinsenk@ere.umontreal.ca (Hinsen Konrad)
Date: Fri, 2 Feb 1996 14:51:08 -0500
Subject: [PYTHON MATRIX-SIG] Bug-Fix Release 0.33 available
In-Reply-To: <9602021803.AA02697@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199602021951.OAA07396@cyclone.ERE.UMontreal.CA>
This is the first time that I've tried distributing a patch, so
please let me know whether or not this works for you.
No. Patch kept asking me which file it should patch (as if I could
know that!), and I preferred downloading the complete tar file
to ruining my installation.
Otherwise the new version seems to work; at least the test suite runs
without problems. The new function copyAndReshape doesn't work at all
- it references an undefined name "ArrayType". I replaced that by
type(array([0])), but then I got another error later on, telling me
that the "dimensions are not legal". That of course should never
happen if the function does what it is supposed to do.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From chris.chase@jhuapl.edu Fri Feb 2 10:54:20 1996
From: chris.chase@jhuapl.edu (Chris Chase S1A)
Date: Fri, 2 Feb 1996 15:54:20 +0500
Subject: [PYTHON MATRIX-SIG] Bug-Fix Release 0.33 available
In-Reply-To: <199602021951.OAA07396@cyclone.ERE.UMontreal.CA>
References: <9602021803.AA02697@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
<199602021951.OAA07396@cyclone.ERE.UMontreal.CA>
Message-ID: <9602022054.AA04256@hurlbut.>
Hinsen> No. Patch kept asking me which file it should patch (as if I could
Hinsen> know that!), and I preferred downloading the complete tar file
Hinsen> to ruining my installation.
I got the patch to work. After patching the first few files, my patch
program also started asking what file to patch. I just copied and
pasted the file name shown from the patch header, i.e.,
when patch showed:
diff -c -r OldVersion/Modules/ofuncobject.c ./Modules/ofuncobject.c
*** OldVersion/Modules/ofuncobject.c Mon Jan 29 10:29:30 1996
--- ./Modules/ofuncobject.c Thu Feb 1 16:57:23 1996
***************
And asked what file to patch I copied and pasted to the prompt
./Modules/ofuncobject.c
I don't know why patch needed the file names after being succesful the
first few patches.
Chris
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Fri Feb 2 23:11:57 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Fri, 2 Feb 1996 18:11:57 -0500
Subject: [PYTHON MATRIX-SIG] New pretty printer
Message-ID: <199602022311.SAA23116@cyclone.ERE.UMontreal.CA>
My pretty printer has made some progress, and I'll include the latest
version below. Just put it into your copy of Numeric.py. Line wrapping
is still missing. (BTW, what would be a good default line width? In the
VT100 days that would have been 80 characters, but does this still
make sense?) I can't say that I am satisfied with float formatting,
but anything better than now will be difficult to do and, most of
all, slow. The two problems are
1) Trailing zeros in non-exponential format. Right now there are
always eight digits after the decimal point. It would be better
to replace trailing zeros by spaces (easy) and reduce the width
of the output fields to match the longest remaining number.
But that requires a scan of the string representations of all
the elements in an array, which is an expensive operation.
2) In exponential notation, numbers with three-digit exponents will
not line up correctly with others. It would be necessary to
have three-digit exponents for all numbers when at least
one element requires it, but there is no formatting option
to specify the number of exponent digits. The only solution
would be cosmetic surgery on the final string, but that again
is slow.
There are currently some problems with complex number output due to a
remaining bug in the array module. Normally it will only lead to a
wrong format selection, so it's not fatal.
And now the code:
def arrayToString(a, max_line_length = 80):
if a.contiguous():
data = a.reshape(None)
else:
data = a.copy().reshape(None)
type = a.typecode()
items_per_line = a.shape[-1]
if type == 'b' or type == '1' or type == 's' or type == 'i' \
or type == 'l':
max_str_len = max(len(str(maximum.reduce(data))),
len(str(minimum.reduce(data))))
format = '%' + str(max_str_len) + 'd'
type = 'l'
item_length = max_str_len+1
elif type == 'f' or type == 'd':
format, item_length = _floatFormat(data)
type = 'd'
elif type == 'F' or type == 'D':
real_format, real_item_length = _floatFormat(data.real, sign=0)
imag_format, imag_item_length = _floatFormat(data.imaginary, sign=1)
format = '(' + real_format + imag_format + 'j)'
item_length = real_item_length + imag_item_length + 2
type = 'D'
elif type == 'c':
format = '%s'
item_length = 1
else: # nothing for 'O'
return str(a)
line_length = item_length*items_per_line - (type != 'c')
if line_length > max_line_length:
pass
return _arrayToString(a, type, format, len(a.shape), line_length)[:-1]
def _floatFormat(data, sign = 0):
exp_format = 0
non_zero = abs(compress(data.notEqual(0), data))
print str(data), str(data.notEqual(0)), str(compress(data.notEqual(0), data))
if len(non_zero) == 0:
max_val = 0.
min_val = 0.
else:
max_val = maximum.reduce(non_zero)
min_val = minimum.reduce(non_zero)
if max_val >= 1.e12 or min_val < 0.0001 or max_val/min_val > 1000.:
exp_format = 1
if exp_format:
max_str_len = 16 + (0 < min_val < 1e-99 or max_val >= 1e100)
if sign: format = '%+'
else: format = '%'
format = format + str(max_str_len) + '.8e'
item_length = max_str_len + 1
else:
max_str_len = len(str(int(max_val))) + 10
if sign: format = '%+'
else: format = '%'
format = format + str(max_str_len) + '.8f'
item_length = max_str_len + 1
return (format, item_length)
def _arrayToString(a, type, format, rank, line_length):
if rank == 0:
return str(a[0])
elif rank == 1:
s = ''
if type == 'c':
for i in range(a.shape[0]):
s = s + (format % a[i])
s = s + '\n'
elif type == 'D':
for i in range(a.shape[0]):
s = s + (format % (a[i].real, a[i].imag)) + ' '
s = s[:-1] + '\n'
else:
for i in range(a.shape[0]):
s = s + (format % a[i]) + ' '
s = s[:-1] + '\n'
else:
s = ''
for i in range(a.shape[0]-1):
s = s + _arrayToString(a[i], type, format, rank-1, line_length)
if rank == 3:
s = s + '\n'
elif rank > 3:
s = s + (rank-3)*(line_length*'-'+'\n')
s = s + _arrayToString(a[a.shape[0]-1], type, format, rank-1, line_length)
return s
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From guido@CNRI.Reston.VA.US Sat Feb 3 00:06:42 1996
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Fri, 02 Feb 1996 19:06:42 -0500
Subject: [PYTHON MATRIX-SIG] New pretty printer
In-Reply-To: Your message of "Fri, 02 Feb 1996 18:11:57 EST."
<199602022311.SAA23116@cyclone.ERE.UMontreal.CA>
References: <199602022311.SAA23116@cyclone.ERE.UMontreal.CA>
Message-ID: <199602030006.TAA10516@monty>
> My pretty printer has made some progress, and I'll include the latest
> version below. Just put it into your copy of Numeric.py. Line wrapping
> is still missing. (BTW, what would be a good default line width? In the
> VT100 days that would have been 80 characters, but does this still
> make sense?)
Yes! I'd even say 77, so at least one level of email quoting won't
cause it to wrap. I keep my Emacs religiously at 80 columns wide when
programming -- this way I can fit two columns of windows plus some
icons on my screen. Also, the default wrap columns for printing text
seems to be 80 still.
> I can't say that I am satisfied with float formatting,
> but anything better than now will be difficult to do and, most of
> all, slow. The two problems are
>
> 1) Trailing zeros in non-exponential format. Right now there are
> always eight digits after the decimal point. It would be better
> to replace trailing zeros by spaces (easy) and reduce the width
> of the output fields to match the longest remaining number.
> But that requires a scan of the string representations of all
> the elements in an array, which is an expensive operation.
>
> 2) In exponential notation, numbers with three-digit exponents will
> not line up correctly with others. It would be necessary to
> have three-digit exponents for all numbers when at least
> one element requires it, but there is no formatting option
> to specify the number of exponent digits. The only solution
> would be cosmetic surgery on the final string, but that again
> is slow.
(For both case:) Surely, if you're printing a reasonably sized array,
it will be fast enough, and if it is truly huge, nobody will really
look at the numbers... I'd say make it look as good as you can.
Also, perhaps a regular expression can help you do the cosmetic
surgery efficiently.
--Guido van Rossum <guido@CNRI.Reston.VA.US>
URL: <http://www.python.org/~guido/>
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From guido@CNRI.Reston.VA.US Sat Feb 3 00:06:42 1996
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Fri, 02 Feb 1996 19:06:42 -0500
Subject: [PYTHON MATRIX-SIG] New pretty printer
In-Reply-To: Your message of "Fri, 02 Feb 1996 18:11:57 EST."
<199602022311.SAA23116@cyclone.ERE.UMontreal.CA>
References: <199602022311.SAA23116@cyclone.ERE.UMontreal.CA>
Message-ID: <199602030006.TAA10516@monty>
> My pretty printer has made some progress, and I'll include the latest
> version below. Just put it into your copy of Numeric.py. Line wrapping
> is still missing. (BTW, what would be a good default line width? In the
> VT100 days that would have been 80 characters, but does this still
> make sense?)
Yes! I'd even say 77, so at least one level of email quoting won't
cause it to wrap. I keep my Emacs religiously at 80 columns wide when
programming -- this way I can fit two columns of windows plus some
icons on my screen. Also, the default wrap columns for printing text
seems to be 80 still.
> I can't say that I am satisfied with float formatting,
> but anything better than now will be difficult to do and, most of
> all, slow. The two problems are
>
> 1) Trailing zeros in non-exponential format. Right now there are
> always eight digits after the decimal point. It would be better
> to replace trailing zeros by spaces (easy) and reduce the width
> of the output fields to match the longest remaining number.
> But that requires a scan of the string representations of all
> the elements in an array, which is an expensive operation.
>
> 2) In exponential notation, numbers with three-digit exponents will
> not line up correctly with others. It would be necessary to
> have three-digit exponents for all numbers when at least
> one element requires it, but there is no formatting option
> to specify the number of exponent digits. The only solution
> would be cosmetic surgery on the final string, but that again
> is slow.
(For both case:) Surely, if you're printing a reasonably sized array,
it will be fast enough, and if it is truly huge, nobody will really
look at the numbers... I'd say make it look as good as you can.
Also, perhaps a regular expression can help you do the cosmetic
surgery efficiently.
--Guido van Rossum <guido@CNRI.Reston.VA.US>
URL: <http://www.python.org/~guido/>
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From da@maigret.cog.brown.edu Sun Feb 4 22:54:46 1996
From: da@maigret.cog.brown.edu (David Ascher)
Date: Sun, 4 Feb 1996 17:54:46 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] slicing rank-1 arrays
Message-ID: <199602042254.RAA08528@maigret>
Given the following, which seems fine:
>>> a
0 1 2 3 4
0 1 2 3 4
0 1 2 3 4
0 1 2 3 4
>>> a[All,All]
0 1 2 3 4
0 1 2 3 4
0 1 2 3 4
0 1 2 3 4
why doesn't the following work?
>>> b
0 1 2 3 4
>>> b[All]
Traceback (innermost last):
File "<stdin>", line 1, in ?
AttributeError: __len__
>>> b[Reverse]
Traceback (innermost last):
File "<stdin>", line 1, in ?
AttributeError: __len__
[Using 0.33 on an SGI]
--david
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From da@maigret.cog.brown.edu Sun Feb 4 23:00:26 1996
From: da@maigret.cog.brown.edu (David Ascher)
Date: Sun, 4 Feb 1996 18:00:26 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] New Tutorial / Some Questions
Message-ID: <199602042300.SAA08594@maigret>
Given that I'm starting a course on Python on Tuesday for a bunch of
neural net folks, I had to take a good hard look at the Numeric package.
Jim H. has gotten more mail from me than he probably wanted. The good
thing is that I didn't see myself teaching them about a feature that had
no documentation. So, I started a more detailed tutorial than that Jim
shipped in 0.33. I aimed it at people who have very little python
experience, and no prior experience w/ matlab, basis, yorick or anything
like it (i.e. undergraduates =). It's obviously too basic for the
folks in this SIG, but I'd appreciate it if you could give it a
once-over and make sure I'm not lying outright.
It's available at http://maigret.cog.brown.edu/python/arraytut.html
for now. It is very much under construction (I haven't explained rubber
indices and pseudo indices, for example, mostly because I'm not sure I
understand them yet).
I came up with a few questions in the process. Some have gone to jim
h. and hinsen, but I thought I'd spread the load:
1. I can't seem to find the counterpart to the toFile() method.
I've tried just the array constructor w/ a file object, which I
thought might work, but I get:
>>> a
0 0 1
0 0 0
1 2 3
>>> f = open('myarray', 'w')
>>> a.toFile(f)
>>> f.close()
>>> f = open('myarray', 'r')
>>> b = array(f)
>>> b
Traceback (innermost last):
File "<stdin>", line 1, in ?
File "/usr/local/lib/python/numeric/Numeric.py", line 37, in
arrayToString
items_per_line = a.shape[-1]
IndexError: tuple index out of range
2. The choose() and take() methods are a little fuzzy for me. Can
someone give me a description of what they do, and more useful even
examples of why they're useful?
3. Shouldn't typecodes look more like real types than strings, sort of
like exceptions?
>>> b.typecode()
'l'
vs.
>>> b.typecode()
LongInteger or whatever.
That's all for now. Let me know what you think of the tutorial.
--david
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Mon Feb 5 16:13:43 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Mon, 5 Feb 96 11:13:43 EST
Subject: [PYTHON MATRIX-SIG] slicing rank-1 arrays
In-Reply-To: <199602042254.RAA08528@maigret> (da@maigret.cog.brown.edu)
Message-ID: <9602051613.AA26208@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: da@maigret.cog.brown.edu (David Ascher)
Given the following, which seems fine:
>>> a
0 1 2 3 4
0 1 2 3 4
0 1 2 3 4
0 1 2 3 4
>>> a[All,All]
0 1 2 3 4
0 1 2 3 4
0 1 2 3 4
0 1 2 3 4
why doesn't the following work?
>>> b
0 1 2 3 4
>>> b[All]
Traceback (innermost last):
File "<stdin>", line 1, in ?
AttributeError: __len__
>>> b[Reverse]
Traceback (innermost last):
File "<stdin>", line 1, in ?
AttributeError: __len__
[Using 0.33 on an SGI]
This is a bug. I'll fix it.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Mon Feb 5 15:56:03 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Mon, 5 Feb 96 10:56:03 EST
Subject: [PYTHON MATRIX-SIG] New Tutorial / Some Questions
In-Reply-To: <199602042300.SAA08594@maigret> (da@maigret.cog.brown.edu)
Message-ID: <9602051556.AA26139@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: da@maigret.cog.brown.edu (David Ascher)
Given that I'm starting a course on Python on Tuesday for a bunch of
neural net folks, I had to take a good hard look at the Numeric package.
Jim H. has gotten more mail from me than he probably wanted. The good
Actually, I love the mail. This is the best way to really shake down
the package.
I came up with a few questions in the process. Some have gone to jim
h. and hinsen, but I thought I'd spread the load:
Well, since nobody else has answered yet (and since I'd LOVE to see
your course succeed).
1. I can't seem to find the counterpart to the toFile() method.
There used to be such a function in Numeric.py. The reason it's not
there any more is to strongly encourage people to use pickling of
matrices and better yet of objects containing matrices instead of the
raw toFile and fromFile methods. This is the method I use to store
the fairly complicated GaussianClassifier objects that I use for
speech recognition. This approach will gain you a certain degree of
architecture independence (though 64-bit longs on alpha's give it
trouble) and will cost almost nothing in performance terms.
Note, you can still read in an array from a file if you need to for
compatibility reasons as follows:
a = fromString(fp.read())
2. The choose() and take() methods are a little fuzzy for me. Can
someone give me a description of what they do, and more useful even
examples of why they're useful?
Actually, Konrad or Paul could probably do a better job of this than
me, but here are my two favorite examples.
static char doc_choose[] = "m.choose(m_0, ..., m_(n-1)). m is a array
of integers from 0 to n-1. Each entry in m will be taken from the
corresponding entry in m_i, where i is the value of m. Frequently
used with exactly two arguments for a boolean valued m,
ie. m.choose(m_false, m_true).";
Assume a is an array that you want to "clip" so that no values are
greater than 100.0.
(a.greater(100.0)).choose(a, 100.0)
Everywhere that a.greater(100.0) is false (ie. 0) this will "choose"
the corresponding value in a. Everywhere else it will "choose" 100.0.
static char doc_take[] = "m.take([i_0, ..., i_n], dimension=0).
Selects the items from m along dimension";
Take is the equivalent of "full product form indexing" that for a
number of reasons is not available in normal indexing. The most
common use I have for take is to implement decoding from mu-law to
16-bit linear sound files. mu-law files map the bytes from 0-255 to a
given 16-bit number using a non-linear scale as a form of compression.
Assuming you have an array mu_law_array (of length 256) where
mu_law_array[value_mu_law] = value_linear
linear_encoded_array = mu_law_array.take(mu_law_encoded_array)
3. Shouldn't typecodes look more like real types than strings, sort of
like exceptions?
>>> b.typecode()
'l'
vs.
>>> b.typecode()
LongInteger or whatever.
The only hassle is that doing this would lead to yet another new
object type added to python (after already adding array's, ufunc's,
and slices (hopefully soon). I'm willing to be convinced that this is
the right choice, but for now I opted for the minimal approach.
That's all for now. Let me know what you think of the tutorial.
Will look at it ASAP.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From nick@osg.saic.com Mon Feb 5 14:34:21 1996
From: nick@osg.saic.com (Nick Seidenman)
Date: Mon, 5 Feb 1996 09:34:21 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] NumPython
Message-ID: <9602051434.AA02953@typhoon.osg.saic.com>
Just downloaded and built 0.33. No problems so far! Compiled right out
of the box on a linux 1.2.13 system rnning on my handy-dandy pentium
laptop! Thanks, Jim!
nick
---------------------------
Nick Seidenman
Science Applications International Corporation
1710 Goodridge Drive
McLean, VA 22102
Phone: (703) 448-6497
(fax): (703) 821-3576
email: nick@osg.saic.com
http://www.osg.saic.com/~nick/
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Mon Feb 5 16:40:38 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Mon, 5 Feb 1996 11:40:38 -0500
Subject: [PYTHON MATRIX-SIG] New Tutorial / Some Questions
In-Reply-To: <199602042300.SAA08594@maigret> (da@maigret.cog.brown.edu)
Message-ID: <199602051640.LAA05335@cyclone.ERE.UMontreal.CA>
3. Shouldn't typecodes look more like real types than strings, sort of
like exceptions?
>>> b.typecode()
'l'
vs.
>>> b.typecode()
LongInteger or whatever.
Definitely. Especially now that we have the stuff in Precision.py
for specifying types, it should be used consistently wherever
possible.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Mon Feb 5 21:23:57 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Mon, 5 Feb 1996 16:23:57 -0500
Subject: [PYTHON MATRIX-SIG] New Tutorial / Some Questions
In-Reply-To: <9602051556.AA26139@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199602052123.QAA24690@cyclone.ERE.UMontreal.CA>
1. I can't seem to find the counterpart to the toFile() method.
There used to be such a function in Numeric.py. The reason it's not
there any more is to strongly encourage people to use pickling of
matrices and better yet of objects containing matrices instead of the
raw toFile and fromFile methods. This is the method I use to store
That makes sense, but then toFile() should also disappear.
Take is the equivalent of "full product form indexing" that for a
number of reasons is not available in normal indexing. The most
common use I have for take is to implement decoding from mu-law to
16-bit linear sound files. mu-law files map the bytes from 0-255 to a
given 16-bit number using a non-linear scale as a form of compression.
Assuming you have an array mu_law_array (of length 256) where
mu_law_array[value_mu_law] = value_linear
linear_encoded_array = mu_law_array.take(mu_law_encoded_array)
Translation tables are one of the important applications of this
operation. Another is sorting, ...ehhh... *would* be sorting if it
were implemented in a more flexible way. Right now we have the
built-in function sort() which works also on arrays. What is missing
(and should be added) is a function that returns the indices of the
elements in sorted order. As an example, suppose you have an array of
measurements of some function f(x) as a rank-2 array a; a[0] being the
evaluation points and a[1] the values. You want to sort the array by
evaluation points. The answer is
a.take(a[0].sortIndex(), 1)
BTW, it would be extremely helpful to have a list of all existing
functions/methods with a short definition. It is difficult to comment
on whether the current set is sufficient and/or appropriately defined
and names without knowing what is out there! Do you happen to have
something suitable?
3. Shouldn't typecodes look more like real types than strings, sort of
like exceptions?
>>> b.typecode()
'l'
vs.
>>> b.typecode()
LongInteger or whatever.
The only hassle is that doing this would lead to yet another new
object type added to python (after already adding array's, ufunc's,
and slices (hopefully soon). I'm willing to be convinced that this is
the right choice, but for now I opted for the minimal approach.
There is no need for a new object type, since the typecode could be of
type "type". All that has to be added is type objects for the new
types.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From da@maigret.cog.brown.edu Mon Feb 5 21:49:33 1996
From: da@maigret.cog.brown.edu (David Ascher)
Date: Mon, 5 Feb 1996 16:49:33 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] New Tutorial / Some Questions
In-Reply-To: <199602052123.QAA24690@cyclone.ERE.UMontreal.CA> from "Konrad HINSEN" at Feb 5, 96 04:23:57 pm
Message-ID: <199602052149.QAA11829@maigret>
> BTW, it would be extremely helpful to have a list of all existing
> functions/methods with a short definition. It is difficult to comment
> on whether the current set is sufficient and/or appropriately defined
> and names without knowing what is out there! Do you happen to have
> something suitable?
I've tried to list them all in the tutorial. Well, at least they should
all be mentioned in the tutorial. A whole bunch of them have no
definitions yet, but that will get fixed. I just got them from browsing
the source, so I may very well have missed some.
--david
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dubois1@llnl.gov Mon Feb 5 22:16:04 1996
From: dubois1@llnl.gov (Paul. Dubois)
Date: Mon, 05 Feb 1996 14:16:04 -0800
Subject: [PYTHON MATRIX-SIG] New pretty printer
References: <199602022311.SAA23116@cyclone.ERE.UMontreal.CA> <199602030006.TAA10516@monty>
Message-ID: <311681A4.1E79@llnl.gov>
I suggest a default size of 79 (or 77 as suggested by Guido). Exactly 80
causes wrap in some display vehicles.
I didn't look yet at how you did it but perhaps a user-settable
precision could be managed. I have done that in both Basis and
EiffelMath. Guido is right about the timing too; if there are enough
components that you have to worry about the timing to print it, nobody
is going to print it anyway or look at it if they do.
One thing I did in Basis was an optional "compression" that compressed
identical rows (or strings of identical elements in 1-D).
Basis> ones(10000)
ones(10000) shape: (10000)
1: 9976 - 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1
9977: - 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
Basis> shape([1,2,3]//ones(10000)//[5,6],2001,5)
shape([1,2,3]//ones(10000)//[5,6],2001,5) shape: (2001,5)
row col = 1 2 3 4 5
1: - 1 1 1 1 1
2: - 2 1 1 1 1
3: - 3 1 1 1 1
4:1999 - 1 1 1 1 1
2000: - 1 1 1 1 5
2001: - 1 1 1 1 6
Basis>
This has proven useful.
--
Paul F. Dubois, L-472 (510)-422-5426
Lawrence Livermore National Laboratory FAX (510)-423-9969
Livermore, CA 94550 dubois1@llnl.gov
Consulting: PaulDubois@aol.com
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Feb 6 00:11:50 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Mon, 5 Feb 1996 19:11:50 -0500
Subject: [PYTHON MATRIX-SIG] New pretty printer
In-Reply-To: <311681A4.1E79@llnl.gov> (dubois1@llnl.gov)
Message-ID: <199602060011.TAA05144@cyclone.ERE.UMontreal.CA>
I didn't look yet at how you did it but perhaps a user-settable
precision could be managed. I have done that in both Basis and
Just implemented :-) The problem with both this and the line width is
that there is no convenient way to set them when simply printing
arrays. Right now, you have to call the function arrayToString()
explicitly with additional parameters. I could of course define global
variables in module Numeric to handle this, but I am not sure whether
that is a good idea. Maybe a good compromise is optional parameters
that are set from global variables if undefined. Comments?
EiffelMath. Guido is right about the timing too; if there are enough
components that you have to worry about the timing to print it, nobody
is going to print it anyway or look at it if they do.
I tend to agree, especially since you can always interrupt the output,
as long as the function is written in Python.
One thing I did in Basis was an optional "compression" that compressed
identical rows (or strings of identical elements in 1-D).
That could indeed be useful, and not too complicated to implement.
Another option...
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Tue Feb 6 14:21:02 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Tue, 6 Feb 96 09:21:02 EST
Subject: [PYTHON MATRIX-SIG] New pretty printer
In-Reply-To: <199602060011.TAA05144@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca)
Message-ID: <9602061421.AA20009@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
I didn't look yet at how you did it but perhaps a user-settable
precision could be managed. I have done that in both Basis and
Just implemented :-) The problem with both this and the line width is
that there is no convenient way to set them when simply printing
arrays. Right now, you have to call the function arrayToString()
explicitly with additional parameters. I could of course define global
variables in module Numeric to handle this, but I am not sure whether
that is a good idea. Maybe a good compromise is optional parameters
that are set from global variables if undefined. Comments?
The compromise approach sounds perfect. I think that this is one of
those rare cases where there is no potential for problems to be caused
by global variables because you are not going to be altering the
results of any computation, only the way those results are displayed.
Without global variables there is just no reasonable way to alter the
default behavior for "print a".
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Tue Feb 6 14:32:18 1996
From: jfulton@usgs.gov (Jim Fulton, U.S. Geological Survey)
Date: Tue, 6 Feb 1996 09:32:18 -0500
Subject: [PYTHON MATRIX-SIG] New pretty printer
In-Reply-To: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
"Re: [PYTHON MATRIX-SIG] New pretty printer" (Feb 6, 9:21am)
References: <9602061421.AA20009@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <9602060932.ZM10571@dsdbqvarsa.er.usgs.gov>
On Feb 6, 9:21am, James Hugunin wrote:
> Subject: Re: [PYTHON MATRIX-SIG] New pretty printer
>
> From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
>
> I didn't look yet at how you did it but perhaps a user-settable
> precision could be managed. I have done that in both Basis and
>
> Just implemented :-) The problem with both this and the line width is
> that there is no convenient way to set them when simply printing
> arrays. Right now, you have to call the function arrayToString()
> explicitly with additional parameters. I could of course define global
> variables in module Numeric to handle this, but I am not sure whether
> that is a good idea. Maybe a good compromise is optional parameters
> that are set from global variables if undefined. Comments?
>
> The compromise approach sounds perfect. I think that this is one of
> those rare cases where there is no potential for problems to be caused
> by global variables because you are not going to be altering the
> results of any computation, only the way those results are displayed.
But other modules may have different display needs, so you can still effect a
different piece of code.
> Without global variables there is just no reasonable way to alter the
> default behavior for "print a".
You could make this parameter a part of the state of an array.
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@CHIMCN.UMontreal.CA Wed Feb 7 15:11:56 1996
From: hinsenk@CHIMCN.UMontreal.CA (Hinsen Konrad)
Date: Wed, 7 Feb 1996 10:11:56 -0500
Subject: [PYTHON MATRIX-SIG] FTP address
Message-ID: <199602071511.KAA03680@chims1.CHIMCN.UMontreal.CA>
Due to a major server breakdown I am cut off from all my files and my
normal e-mail address. Nevertheless I'd like to go on playing with
the array package. I have already reinstalled Python, but I have to
get the numerics extension again. Could anyone please send me the
address of Jim's FTP server?
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@chims1.chimcn.umontreal.ca
Departement de Chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. A | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Wed Feb 7 16:52:33 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Wed, 7 Feb 1996 11:52:33 -0500
Subject: [PYTHON MATRIX-SIG] New pretty printer
In-Reply-To: <9602060932.ZM10571@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov)
Message-ID: <199602071652.LAA04701@cyclone.ERE.UMontreal.CA>
But other modules may have different display needs, so you can still effect a
different piece of code.
> Without global variables there is just no reasonable way to alter the
> default behavior for "print a".
You could make this parameter a part of the state of an array.
That implies that precision and line width are properties of an
array. For precision this may be justified, but certainly not
for line width!
Such things really belong into a "system parameters" collection.
The closest Python equivalent would be the module "sys". But
that doesn't solve the problem of who is allowed to modify them
how and when.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From guido@CNRI.Reston.VA.US Wed Feb 7 17:01:39 1996
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Wed, 07 Feb 1996 12:01:39 -0500
Subject: [PYTHON MATRIX-SIG] New pretty printer
In-Reply-To: Your message of "Wed, 07 Feb 1996 11:52:33 EST."
<199602071652.LAA04701@cyclone.ERE.UMontreal.CA>
References: <199602071652.LAA04701@cyclone.ERE.UMontreal.CA>
Message-ID: <199602071701.MAA20712@monty.cnri.reston.va.us>
> Such things really belong into a "system parameters" collection.
> The closest Python equivalent would be the module "sys". But
> that doesn't solve the problem of who is allowed to modify them
> how and when.
It makes sense for it to be a property of the file object. But since
adding properties to those is no fun, I'd suggest to go for a
parameter is sys. Whoever feels like they can change sys.stdout can
also change the line width if they feel like it.
--Guido van Rossum <guido@CNRI.Reston.VA.US>
URL: <http://www.python.org/~guido/>
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From guido@CNRI.Reston.VA.US Wed Feb 7 17:01:39 1996
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Wed, 07 Feb 1996 12:01:39 -0500
Subject: [PYTHON MATRIX-SIG] New pretty printer
In-Reply-To: Your message of "Wed, 07 Feb 1996 11:52:33 EST."
<199602071652.LAA04701@cyclone.ERE.UMontreal.CA>
References: <199602071652.LAA04701@cyclone.ERE.UMontreal.CA>
Message-ID: <199602071701.MAA20712@monty.cnri.reston.va.us>
> Such things really belong into a "system parameters" collection.
> The closest Python equivalent would be the module "sys". But
> that doesn't solve the problem of who is allowed to modify them
> how and when.
It makes sense for it to be a property of the file object. But since
adding properties to those is no fun, I'd suggest to go for a
parameter is sys. Whoever feels like they can change sys.stdout can
also change the line width if they feel like it.
--Guido van Rossum <guido@CNRI.Reston.VA.US>
URL: <http://www.python.org/~guido/>
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Wed Feb 7 17:11:21 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Wed, 7 Feb 1996 12:11:21 -0500
Subject: [PYTHON MATRIX-SIG] New pretty printer
In-Reply-To: <199602071701.MAA20712@monty.cnri.reston.va.us> (message from Guido van Rossum on Wed, 07 Feb 1996 12:01:39 -0500)
Message-ID: <199602071711.MAA07277@cyclone.ERE.UMontreal.CA>
It makes sense for it to be a property of the file object. But since
adding properties to those is no fun, I'd suggest to go for a
parameter is sys. Whoever feels like they can change sys.stdout can
also change the line width if they feel like it.
OK. Then it makes sense to add another parameter for printing
precision and use it also for scalars.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Wed Feb 7 19:48:13 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Wed, 7 Feb 96 14:48:13 EST
Subject: [PYTHON MATRIX-SIG] Steadily approaching BETA release
Message-ID: <9602071948.AA17725@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Chris Chase just sent me a bunch of patches to allow a[::-1, 1:-1]
style indexing. So the grammar is solidifying.
David Ascher's tutorial (and my own TUTORIAL.NumPy) are starting to
produce something like an acceptable first pass at documentation.
Konrad Hinsen's print function offers acceptable printing of arrays.
I've received very few bug reports this week, so the base code is
starting to look stable.
So, what's left before a general BETA release?
Note: I'm going to comment on each of these in a seperate follow-up
piece of e-mail in order to try and seperate these two different issues.
1) Type coercion
2) Naming conventions
What I've decided is not important to resolve before the BETA release:
1) static vs. dynamic linking (static linking is fine for now)
2) Contents of Numeric.py (I'm going to rule by fiat here)
3) The exact details of Konrad's print function (Unlikley that changes
here will break anyone's code)
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Wed Feb 7 20:18:00 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Wed, 7 Feb 96 15:18:00 EST
Subject: [PYTHON MATRIX-SIG] Type Coercion
Message-ID: <9602072018.AA17817@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
I still don't see a great solution to this one.
Here's my attempt to boil the issue down to its essence:
(While I only talk about floats and doubles here, this same issue
comes up with longs and shorts and bytes as well as float and double
complex numbers).
C (and FORTRAN) has two kinds of floating point numbers, float and
double. A double has more precision and range than a float.
In C if I multiply a double by a float I get a double.
Python has only one floating point type, call this a PyFloat.
Internally, these numbers store their data using C doubles. However,
the standard C interface to PyFloats allows them to be extracted into
either a C float or a C double (PyArg_ParseTuple), so its not clear
that a PyFloat should really be considered either a float or a double.
Nevertheless, its obvious that PyFloats are more closely related to
doubles than floats.
An array can be of type float or double.
It is reasonable to have the result of multiplying an array of floats
by an array of doubles be an array of doubles.
What should happen if I multiply an array of floats by a PyFloat?
Three things can happen:
1) Return an array of floats (possible undesired loss of precision)
2) Return an array of doubles (possible undesired gain of precision)
3) Raise an exception (very annoying to the user)
Currently there is also the convention that when an array of floats or
doubles returns a single element, that element will be a PyFloat.
Perhaps this behavior should be altered as the following examples of
confusing behavior using either approach 1 or approach 2 show.
a_float and a_double are 1d arrays of the appropriate type.
Using Method 1)
a_float[0]*a_float -> a_float (good)
a_double[0]*a_float -> a_float (not good)
Using Method 2)
a_float[0]*a_float -> a_double (not good)
a_double[0]*a_float -> a_double (good)
This particular problem could be eliminated by having a_float[0]
return a 0d array of floats with a single element. Perhaps this is
worth considering.
The next confusing behavior involves python numeric constants.
Again, considering the two non-exception producing methods discussed
above:
very_precise_constant = 1.2345678910
Using method 1)
0.5*a_float -> a_float (good)
very_precise_constant*a_float -> a_float (probably not good)
Using method 2)
0.5*a_float -> a_double (not good)
very_precise_constant*a_float -> a_double (probably good)
Using this method, the only way to write 0.5*a_float and get an array
of floats back is to write:
array(0.5, Float(32))*a_float
This becomes very ugly for any reasonably sized expression.
Currently, method 3 is used because it seems to make everybody
unhappy, yet it doesn't do anything terribly "wrong".
Personally I would prefer to use method 1, combined with the major
change to the existing system that scalars are returned as 0d arrays,
not as python scalars.
This means that python scalars will be considered to have a malleable
type (as they in fact seem to) and can be cast to either a float or a
double as needed in the particular equation. If you have a
high-precision number whose precision you need to hold on to, you will
need to say:
very_precise_constant = array(1.2345678910, Float(64))
As I said at the beginning, none of these three options really appeals
to me, yet I feel that this is a very important issue that really
needs to be resolved before the BETA release.
Opinions? (like I have to ask on this issue) - Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Wed Feb 7 20:51:28 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Wed, 7 Feb 1996 15:51:28 -0500
Subject: [PYTHON MATRIX-SIG] Steadily approaching BETA release
In-Reply-To: <9602071948.AA17725@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199602072051.PAA21905@cyclone.ERE.UMontreal.CA>
So, what's left before a general BETA release?
Note: I'm going to comment on each of these in a seperate follow-up
piece of e-mail in order to try and seperate these two different issues.
1) Type coercion
2) Naming conventions
3) Practical experience
I don't know how much real-life code others have produced with
the current release, but I find that even with the relatively
modest amount of application code I have written, I keep finding
weak spots in the current collection of functions, i.e. operations
that are impossible or difficult to code efficiently.
It may seem that this is not a major problem, since more functions can
always be added later. But that will quickly lead to the same type of
function chaos that other "historically evolved" systems suffer
from. Sometimes new functionality is best implemented by extending the
definition of an existing function, which should however be reflected
by a new name - and at that point compatibility considerations make
the best solution impossible once the core function set is frozen.
To give an example: I need to repeat certain items in an array
a specified number of times, e.g. given
n = array([1,1,2,3])
x = array([2.,1.,6.,3.5])
I want to obtain something like
y = repeat(n,x)
yielding
array([2.,1.,6.,6.,3.5,3.5,3.5])
I don't see any reasonably efficient way to implement the function
repeat() with the current array operations. Supposing that this
operation is generally useful (and my APL/J experience definitely
indicates that it is), it should be implemented in C in the array
module. But a little reflection shows that this is a simple
generalization of the already existing function compress(); indeed
compress() is identical to repeat() as long as the first argument
contains only 0s and 1s. So the most rational design would be to have
repeat() and eliminate compress().
This leaves the question of how to find out which operations are
necessary and which aren't. Apart from practical experience (which
takes a lot of time to accumulate, and much more users than we
currently have), it makes sense to look what other array languages
have. It has been my intention to check whether common APL/J
function can be implemented easily with what we have, but without
knowing the existing function set, this is very difficult!
What I've decided is not important to resolve before the BETA release:
1) static vs. dynamic linking (static linking is fine for now)
2) Contents of Numeric.py (I'm going to rule by fiat here)
3) The exact details of Konrad's print function (Unlikley that changes
here will break anyone's code)
I agree.
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Wed Feb 7 20:52:09 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Wed, 7 Feb 1996 15:52:09 -0500
Subject: [PYTHON MATRIX-SIG] Steadily approaching BETA release
In-Reply-To: <9602071948.AA17725@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199602072052.PAA22142@cyclone.ERE.UMontreal.CA>
So, what's left before a general BETA release?
Note: I'm going to comment on each of these in a seperate follow-up
piece of e-mail in order to try and seperate these two different issues.
1) Type coercion
2) Naming conventions
3) Practical experience
I don't know how much real-life code others have produced with
the current release, but I find that even with the relatively
modest amount of application code I have written, I keep finding
weak spots in the current collection of functions, i.e. operations
that are impossible or difficult to code efficiently.
It may seem that this is not a major problem, since more functions can
always be added later. But that will quickly lead to the same type of
function chaos that other "historically evolved" systems suffer
from. Sometimes new functionality is best implemented by extending the
definition of an existing function, which should however be reflected
by a new name - and at that point compatibility considerations make
the best solution impossible once the core function set is frozen.
To give an example: I need to repeat certain items in an array
a specified number of times, e.g. given
n = array([1,1,2,3])
x = array([2.,1.,6.,3.5])
I want to obtain something like
y = repeat(n,x)
yielding
array([2.,1.,6.,6.,3.5,3.5,3.5])
I don't see any reasonably efficient way to implement the function
repeat() with the current array operations. Supposing that this
operation is generally useful (and my APL/J experience definitely
indicates that it is), it should be implemented in C in the array
module. But a little reflection shows that this is a simple
generalization of the already existing function compress(); indeed
compress() is identical to repeat() as long as the first argument
contains only 0s and 1s. So the most rational design would be to have
repeat() and eliminate compress().
This leaves the question of how to find out which operations are
necessary and which aren't. Apart from practical experience (which
takes a lot of time to accumulate, and much more users than we
currently have), it makes sense to look what other array languages
have. It has been my intention to check whether common APL/J
function can be implemented easily with what we have, but without
knowing the existing function set, this is very difficult!
What I've decided is not important to resolve before the BETA release:
1) static vs. dynamic linking (static linking is fine for now)
2) Contents of Numeric.py (I'm going to rule by fiat here)
3) The exact details of Konrad's print function (Unlikley that changes
here will break anyone's code)
I agree.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Wed Feb 7 21:37:24 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Wed, 7 Feb 96 16:37:24 EST
Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience
Message-ID: <9602072137.AA18068@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Personally, I'm not terribly unhappy with the current naming
conventions and feature set. I agree that they could be better, but I
lack the time to sit down and do a careful redesign. The current
feature set is significantly more powerful than the basic array
operations provided by matlab, the most popular array processing
language that people buy.
Rather than complaining about the lack of a list of the available
functions, David Ascher recently sat down and pulled them out of the C
and python source code (not too hard to do, since I did include doc
strings for everything). Check out his beta tutorial
(http://maigret.cog.brown.edu/python/arraytut.html) for the current
set of names in use. It's still an early draft with the expected
bugs, but its getting close to a fairly complete intro.
I'm going to step out of my current role of moderator for this issue.
If somebody else out there has the time and energy to design a
complete set of functions and methods for the system, go for it. If
you write this up nicely and can convince the people in this SIG that
your feature set makes sense, I'll be happy to implement it (well, so
long as you don't ask me to solve the halting problem). Otherwise,
I'm tired of people asking for me to do things like make a better
distinction between functions and methods. This is just not useful
unless I had more free time than I do.
Sorry for the negative tone, but I've been getting a few too many
questions like "So how's your speech recognition system coming along?"
from my advisor lately and I'm eager to get the array object "out the
door". Bug fixes take up very little of my intellectual energy, but
these issues of naming conventions and function sets use up way too
much.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Wed Feb 7 21:39:08 1996
From: jfulton@usgs.gov (Jim Fulton)
Date: Wed, 7 Feb 1996 16:39:08 -0500
Subject: [PYTHON MATRIX-SIG] New pretty printer
In-Reply-To: hinsenk@ere.umontreal.ca (Konrad HINSEN)
"Re: [PYTHON MATRIX-SIG] New pretty printer" (Feb 7, 12:11pm)
References: <199602071711.MAA07277@cyclone.ERE.UMontreal.CA>
Message-ID: <9602071639.ZM12755@dsdbqvarsa.er.usgs.gov>
On Feb 7, 12:11pm, Konrad HINSEN wrote:
> Subject: Re: [PYTHON MATRIX-SIG] New pretty printer
>
> It makes sense for it to be a property of the file object. But since
> adding properties to those is no fun, I'd suggest to go for a
> parameter is sys. Whoever feels like they can change sys.stdout can
> also change the line width if they feel like it.
>
> OK. Then it makes sense to add another parameter for printing
> precision and use it also for scalars.
I'd rather see precision handled as an array attribute.
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dubois1@llnl.gov Wed Feb 7 21:44:09 1996
From: dubois1@llnl.gov (Paul. Dubois)
Date: Wed, 07 Feb 1996 13:44:09 -0800
Subject: [PYTHON MATRIX-SIG] Type Coercion
References: <9602072018.AA17817@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <31191D29.D42@llnl.gov>
I hold these truths to be self-evident, your argument notwithstanding:
a. A PyFloat is a C double
The fact that some of the Python API lets you convert one to a C
float easily doesn't change this.
b. It is a GOOD THING that you can count on the precision of a PyFloat.
Writing portable code between platforms is easy with Python, since
even on 64 bit machines C doubles are implemented as 64 bit floats. (I
know that isn't for certain but as a practical truth, it is true).
c. It is a GOOD THING not to lose precision.
d. It is a GOOD THING not to get an exception.
The original design functioned perfectly, as far as I was concerned.
The change has been made because of a desire not to GAIN precision. This
is only a concern if you have a concern about space, really. (The time
penalty on most workstations for double vs. float is nowhere near 2).
So this means you must have a humongous amount of data. In which case,
Python is a lousy choice anyway, since it will make lots of temporaries
in the act of expression evaluation, while a nice Fortran routine won't.
In short, I'd argue that it is not a good idea to pervert the matrix
class in Python to satisfy a marginal need.
But the alternative is unclean. The minute you find yourself proposing
that elements of an array are zero-d arrays you should realize the slope
has gotten too slippery.
I return to my previous horror example:
(1./3.) * a_float[i]
vs.
((1./3.) * a_float)[i]
Can you really stand for those to be different? Vectorize a loop and
change the answer?
--
Paul F. Dubois, L-472 (510)-422-5426
Lawrence Livermore National Laboratory FAX (510)-423-9969
Livermore, CA 94550 dubois1@llnl.gov
Consulting: PaulDubois@aol.com
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Wed Feb 7 21:46:08 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Wed, 7 Feb 1996 16:46:08 -0500
Subject: [PYTHON MATRIX-SIG] New pretty printer
In-Reply-To: <9602071639.ZM12755@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov)
Message-ID: <199602072146.QAA24976@cyclone.ERE.UMontreal.CA>
> OK. Then it makes sense to add another parameter for printing
> precision and use it also for scalars.
I'd rather see precision handled as an array attribute.
But it isn't. If precision were an attribute of an object, you would
expect arithmetic etc. to respect it. But what we are discussing is
just a matter of output formatting. To be consistent, you would
also have to require that the radix for integer output be a
property of an object, creating separate types for decimal and
hex numbers.
If precision were an attribute, we would also have to decide what the
precision of the result of an array operation is. This just doesn't
make sense.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Wed Feb 7 21:55:36 1996
From: jfulton@usgs.gov (Jim Fulton)
Date: Wed, 7 Feb 1996 16:55:36 -0500
Subject: [PYTHON MATRIX-SIG] New pretty printer
In-Reply-To: hinsenk@ERE.UMontreal.CA (Konrad HINSEN)
"Re: [PYTHON MATRIX-SIG] New pretty printer" (Feb 7, 4:46pm)
References: <199602072146.QAA24976@cyclone.ERE.UMontreal.CA>
Message-ID: <9602071655.ZM12804@dsdbqvarsa.er.usgs.gov>
On Feb 7, 4:46pm, Konrad HINSEN wrote:
> Subject: Re: [PYTHON MATRIX-SIG] New pretty printer
>
> > OK. Then it makes sense to add another parameter for printing
> > precision and use it also for scalars.
>
> I'd rather see precision handled as an array attribute.
>
> But it isn't. If precision were an attribute of an object, you would
> expect arithmetic etc. to respect it. But what we are discussing is
> just a matter of output formatting. To be consistent, you would
> also have to require that the radix for integer output be a
> property of an object, creating separate types for decimal and
> hex numbers.
>
> If precision were an attribute, we would also have to decide what the
> precision of the result of an array operation is. This just doesn't
> make sense.
Perhaps precision is a poor term. "output precision" is fine with me.
My opinion is based on two ideas:
- The output precision is needed specifically for printing arrays, mainly
because you have lots of data to output.
- Global variables are bad. A correlary is that changes to core things like
sys is bad. I don't mind output width beig added to sys, because it
clearly is not array specific. (Well, maybe I do. :)
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Wed Feb 7 21:56:30 1996
From: jfulton@usgs.gov (Jim Fulton)
Date: Wed, 7 Feb 1996 16:56:30 -0500
Subject: [PYTHON MATRIX-SIG] Type Coercion
In-Reply-To: "Paul. Dubois" <dubois1@llnl.gov>
"Re: [PYTHON MATRIX-SIG] Type Coercion" (Feb 7, 1:44pm)
References: <9602072018.AA17817@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
<31191D29.D42@llnl.gov>
Message-ID: <9602071656.ZM12810@dsdbqvarsa.er.usgs.gov>
I agree.
On Feb 7, 1:44pm, Paul. Dubois wrote:
> Subject: Re: [PYTHON MATRIX-SIG] Type Coercion
> I hold these truths to be self-evident, your argument notwithstanding:
> a. A PyFloat is a C double
> The fact that some of the Python API lets you convert one to a C
> float easily doesn't change this.
>
> b. It is a GOOD THING that you can count on the precision of a PyFloat.
> Writing portable code between platforms is easy with Python, since
> even on 64 bit machines C doubles are implemented as 64 bit floats. (I
> know that isn't for certain but as a practical truth, it is true).
>
> c. It is a GOOD THING not to lose precision.
> d. It is a GOOD THING not to get an exception.
>
> The original design functioned perfectly, as far as I was concerned.
>
> The change has been made because of a desire not to GAIN precision. This
> is only a concern if you have a concern about space, really. (The time
> penalty on most workstations for double vs. float is nowhere near 2).
> So this means you must have a humongous amount of data. In which case,
> Python is a lousy choice anyway, since it will make lots of temporaries
> in the act of expression evaluation, while a nice Fortran routine won't.
>
> In short, I'd argue that it is not a good idea to pervert the matrix
> class in Python to satisfy a marginal need.
>
> But the alternative is unclean. The minute you find yourself proposing
> that elements of an array are zero-d arrays you should realize the slope
> has gotten too slippery.
>
> I return to my previous horror example:
>
> (1./3.) * a_float[i]
> vs.
> ((1./3.) * a_float)[i]
>
> Can you really stand for those to be different? Vectorize a loop and
> change the answer?
> --
> Paul F. Dubois, L-472 (510)-422-5426
> Lawrence Livermore National Laboratory FAX (510)-423-9969
> Livermore, CA 94550 dubois1@llnl.gov
> Consulting: PaulDubois@aol.com
>
> =================
> MATRIX-SIG - SIG on Matrix Math for Python
>
> send messages to: matrix-sig@python.org
> administrivia to: matrix-sig-request@python.org
> =================
>-- End of excerpt from Paul. Dubois
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Wed Feb 7 21:59:44 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Wed, 7 Feb 1996 16:59:44 -0500
Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience
In-Reply-To: <9602072137.AA18068@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199602072159.QAA25799@cyclone.ERE.UMontreal.CA>
lack the time to sit down and do a careful redesign. The current
feature set is significantly more powerful than the basic array
operations provided by matlab, the most popular array processing
language that people buy.
Also the worst one, mostly because it is "historically grown" from a
very restrictive basis (LINPACK).
Rather than complaining about the lack of a list of the available
functions, David Ascher recently sat down and pulled them out of the C
and python source code (not too hard to do, since I did include doc
strings for everything). Check out his beta tutorial
I know, and in fact I have postponed my own attempts waiting for him
to be continue the investigation. I wasn't really complaining, but
providing an excuse for not having done something until now.
If somebody else out there has the time and energy to design a
complete set of functions and methods for the system, go for it. If
I volunteer to do so, but can't promise any deadline. I am also doing
all my work on Python as a side project, so I fully sympathize with
your problems of allocating time to it!
Sorry for the negative tone, but I've been getting a few too many
questions like "So how's your speech recognition system coming along?"
from my advisor lately and I'm eager to get the array object "out the
door". Bug fixes take up very little of my intellectual energy, but
I fully understand that, but I don't see the need to rush. After all,
we are doing something close to language design, which has never
profited from tight deadlines. Rather than rush out a beta version to
the public, I advocate a public alpha release that comes with no
promise about stability of function names etc. We need all the test
users we can get! Why make any commitments without anybody forcing us?
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Wed Feb 7 21:36:58 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Wed, 7 Feb 1996 16:36:58 -0500
Subject: [PYTHON MATRIX-SIG] Type Coercion
In-Reply-To: <9602072018.AA17817@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199602072136.QAA24352@cyclone.ERE.UMontreal.CA>
Python has only one floating point type, call this a PyFloat.
Internally, these numbers store their data using C doubles. However,
the standard C interface to PyFloats allows them to be extracted into
either a C float or a C double (PyArg_ParseTuple), so its not clear
that a PyFloat should really be considered either a float or a double.
The association of PyFloats with C floats exists only in the C
interface, presumably to facilitate interfacing to existing C code. No
part of the standard language makes use of it, and to an average
Python user PyFloat are definitely C doubles.
Three things can happen:
1) Return an array of floats (possible undesired loss of precision)
2) Return an array of doubles (possible undesired gain of precision)
3) Raise an exception (very annoying to the user)
The potential problems in 1) and 2) are also very annoying to the
user!
This particular problem could be eliminated by having a_float[0]
return a 0d array of floats with a single element. Perhaps this is
worth considering.
I don't see any serious problem with that. Most Python code will work
just as well with an array of rank 0 as with a scalar. Only code that
does explicit type checks will have to take care of the
difference. And explicit conversion would always be possible using the
standard conversion functions.
Personally I would prefer to use method 1, combined with the major
change to the existing system that scalars are returned as 0d arrays,
not as python scalars.
Personally I see this as the least desirable option, my preference
being in the order 3) 2) 1). My argument is again the principle
of least surprise: if something unexpected is going to happen,
I prefer it to be
1) an exception
2) a loss in efficiency
3) a loss in accuracy.
in that order.
Let me propose another solution, fully knowing that Guido might
throw a 16-ton weight at me:
Suppose we modify the Python float object by adding a flag "single
precision". This flag would be set if either
1) A constant is created that can be represented in a C float
(such as 0.5).
2) A constant is created with a special suffix like 'S'
(to be used for low-precision constants with no exact
binary representation, such as 0.1, if precision doesn't
matter).
3) When a PyFloat is generated as a rank-0 float array.
4) Possibly as the result of an explicit conversion to a
"low precision scalar", if that turns out to be useful.
The default behaviour of PyFloats would not change at all, and so
there are no compatibility problems. Any result of a scalar operation
would not have the flag set, of course. There is also no significant
speed penalty, since only the creation of a constant becomes more
involved, and that happens at compile time. The only price to pay
would be an increase in the size of a float object.
The only place where the new flag would matter is in coercion to an
array object. It should take care of most the problems of method 2),
which after all is the most common coercion convention in programming
languages. Unwanted promotion to double can still occur in situations
such as
2.*pi*array([1.,2.,3.], Float(32)),
because 2.*pi would be "non-degradable" even if pi was, because
it is the result of a scalar operation. But such standard cases
can be handled by something like
twopi = short_float(2.*pi),
where short_float() is the conversion function mentioned above.
To help pinning down unwanted coercion to double, one could further
offer some option that raises an exception for such operations. This
would have the status of a debugging aid rather than a language
feature and could, for example, be activated by a command line switch.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Wed Feb 7 22:13:56 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Wed, 7 Feb 1996 17:13:56 -0500
Subject: [PYTHON MATRIX-SIG] New pretty printer
In-Reply-To: <9602071655.ZM12804@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov)
Message-ID: <199602072213.RAA26801@cyclone.ERE.UMontreal.CA>
Perhaps precision is a poor term. "output precision" is fine with me.
I guess the name is the least problem... The main question remains: If
precision becomes a property, what is the precision of a+b, if a has
precision 2 and b has precision 7? And why? If this question doesn't make
sense, then output precision should not be an attribute.
My opinion is based on two ideas:
- The output precision is needed specifically for printing arrays, mainly
because you have lots of data to output.
Needed, yes. But it makes sense and could be easily implemented for
scalars as well.
- Global variables are bad. A correlary is that changes to core things like
sys is bad. I don't mind output width beig added to sys, because it
clearly is not array specific. (Well, maybe I do. :)
Line width is just as much array specific, as long as no other module
makes use of it.
I think we are overly dramatizing the "global variable" problem here.
There is very little reason to change this global variable except for
interactive use. Any module that want output with a specific format
would call the output function Numeric.arrayToString() directly and
supply line width and precision as optional arguments. The default
values from the global variables would be used *only* when someone
writes "print a". They should be set only by the end user, never by
any library code.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dubois1@llnl.gov Wed Feb 7 22:40:23 1996
From: dubois1@llnl.gov (Paul. Dubois)
Date: Wed, 07 Feb 1996 14:40:23 -0800
Subject: [PYTHON MATRIX-SIG] printing arrays
References: <199602071711.MAA07277@cyclone.ERE.UMontreal.CA> <9602071639.ZM12755@dsdbqvarsa.er.usgs.gov>
Message-ID: <31192A57.6665@llnl.gov>
Here is my take on this:
>From a purely object-oriented point of view, the width/precision are
attributes of the *array formatter*, ie an object that knows how to
print arrays. At the moment, this gets confounded with the array itself.
The default formatter object could be a class variable (i.e., when an
array is created, it has as its default formatter a single shared
formatter). This means that changing the precision in the shared
formatter would change it across the board.
The discussion has really arisen because array isn't really a class and
the "object" that does the printing in it is a function that doesn't
have any state, so we've started to talk about where to put that state.
In general, while I appreciate Jim Fulton's concern about
compartmentalizing things, I think in practice the idea of a
user-changeable default printing precision is well established in
similar products. "How do I increase the number of decimals" is
certainly a FAQ for Basis users. They don't ask, "How do I increase the
number of decimals for printing THIS array only."
--
Paul F. Dubois, L-472 (510)-422-5426
Lawrence Livermore National Laboratory FAX (510)-423-9969
Livermore, CA 94550 dubois1@llnl.gov
Consulting: PaulDubois@aol.com
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Wed Feb 7 22:52:47 1996
From: jfulton@usgs.gov (Jim Fulton)
Date: Wed, 7 Feb 1996 17:52:47 -0500
Subject: [PYTHON MATRIX-SIG] printing arrays
In-Reply-To: "Paul. Dubois" <dubois1@llnl.gov>
"[PYTHON MATRIX-SIG] printing arrays" (Feb 7, 2:40pm)
References: <199602071711.MAA07277@cyclone.ERE.UMontreal.CA>
<9602071639.ZM12755@dsdbqvarsa.er.usgs.gov>
<31192A57.6665@llnl.gov>
Message-ID: <9602071752.ZM12949@dsdbqvarsa.er.usgs.gov>
On Feb 7, 2:40pm, Paul. Dubois wrote:
> Subject: [PYTHON MATRIX-SIG] printing arrays
> Here is my take on this:
>
> >From a purely object-oriented point of view, the width/precision are
> attributes of the *array formatter*, ie an object that knows how to
> print arrays. At the moment, this gets confounded with the array itself.
> The default formatter object could be a class variable (i.e., when an
> array is created, it has as its default formatter a single shared
> formatter). This means that changing the precision in the shared
> formatter would change it across the board.
>
> The discussion has really arisen because array isn't really a class and
> the "object" that does the printing in it is a function that doesn't
> have any state, so we've started to talk about where to put that state.
>
> In general, while I appreciate Jim Fulton's concern about
> compartmentalizing things, I think in practice the idea of a
> user-changeable default printing precision is well established in
> similar products. "How do I increase the number of decimals" is
> certainly a FAQ for Basis users. They don't ask, "How do I increase the
> number of decimals for printing THIS array only."
OK, I have no problem with a user settable default precision. Why not have a
default set in Numeric and let arrays also have attributes that are initialized
with the default, and can be set by the user?
Actually, my main gripe at this point is that I don't like diddling sys for. I
can live with having global variables set in some array-specific module.
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Wed Feb 7 22:26:00 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Wed, 7 Feb 1996 17:26:00 -0500
Subject: [PYTHON MATRIX-SIG] Type Coercion
In-Reply-To: <31191D29.D42@llnl.gov> (dubois1@llnl.gov)
Message-ID: <199602072226.RAA27524@cyclone.ERE.UMontreal.CA>
The change has been made because of a desire not to GAIN precision. This
is only a concern if you have a concern about space, really. (The time
penalty on most workstations for double vs. float is nowhere near 2).
So this means you must have a humongous amount of data. In which case,
Python is a lousy choice anyway, since it will make lots of temporaries
in the act of expression evaluation, while a nice Fortran routine won't.
You might need floats just for interfacing with existing code. Or for
a big application in which most of the processing happens in C/Fortran
modules. Anyway, if there is a need to have float arrays, then there
is also a need to keep them to float size in a reasonable way.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From da@maigret.cog.brown.edu Wed Feb 7 23:09:29 1996
From: da@maigret.cog.brown.edu (David Ascher)
Date: Wed, 7 Feb 1996 18:09:29 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience
In-Reply-To: <199602072159.QAA25799@cyclone.ERE.UMontreal.CA> from "Konrad HINSEN" at Feb 7, 96 04:59:44 pm
Message-ID: <199602072309.SAA18847@maigret>
> If somebody else out there has the time and energy to design a
> complete set of functions and methods for the system, go for it. If
>
> I volunteer to do so, but can't promise any deadline. I am also doing
> all my work on Python as a side project, so I fully sympathize with
> your problems of allocating time to it!
You know, I think we should setup a support group for people who'se
lives are disrupted by this damn computer language. Hell, my SO would
qualify under that definition. With a special low membership fee for
PhD students, of course.
--da
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dubois1@llnl.gov Wed Feb 7 23:10:40 1996
From: dubois1@llnl.gov (Paul. Dubois)
Date: Wed, 07 Feb 1996 15:10:40 -0800
Subject: [PYTHON MATRIX-SIG] Type Coercion
References: <199602072226.RAA27524@cyclone.ERE.UMontreal.CA>
Message-ID: <31193170.31B3@llnl.gov>
Konrad remarks that you have to "keep" yourself at the lower precision
so you can interact to C/Fortran. This is not strictly correct. You just
have to get yourself *back* first. But:
We have been doing just that lately and in fact you have to do a
conversion of some sort nearly every time if you want to be safe,
anyway. This has to do with the contiguous issue. For example, you have
a C plotting array that wants an array of C floats. You might even have
an array of C floats but in general you don't know it is contiguous. It
might be a slice of something else, or the result of any number of
calculations that destroy contiguity (if that is a word!).
So even if you have made yourself double, you aren't any worse off as
you repack yourself contiguous/float to get to the C routine.
--
Paul F. Dubois, L-472 (510)-422-5426
Lawrence Livermore National Laboratory FAX (510)-423-9969
Livermore, CA 94550 dubois1@llnl.gov
Consulting: PaulDubois@aol.com
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Wed Feb 7 23:10:49 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Wed, 7 Feb 1996 18:10:49 -0500
Subject: [PYTHON MATRIX-SIG] printing arrays
In-Reply-To: <9602071752.ZM12949@dsdbqvarsa.er.usgs.gov> (jfulton@usgs.gov)
Message-ID: <199602072310.SAA01109@cyclone.ERE.UMontreal.CA>
Actually, my main gripe at this point is that I don't like diddling sys for. I
can live with having global variables set in some array-specific module.
I don't care too much whether this goes to Numeric or sys. I still
consider sys more appropriate just because these definitions are not
specific to arrays, but I won't insist.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dubois1@llnl.gov Wed Feb 7 23:12:22 1996
From: dubois1@llnl.gov (Paul. Dubois)
Date: Wed, 07 Feb 1996 15:12:22 -0800
Subject: [PYTHON MATRIX-SIG] [Fwd: Foreign Function Interface Generator available.]
Message-ID: <311931D6.6B1E@llnl.gov>
This is a multi-part message in MIME format.
--------------521F5C6038EE
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
I thought this audience might find this interesting, considering some of
the discussions at Spam III.
--
Paul F. Dubois, L-472 (510)-422-5426
Lawrence Livermore National Laboratory FAX (510)-423-9969
Livermore, CA 94550 dubois1@llnl.gov
Consulting: PaulDubois@aol.com
--------------521F5C6038EE
Content-Type: message/rfc822
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Return-Path: <lth@cs.uoregon.edu>
Received: from icf.llnl.gov by kristen.llnl.gov (5.x/SMI-SVR4)
id AA17149; Wed, 7 Feb 1996 12:28:33 -0800
Received: from pierce.llnl.gov by icf.llnl.gov (4.1/LLNL-1.19)
id AA00439; Wed, 7 Feb 96 12:28:32 PST
Received: by pierce.llnl.gov (8.6.10/LLNL-1.18/llnl.gov-03.95)
id MAA20155; Wed, 7 Feb 1996 12:28:31 -0800
Received: from alpha.xerox.com by pierce.llnl.gov (8.6.10/LLNL-1.18/llnl.gov-03.95)
id MAA20146; Wed, 7 Feb 1996 12:28:29 -0800
Received: from Homer.Parc.Xerox.xns by alpha.xerox.com via XNS id <15450(15)>; Wed, 7 Feb 1996 12:19:11 PST
Received: from skinner.cs.uoregon.edu ([128.223.4.13]) by alpha.xerox.com with SMTP id <15439(2)>; Wed, 7 Feb 1996 12:06:23 PST
Received: from blackrabbit.cs.uoregon.edu by skinner.cs.uoregon.edu with SMTP id AA13990
(5.65/IDA-1.4.2 for ilu@parc.xerox.com); Wed, 7 Feb 96 12:05:22 -0800
Received: from blackrabbit.cs.uoregon.edu (localhost [127.0.0.1]) by blackrabbit.cs.uoregon.edu (8.6.12/8.6.12) with ESMTP id MAA20417; Wed, 7 Feb 1996 12:02:27 -0800
X-Ns-Transport-Id: 0800207424C500CC559B
Date: Wed, 7 Feb 1996 12:02:26 PST
From: Lars Thomas Hansen <lth@cs.uoregon.edu>
Subject: Foreign Function Interface Generator available.
To: fjh%cs.mu.OZ.AU@uunet.uu.net, will@ccs.neu.edu, jcg@cs.cmu.edu,
gurr@snap.med.ge.com, nr@cs.purdue.edu, Geoff.Wyant@east.sun.com,
kalsow@sctc.com, nayeri@gte.com, ilu@parc.xerox.com,
scheme@mc.lcs.mit.edu
Cc: lth@cs.uoregon.edu, malony@cs.uoregon.edu, hersey@cs.uoregon.edu
Message-Id: <199602072002.MAA20417@blackrabbit.cs.uoregon.edu>
Content-Type: text
Content-Length: 901
X-Mozilla-Status: 0001
FFIGEN Release 1 is now available from http://www.cs.uoregon.edu/~lth/ffigen.
FFIGEN (Foreign Function Interface GENerator) is a program suite which
facilitates the writing of translators from C header files to foreign
function interfaces for particular language implementations. It is
based on the lcc C compiler and handles nearly all of ANSI C, including
the preprocessor. (Missing features are bitfields and general type
qualifiers; support for those will be included in a future version).
This release includes the entire front-end, a back-end for Chez Scheme
version 5, and documents which explain the ideas behind FFIGEN, its use,
and how to write of custom back-ends for you language.
This is a preliminary release; it works but is neither full-function nor
polished. In particular, installation is crude.
See the Web page for further details, download information, and so on.
--lars
--------------521F5C6038EE--
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Wed Feb 7 23:21:19 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Wed, 7 Feb 1996 18:21:19 -0500
Subject: [PYTHON MATRIX-SIG] Type Coercion
In-Reply-To: <31193170.31B3@llnl.gov> (dubois1@llnl.gov)
Message-ID: <199602072321.SAA01631@cyclone.ERE.UMontreal.CA>
a C plotting array that wants an array of C floats. You might even have
an array of C floats but in general you don't know it is contiguous. It
might be a slice of something else, or the result of any number of
calculations that destroy contiguity (if that is a word!).
Believe it or not, it is accepted by "spell" on my system ;-)
Contiguity might not be an issue when interfacing to Fortran code,
which can often handle non-contiguous arrays. Anyway, the C-API should
have the utility functions "copy if not contiguous" and "copy if not
sufficiently contiguous to be passed to a Fortran function". There
could be versions that include a cast.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Wed Feb 7 23:30:50 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Wed, 7 Feb 96 18:30:50 EST
Subject: [PYTHON MATRIX-SIG] Type Coercion
In-Reply-To: <199602072321.SAA01631@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca)
Message-ID: <9602072330.AA18989@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
a C plotting array that wants an array of C floats. You might even have
an array of C floats but in general you don't know it is contiguous. It
might be a slice of something else, or the result of any number of
calculations that destroy contiguity (if that is a word!).
Believe it or not, it is accepted by "spell" on my system ;-)
Contiguity might not be an issue when interfacing to Fortran code,
which can often handle non-contiguous arrays. Anyway, the C-API should
have the utility functions "copy if not contiguous" and "copy if not
sufficiently contiguous to be passed to a Fortran function". There
could be versions that include a cast.
Just to clarify the facts on this issue.
The C-API has the standard function PyArray_ContiguousFromObject which
is the standard method of obtaining an array.
If the object is a contiguous PyArray of the same type you desire, no
memory copying occurs. Otherwise there is obviously a copying step
and possibly a cast required.
The two other "standard" ways of obtaining an array from a python
object are
PyArray_CopyFromObject which guarantees you get a copy and can do with
the memory area what you please.
PyArray_FromObject which can give you discontiguous arrays if you want
to deal with them (this is what the internal ofuncs use).
So the facts are that there is in fact a cost proportional to the size
of the array when you pass an array of doubles to one expecting
floats. However, the cast operation is fairly efficient, so I doubt
that this particular case really matters in practice at all.
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Thu Feb 8 00:00:54 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Wed, 7 Feb 1996 19:00:54 -0500
Subject: [PYTHON MATRIX-SIG] Pretty Printer
Message-ID: <199602080000.TAA04035@cyclone.ERE.UMontreal.CA>
And here's another pretty printer. Float and complex formatting is
much improved, but there is still no line wrapping. Precision control
works, however.
Note that due to the problem in compress() complex arrays will still
often be printed with an inappropriate format. It is also still
impossible to print a float array whose elements are all zero. This
will be possible with one of Jim's upcoming bug fixes.
Line width and precision are currently taken from global variables in
sys if these exist and no explicit parameters are given. If the
general opinion favors Numeric, I'll be happy to change this. For now
you should not forget to put an "import sys" in your Numeric.py.
Before I forget: Precision is indicated by an integer specifying
the number of decimal digits after the decimal point. If there
are good arguments for another specification, I'll be happy to
change this.
I am now quite happy with formatting of floats and complex numbers,
although I admit that I haven't done extensive tests. I hope
there won't be complaints about speed or lack thereof.
I'd appreciate any feedback on how you like the float formatting. I
now remove all trailing zeros, although Python scalars get printed
with at least one. Would you prefer full compatibility in this case?
Or would you prefer to have no decimal point if there are no
fractional digits?
And now the code:
---------------------------------------------------------------------------
import sys
def arrayToString(a, max_line_width = None, precision = None):
if not max_line_width:
try:
max_line_width = sys.output_line_width
except AttributeError:
max_line_width = 77
if not precision:
try:
precision = sys.float_output_precision
except AttributeError:
precision = 8
if a.contiguous():
data = a.reshape(None)
else:
data = a.copy().reshape(None)
type = a.typecode()
items_per_line = a.shape[-1]
if type == 'b' or type == '1' or type == 's' or type == 'i' \
or type == 'l':
max_str_len = max(len(str(maximum.reduce(data))),
len(str(minimum.reduce(data))))
format = '%' + str(max_str_len) + 'd'
item_length = max_str_len+1
format_function = lambda x, f = format: _formatInteger(x, f)
elif type == 'f' or type == 'd':
format, item_length = _floatFormat(data, precision)
format_function = lambda x, f = format: _formatFloat(x, f)
elif type == 'F' or type == 'D':
real_format, real_item_length = _floatFormat(data.real, precision,
sign=0)
imag_format, imag_item_length = _floatFormat(data.imaginary, precision,
sign=1)
item_length = real_item_length + imag_item_length + 2
format_function = lambda x, f1 = real_format, f2 = imag_format: \
_formatComplex(x, f1, f2)
elif type == 'c':
format = '%s'
item_length = 1
format_function = lambda x, f = format: _formatCharacter(x, f)
else: # nothing for 'O'
return str(a)
line_width = item_length*items_per_line - (type != 'c')
if line_width > max_line_width:
pass
return _arrayToString(a, format_function, len(a.shape), line_width)[:-1]
def _floatFormat(data, precision, sign = 0):
exp_format = 0
non_zero = abs(compress(data.notEqual(0), data))
if len(non_zero) == 0:
max_val = 0.
min_val = 0.
else:
max_val = maximum.reduce(non_zero)
min_val = minimum.reduce(non_zero)
if max_val >= 1.e12 or min_val < 0.0001 or max_val/min_val > 1000.:
exp_format = 1
if exp_format:
large_exponent = 0 < min_val < 1e-99 or max_val >= 1e100
max_str_len = 8 + precision + large_exponent
if sign: format = '%+'
else: format = '%'
format = format + str(max_str_len) + '.' + str(precision) + 'e'
if large_exponent: format = format + '3'
item_length = max_str_len + 1
else:
format = '%.' + str(precision) + 'f'
precision = min(precision, apply(max, tuple(map(lambda x, p=precision,
f=format: _digits(x,p,f),
data))))
max_str_len = len(str(int(max_val))) + precision + 2
if sign: format = '%#+'
else: format = '%#'
format = format + str(max_str_len) + '.' + str(precision) + 'f'
item_length = max_str_len + 1
return (format, item_length)
def _digits(x, precision, format):
s = format % x
zeros = len(s)
while s[zeros-1] == '0': zeros = zeros-1
return precision-len(s)+zeros
def _arrayToString(a, format_function, rank, line_width):
if rank == 0:
return str(a[0])
elif rank == 1:
s = ''
for i in range(a.shape[0]):
s = s + format_function(a[i])
if s[-1] == ' ': s = s[:-1]
s = s + '\n'
else:
s = ''
for i in range(a.shape[0]-1):
s = s + _arrayToString(a[i], format_function, rank-1, line_width)
if rank == 3:
s = s + '\n'
elif rank > 3:
s = s + (rank-3)*(line_width*'-'+'\n')
s = s + _arrayToString(a[a.shape[0]-1], format_function,
rank-1, line_width)
return s
def _formatInteger(x, format):
return format % x + ' '
def _formatFloat(x, format, strip_zeros = 1):
if format[-1] == '3':
format = format[:-1]
s = format % x
third = s[-3]
if third == '+' or third == '-':
s = s[1:-2] + '0' + s[-2:]
elif format[-1] == 'f':
s = format % x
if strip_zeros:
zeros = len(s)
while s[zeros-1] == '0': zeros = zeros-1
s = s[:zeros] + (len(s)-zeros)*' '
else:
s = format % x
return s + ' '
def _formatComplex(x, real_format, imag_format):
r = _formatFloat(x.real, real_format)[:-1]
i = _formatFloat(x.imag, imag_format, 0)[:-1]
spaces = 0
while r[spaces] == ' ': spaces = spaces + 1
r = spaces*' ' + '(' + r[spaces:]
if imag_format[-1] == 'f':
zeros = len(i)
while i[zeros-1] == '0': zeros = zeros-1
i = i[:zeros] + 'j)' + (len(i)-zeros)*' '
else:
i = i + 'j)'
return r + i + ' '
def _formatCharacter(x, format):
return format % x
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From stoll@atr-sw.atr.co.jp Thu Feb 8 03:11:01 1996
From: stoll@atr-sw.atr.co.jp (Perry A. Stoll)
Date: Thu, 08 Feb 1996 12:11:01 +0900
Subject: [PYTHON MATRIX-SIG] Pretty Printer
In-Reply-To: Your message of "Wed, 07 Feb 1996 19:00:54 JST"
References: <199602080000.TAA04035@cyclone.ERE.UMontreal.CA>
Message-ID: <199602080311.MAA03526@ciris21.atr-sw.atr.co.jp>
As Konrad's pretty printer continues to evolve (please!) it seems to
be taking over most of the Numeric module. How about we make it a
separate file?
I've done the following in my Python:
1) Save his pretty printer as a separate python file ArrayFormatter.py
2) Add "import Numeric" to the top of ArrayFormatter.py file
and prepend "Numeric." to the compress, maximum, and minimum calls.
3) Put the following near the end of Numeric.py:
from ArrayFormatter import arrayToString
# Set the official array printing function
set_print_function(arrayToString)
Seems like a nice division.
-Perry
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From fredrik_lundh@ivab.se Thu Feb 8 08:04:22 1996
From: fredrik_lundh@ivab.se (Fredrik Lundh)
Date: Thu, 8 Feb 1996 09:04:22 +0100
Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience
In-Reply-To: <199602072159.QAA25799@cyclone.ERE.UMontreal.CA>
(hinsenk@ere.umontreal.ca)
Message-ID: <9602080804.AA26470@arnold.image.ivab.se>
> Rather than rush out a beta version to the public, I advocate a
> public alpha release that comes with no promise about stability of
> function names etc. We need all the test users we can get! Why make
> any commitments without anybody forcing us?
I support this view, if that's any help :-)
/F
BTW: what about work on standard packages?; I'd very much like some
linear algebra stuff, for example to invert matrices (but don't know
enough about it to contribute). In addition, I'd would like to use
the matrix extension to do some of the serious processing stuff in my
forthcoming Python Imaging library, like FFTs and 2D convolutions.
Any ideas on how to achieve this?
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From fredrik_lundh@ivab.se Thu Feb 8 08:04:22 1996
From: fredrik_lundh@ivab.se (Fredrik Lundh)
Date: Thu, 8 Feb 1996 09:04:22 +0100
Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience
In-Reply-To: <199602072159.QAA25799@cyclone.ERE.UMontreal.CA>
(hinsenk@ere.umontreal.ca)
Message-ID: <9602080804.AA26470@arnold.image.ivab.se>
> Rather than rush out a beta version to the public, I advocate a
> public alpha release that comes with no promise about stability of
> function names etc. We need all the test users we can get! Why make
> any commitments without anybody forcing us?
I support this view, if that's any help :-)
/F
BTW: what about work on standard packages?; I'd very much like some
linear algebra stuff, for example to invert matrices (but don't know
enough about it to contribute). In addition, I'd would like to use
the matrix extension to do some of the serious processing stuff in my
forthcoming Python Imaging library, like FFTs and 2D convolutions.
Any ideas on how to achieve this?
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From stoll@atr-sw.atr.co.jp Thu Feb 8 09:22:12 1996
From: stoll@atr-sw.atr.co.jp (Perry A. Stoll)
Date: Thu, 08 Feb 1996 18:22:12 +0900
Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience
In-Reply-To: Your message of "Thu, 08 Feb 1996 09:04:22 JST"
References: <9602080804.AA26470@arnold.image.ivab.se>
Message-ID: <199602080922.SAA04171@ciris21.atr-sw.atr.co.jp>
>>>>> "FL" == Fredrik Lundh <Fredrik_Lundh@ivab.se> writes:
FL> would like to use the matrix extension to do some of the
FL> serious processing stuff in my forthcoming Python Imaging
FL> library, like FFTs and 2D convolutions. Any ideas on how to
FL> achieve this?
I'm working on something similar. I've got various FFT packages off
the net - I'll try to interface them, but I don't have time right
now. If you are interested in doing it, let me know.
In the mean time, for those wanting to play take a look at the file
attached at the end. It's not the stuff legends are made of, but it
should work.
-Perry
--------------------- cut here ------------------------------------
""" conv2 - support convolution of a 2 dimensional matrix by a 2
dimensional kernel.
This routine was inspired by the matlab(c) version of conv2.
Sample usage:
>>> import conv2
# a sobel edge finding kernel
>>> sobel = Numeric.array([[1.0 ,2, 1],[0, 0, 0],[-1, -2, -1]])
# a test array
>>> A = Numeric.zeros(10,10)
>>> S = Numeric.Slice(2,8)
>>> A[(S,S)] = 1
# do the convolution
>>> conv2.conv2(A,sobel)
"""
# Two dimension convolution for the Python Numeric package.
# Should be redone in C.
#
# questions to:
# Perry A. Stoll
# <stoll@atr-sw.atr.co.jp> or <pas@lems.brown.edu>
import Numeric,umath
class AddSlice(Numeric.Slice):
def __init__(self, start=0, stop=None, step=1):
Numeric.Slice.__init__(self,start,stop, step)
def __add__(self,x):
newslice = AddSlice(self.start,self.stop,self.step)
newslice.start = newslice.start + x
if newslice.stop != None:
newslice.stop = newslice.stop + x
return newslice
__radd__ = __add__
class IncSlice(Numeric.Slice):
def __init__(self, start=0, stop=None, step=1):
Numeric.Slice.__init__(self,start,stop,step)
def inc(self):
self.start = self.start + 1
if self.stop != None:
self.stop = self.stop + 1
def conv2(arg1,arg2,shape='full'):
""" C = conv2(A, B) performs the 2-D convolution of matrices A and B.
If [ya,xa] == A.shape and [yb,xb] == B.shape, then
C.shape == [ya+yb-1,xa+xb-1].
"""
if (len(arg1.shape) != 2) or (len(arg2.shape) !=2):
raise TypeError, "both arguments must be 2 dimensional"
# we always want to use the small matrix as arg2,
# so swap args if necessary
swapped = 0
Nmr = Numeric.multiply.reduce
size1,size2 = Nmr(arg1.shape),Nmr(arg2.shape)
if (size2 > size1):
swapped = 1
arg1,arg2 = arg2,arg1
# unpack shape tuples
(y1,x1),(y2,x2) = arg1.shape,arg2.shape
# create output matrix
out = Numeric.zeros(y2+y1-1,x2+x1-1)
cols,rows = IncSlice(0,x2),IncSlice(0,y2)
for j in range(0,y1):
rows.start,rows.stop = 0,y2
for i in range(0,x1):
w = arg1[j,i]
if w != 0:
out[cols,rows] = out[cols,rows] + w*arg2
rows.inc()
cols.inc()
# not sure we need to do this...
#if swapped:
#arg1,arg2 = arg2,arg1
if (shape == 'full'):
pass
elif (shape == 'same'):
rows = int(umath.floor(x2/2.0)) + AddSlice(0,x1)
cols = int(umath.floor(y2/2.0)) + AddSlice(0,y1)
out = out[cols,rows]
elif (shape == 'valid'):
rows = x2 -1 + AddSlice(0,x1-x2 + 1)
cols = y2 -1 + AddSlice(0,y1-y2 + 1)
out = out[cols,rows]
return out
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From mclay@eeel.nist.gov Thu Feb 8 14:30:23 1996
From: mclay@eeel.nist.gov (Michael McLay)
Date: Thu, 8 Feb 96 09:30:23 EST
Subject: [PYTHON MATRIX-SIG] Type Names
In-Reply-To: <31191D29.D42@llnl.gov>
Message-ID: <9602081430.AA17418@acdc.eeel.nist.gov>
Paul. Dubois writes:
> I hold these truths to be self-evident, your argument notwithstanding:
> a. A PyFloat is a C double
> The fact that some of the Python API lets you convert one to a C
> float easily doesn't change this.
>
> b. It is a GOOD THING that you can count on the precision of a PyFloat.
> Writing portable code between platforms is easy with Python, since
> even on 64 bit machines C doubles are implemented as 64 bit floats. (I
> know that isn't for certain but as a practical truth, it is true).
The idea of naming the sizes of numbers "double" and "float" in a
programming language is analogous to continuing the use the U.S. units
of measure. If you're familiar with the units it's not a problem, but
consider the task of introducing the convention to someone new to
programming. Why does the naming convention have to be obscure? To
get a sense of the nonsense of this naming convention, answer the
following question. How many gils are in a gallon? (The first
questions you need to ask of course is whether I mean an Imperial
gallon or a U.S. gallon.)
What do you think of the idea of explicitly naming the numeric types
to reflect the actual machine size of the number? This would make the
size of an object perfectly clear. It would also eliminate all future
problems with portability of numbers between machine architectures.
It won't break existing code since the old names can still be used as
aliases for the new names. Here's all the names that would be needed.
float32
float64 (alias is float)
float128
int8
int16
int32 (alias for int)
int64
int128
The advantages are:
1) It would make the size of a number obvious thereby eliminating
one more source of stupid programming errors.
2) It would eliminate one more thing that must be learned and
remembered by rote.
3) It will be easy for new users to grasp and remember the convention.
Some of the disadvantages are:
1) It's something new.
2) It may require introducing some new keywords. (This probably
isn't necessary.)
3) It isn't an "elegant" solution.
4) For individuals who already know what a float is, and who have
a talent for remembering minute details, the change may seem
gratuitous.
While an "elegant", theoretical solution, such as Ada's ability to
declare types for numbers of any size, may be possible, an
implementation that allowed this would be slow for any size number
other than those supported directly in hardware. (With the exception
of float128, the type names listed above are close to being
universally supported in modern hardware.) Consequently, these are
probably the only numbers that will be of interest for number
crunching applications. For problems where speed isn't as important
and more control over the calculation is needed then the application
could use an arbitrary length numbers class and add size constraints
as needed.
Isn't it time to drop a convention that was introduced when machine words
varied considerably from system to system?
Michael
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jfulton@usgs.gov Thu Feb 8 14:40:39 1996
From: jfulton@usgs.gov (Jim Fulton)
Date: Thu, 8 Feb 1996 09:40:39 -0500
Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience
In-Reply-To: Fredrik Lundh <Fredrik_Lundh@ivab.se>
"Re: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience" (Feb 8, 9:04am)
References: <9602080804.AA26470@arnold.image.ivab.se>
Message-ID: <9602080940.ZM13456@dsdbqvarsa.er.usgs.gov>
On Feb 8, 9:04am, Fredrik Lundh wrote:
> Subject: Re: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experien
>
> > Rather than rush out a beta version to the public, I advocate a
> > public alpha release that comes with no promise about stability of
> > function names etc. We need all the test users we can get! Why make
> > any commitments without anybody forcing us?
>
> I support this view, if that's any help :-)
Me too.
Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Thu Feb 8 14:53:24 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Thu, 8 Feb 1996 09:53:24 -0500
Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience
In-Reply-To: <9602080804.AA26470@arnold.image.ivab.se> (message from Fredrik Lundh on Thu, 8 Feb 1996 09:04:22 +0100)
Message-ID: <199602081453.JAA19532@cyclone.ERE.UMontreal.CA>
BTW: what about work on standard packages?; I'd very much like some
linear algebra stuff, for example to invert matrices (but don't know
enough about it to contribute). In addition, I'd would like to use
I agree that this would be useful, both directly and indirectly by
providing a real-life example of a C or Fortran extension.
Such extensions should also be considered in the big naming debate. A
"everything in methods" strategy won't work, because a module can't
define methods for objects implemented in another module. So all
linear-algebra (or other application-specific) operations have to be
implemented as functions. This suggests that built-in functions of the
array module should also be functions unless there is some clear
semantic difference.
A difficult decision for linear algebra is which library to use. My
experience is limited to Fortran libraries, of which I had the best
results with LAPACK. There is a C version of LAPACK, which however I
have never used. There are also a couple of LA libraries directly
designed for C, rather than being translations from Fortran.
It would certainly be easiest to use a C library, which eliminates all
the Fortran interfacing machine dependencies and doesn't force users
to have a Fortran compiler. On the other hand, Fortran libraries are
better tested and on some systems (such as SGI) the performance
difference is enormous.
Any comments? Does anyone have experience with a C library for linear
algebra?
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From da@maigret.cog.brown.edu Thu Feb 8 15:31:38 1996
From: da@maigret.cog.brown.edu (David Ascher)
Date: Thu, 8 Feb 1996 10:31:38 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience
In-Reply-To: <199602080922.SAA04171@ciris21.atr-sw.atr.co.jp> from "Perry A. Stoll" at Feb 8, 96 06:22:12 pm
Message-ID: <199602081531.KAA20717@maigret>
> I'm working on something similar. I've got various FFT packages off
> the net - I'll try to interface them, but I don't have time right
> now. If you are interested in doing it, let me know.
Maybe we should start a subsig for linalg, one for fft/image
processing?
> In the mean time, for those wanting to play take a look at the file
> attached at the end. It's not the stuff legends are made of, but it
> should work.
I did something similar in Python, and then redid it in C. The C
version was orders of magnitude faster, not surprisingly. I don't know
enough about such things to write really efficient code for this sort of
stuff, though.
--david
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From fredrik_lundh@ivab.se Thu Feb 8 17:02:45 1996
From: fredrik_lundh@ivab.se (Fredrik Lundh)
Date: Thu, 8 Feb 1996 18:02:45 +0100
Subject: [PYTHON MATRIX-SIG] Re: [PYTHON IMAGE-SIG???]
In-Reply-To: <199602081526.KAA20669@maigret> (da@maigret.cog.brown.edu)
Message-ID: <9602081702.AA32049@arnold.image.ivab.se>
> Maybe we should start a subsig for linalg, one for fft/image
> processing?
Why not an Image SIG, targeted for graphics, publishing, GIS and
good ole "scientific image processing" users...
I've almost completed the first release of an image handling and
processing library for Python. This release focuses on file formats
and basic operations (see the attached "preannouncement"). The design
is to some extent ready for more serious image processing (including
support for 32-bit integer and floating point images), and I would
very much like to interface (not merge) this with the matrix package
to avoid duplicating stuff like FFTs etc.
I've also got ok from my bosses to distribute the library for free
use, and to spend some time on promoting it :-)
What do you think?
Regards/F
====================================================================
Fredrik Lundh | mail to fredrik_lundh@ivab.se
IV Image Systems AB | or call +46 13 200100
Teknikringen 9 | or fax to +46 13 214897
583 30 LINKOPING SWEDEN |
====================================================================
--------------------------------------------------------
Preannouncement: The Python Imaging Library, Release 0.1
--------------------------------------------------------
The Python Imaging Library adds an image object to your Python
interpreter. You can load image objects from a variety of file
formats, and apply a rich set of image operations to them.
Features:
+ Identify and extract image characteristics from a large set of image
file formats.
+ Read and write PPM, BMP, GIF, and JPEG files. Supports progressive
reading of all formats through ImageIO objects.
+ BW, Palette, RGB, RGBA and CMYK pixel formats.
+ Crop, cut/paste, convert between pixel formats.
+ Image geometry: resize, rotate, transpose, user defined affine
transform. Nearest neighbour only.
+ Image enhancement: blur, contour, detail, edge enhance, emboss,
find edges, smooth, sharpen, and user defined 3x3 and 5x5 integer
kernels. User defined point transforms.
+ Image analysis: histogram, global statistics.
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From fredrik_lundh@ivab.se Thu Feb 8 17:02:45 1996
From: fredrik_lundh@ivab.se (Fredrik Lundh)
Date: Thu, 8 Feb 1996 18:02:45 +0100
Subject: [PYTHON MATRIX-SIG] Re: [PYTHON IMAGE-SIG???]
In-Reply-To: <199602081526.KAA20669@maigret> (da@maigret.cog.brown.edu)
Message-ID: <9602081702.AA32049@arnold.image.ivab.se>
> Maybe we should start a subsig for linalg, one for fft/image
> processing?
Why not an Image SIG, targeted for graphics, publishing, GIS and
good ole "scientific image processing" users...
I've almost completed the first release of an image handling and
processing library for Python. This release focuses on file formats
and basic operations (see the attached "preannouncement"). The design
is to some extent ready for more serious image processing (including
support for 32-bit integer and floating point images), and I would
very much like to interface (not merge) this with the matrix package
to avoid duplicating stuff like FFTs etc.
I've also got ok from my bosses to distribute the library for free
use, and to spend some time on promoting it :-)
What do you think?
Regards/F
====================================================================
Fredrik Lundh | mail to fredrik_lundh@ivab.se
IV Image Systems AB | or call +46 13 200100
Teknikringen 9 | or fax to +46 13 214897
583 30 LINKOPING SWEDEN |
====================================================================
--------------------------------------------------------
Preannouncement: The Python Imaging Library, Release 0.1
--------------------------------------------------------
The Python Imaging Library adds an image object to your Python
interpreter. You can load image objects from a variety of file
formats, and apply a rich set of image operations to them.
Features:
+ Identify and extract image characteristics from a large set of image
file formats.
+ Read and write PPM, BMP, GIF, and JPEG files. Supports progressive
reading of all formats through ImageIO objects.
+ BW, Palette, RGB, RGBA and CMYK pixel formats.
+ Crop, cut/paste, convert between pixel formats.
+ Image geometry: resize, rotate, transpose, user defined affine
transform. Nearest neighbour only.
+ Image enhancement: blur, contour, detail, edge enhance, emboss,
find edges, smooth, sharpen, and user defined 3x3 and 5x5 integer
kernels. User defined point transforms.
+ Image analysis: histogram, global statistics.
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Thu Feb 8 17:33:45 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Thu, 8 Feb 1996 12:33:45 -0500
Subject: [PYTHON MATRIX-SIG] Type Names
In-Reply-To: <9602081430.AA17418@acdc.eeel.nist.gov> (message from Michael McLay on Thu, 8 Feb 96 09:30:23 EST)
Message-ID: <199602081733.MAA02062@cyclone.ERE.UMontreal.CA>
of measure. If you're familiar with the units it's not a problem, but
consider the task of introducing the convention to someone new to
programming. Why does the naming convention have to be obscure? To
Because we want newcomers to appreciate that we are doing something
very complicated. Isn't that the origin of most technical terms?
following question. How many gils are in a gallon? (The first
questions you need to ask of course is whether I mean an Imperial
gallon or a U.S. gallon.)
Fortunately Canada has changed to the metric system long before I
arrived here!
What do you think of the idea of explicitly naming the numeric types
to reflect the actual machine size of the number? This would make the
size of an object perfectly clear. It would also eliminate all future
I have nothing against this, but I wonder where these names would
actually be used in Python. For the array constructors, I am very
happy with the current system of "type constructors" as implemented
in Precision.py. It would be nice to have the same for output
instead of typecodes, of course, but we are not really adding all
these things as new types to Python.
While an "elegant", theoretical solution, such as Ada's ability to
declare types for numbers of any size, may be possible, an
implementation that allowed this would be slow for any size number
other than those supported directly in hardware. (With the exception
The current scheme in Precision.py of picking always the next larger
size available in hardware seems like a good compromise to me.
crunching applications. For problems where speed isn't as important
and more control over the calculation is needed then the application
could use an arbitrary length numbers class and add size constraints
as needed.
Arbitrary size floats would indeed be a nice addition to Python...
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dubois1@llnl.gov Thu Feb 8 17:56:50 1996
From: dubois1@llnl.gov (Paul. Dubois)
Date: Thu, 08 Feb 1996 09:56:50 -0800
Subject: [PYTHON MATRIX-SIG] Re: ffts
References: <9602081702.AA32049@arnold.image.ivab.se>
Message-ID: <311A3962.5818@llnl.gov>
We have an FFT package in Basis which in the course of time will be
converted to a Python package. If anybody wants to jump the gun, get the
Basis 11.2 distribution from ftp-icf.llnl.gov/pub/basis, and look in
/dist/library/fft. Maybe a little FIDL will do it?
--
Paul F. Dubois, L-472 (510)-422-5426
Lawrence Livermore National Laboratory FAX (510)-423-9969
Livermore, CA 94550 dubois1@llnl.gov
Consulting: PaulDubois@aol.com
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From fredrik_lundh@ivab.se Thu Feb 8 22:15:43 1996
From: fredrik_lundh@ivab.se (Fredrik Lundh)
Date: Thu, 8 Feb 1996 23:15:43 +0100
Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience
In-Reply-To: <199602081453.JAA19532@cyclone.ERE.UMontreal.CA>
(hinsenk@ere.umontreal.ca)
Message-ID: <9602082215.AA01433@arnold.image.ivab.se>
Konrad wrote:
> A "everything in methods" strategy won't work, because a module
> can't define methods for objects implemented in another module.
Try this:
class foo:
def spam(self):
print "spam"
bar = foo()
def egg(self):
print "egg"
foo.egg = egg
bar.spam()
bar.egg()
Yes, it's weird :-)
/F
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Thu Feb 8 22:30:50 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Thu, 8 Feb 1996 17:30:50 -0500
Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience
In-Reply-To: <9602082215.AA01433@arnold.image.ivab.se> (message from Fredrik Lundh on Thu, 8 Feb 1996 23:15:43 +0100)
Message-ID: <199602082230.RAA22293@cyclone.ERE.UMontreal.CA>
Konrad wrote:
> A "everything in methods" strategy won't work, because a module
> can't define methods for objects implemented in another module.
Try this:
class foo:
def spam(self):
print "spam"
...
You are right, I should have specified "C module". For Python classes
this is possible, although not a particularly good idea from the point
of view of modularity. Anyway, arrays are implemented in C...
A possible solution would be to require extensions to use a Python
wrapper like UserArray and subclass it before adding new methods. In
addition to the loss of efficiency that this causes, it would also
make our system more complicated for the casual user.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From fredrik_lundh@ivab.se Thu Feb 8 22:41:09 1996
From: fredrik_lundh@ivab.se (Fredrik Lundh)
Date: Thu, 8 Feb 1996 23:41:09 +0100
Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience
In-Reply-To: <199602082230.RAA22293@cyclone.ERE.UMontreal.CA>
(hinsenk@ERE.UMontreal.CA)
Message-ID: <9602082241.AA01587@arnold.image.ivab.se>
> You are right, I should have specified "C module". For Python
> classes this is possible, although not a particularly good idea from
> the point of view of modularity.
And I completely agree. Finding the perfect balance between methods
and functions can sometimes be tricky, though.
/F
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Thu Feb 8 23:49:21 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Thu, 8 Feb 1996 18:49:21 -0500
Subject: [PYTHON MATRIX-SIG] Final (!?) pretty printer
Message-ID: <199602082349.SAA27150@cyclone.ERE.UMontreal.CA>
Here it is: the fully functional pretty printer for Python arrays.
Unless there are bugs to be fixed or complaints or requests for
reasonable new features, I'd like to leave it at this stage. Line
wrapping is now implemented and respects the maximum line width
whenever possible, but it still puts at least one element on each
line.
Note that continuation lines are indented by at least 6 characters
in such a way as to make sure that their items do *not* align with
the main lines. This makes visual parsing easier.
And now, as you expected, the code:
def arrayToString(a, max_line_width = None, precision = None):
if not max_line_width:
try:
max_line_width = sys.output_line_width
except AttributeError:
max_line_width = 77
if not precision:
try:
precision = sys.float_output_precision
except AttributeError:
precision = 8
if a.contiguous():
data = a.reshape(None)
else:
data = a.copy().reshape(None)
type = a.typecode()
items_per_line = a.shape[-1]
if type == 'b' or type == '1' or type == 's' or type == 'i' \
or type == 'l':
max_str_len = max(len(str(maximum.reduce(data))),
len(str(minimum.reduce(data))))
format = '%' + str(max_str_len) + 'd'
item_length = max_str_len+1
format_function = lambda x, f = format: _formatInteger(x, f)
elif type == 'f' or type == 'd':
format, item_length = _floatFormat(data, precision)
format_function = lambda x, f = format: _formatFloat(x, f)
elif type == 'F' or type == 'D':
real_format, real_item_length = _floatFormat(data.real, precision,
sign=0)
imag_format, imag_item_length = _floatFormat(data.imaginary, precision,
sign=1)
item_length = real_item_length + imag_item_length + 2
format_function = lambda x, f1 = real_format, f2 = imag_format: \
_formatComplex(x, f1, f2)
elif type == 'c':
format = '%s'
item_length = 1
format_function = lambda x, f = format: _formatCharacter(x, f)
else: # give up on 'O'
return str(a)
final_spaces = (type != 'c')
line_width = item_length*items_per_line - final_spaces
if line_width > max_line_width:
indent = 6
if indent == item_length:
indent = 8
items_first = (max_line_width+final_spaces)/item_length
if items_first < 1: items_first = 1
items_continuation = (max_line_width+final_spaces-indent)/item_length
if items_continuation < 1: items_continuation = 1
line_width = max(item_length*items_first,
item_length*items_continuation+indent) - final_spaces
number_of_lines = 1 + (items_per_line-items_first +
items_continuation-1)/items_continuation
line_format = (number_of_lines, items_first, items_continuation,
indent, line_width)
else:
line_format = (1, items_per_line, 0, 0, line_width)
return _arrayToString(a, format_function, len(a.shape), line_format)[:-1]
def _floatFormat(data, precision, sign = 0):
exp_format = 0
non_zero = abs(compress(data.notEqual(0), data))
if len(non_zero) == 0:
max_val = 0.
min_val = 0.
else:
max_val = maximum.reduce(non_zero)
min_val = minimum.reduce(non_zero)
if max_val >= 1.e12 or min_val < 0.0001 or max_val/min_val > 1000.:
exp_format = 1
if exp_format:
large_exponent = 0 < min_val < 1e-99 or max_val >= 1e100
max_str_len = 8 + precision + large_exponent
if sign: format = '%+'
else: format = '%'
format = format + str(max_str_len) + '.' + str(precision) + 'e'
if large_exponent: format = format + '3'
item_length = max_str_len + 1
else:
format = '%.' + str(precision) + 'f'
precision = min(precision, apply(max, tuple(map(lambda x, p=precision,
f=format: _digits(x,p,f),
data))))
max_str_len = len(str(int(max_val))) + precision + 2
if sign: format = '%#+'
else: format = '%#'
format = format + str(max_str_len) + '.' + str(precision) + 'f'
item_length = max_str_len + 1
return (format, item_length)
def _digits(x, precision, format):
s = format % x
zeros = len(s)
while s[zeros-1] == '0': zeros = zeros-1
return precision-len(s)+zeros
def _arrayToString(a, format_function, rank, line_format):
if rank == 0:
return str(a[0])
elif rank == 1:
s = ''
items = line_format[1]
indent = 0
index = 0
for j in range(line_format[0]):
s = s + indent * ' '
for i in range(items):
s = s + format_function(a[index])
index = index + 1
if index == a.shape[0]: break
if s[-1] == ' ': s = s[:-1]
s = s + '\n'
items = line_format[2]
indent = line_format[3]
else:
s = ''
for i in range(a.shape[0]-1):
s = s + _arrayToString(a[i], format_function, rank-1, line_format)
if rank == 3:
s = s + '\n'
elif rank > 3:
s = s + (rank-3)*(line_format[4]*'-'+'\n')
s = s + _arrayToString(a[a.shape[0]-1], format_function,
rank-1, line_format)
return s
def _formatInteger(x, format):
return format % x + ' '
def _formatFloat(x, format, strip_zeros = 1):
if format[-1] == '3':
format = format[:-1]
s = format % x
third = s[-3]
if third == '+' or third == '-':
s = s[1:-2] + '0' + s[-2:]
elif format[-1] == 'f':
s = format % x
if strip_zeros:
zeros = len(s)
while s[zeros-1] == '0': zeros = zeros-1
s = s[:zeros] + (len(s)-zeros)*' '
else:
s = format % x
return s + ' '
def _formatComplex(x, real_format, imag_format):
r = _formatFloat(x.real, real_format)[:-1]
i = _formatFloat(x.imag, imag_format, 0)[:-1]
spaces = 0
while r[spaces] == ' ': spaces = spaces + 1
r = spaces*' ' + '(' + r[spaces:]
if imag_format[-1] == 'f':
zeros = len(i)
while i[zeros-1] == '0': zeros = zeros-1
i = i[:zeros] + 'j)' + (len(i)-zeros)*' '
else:
i = i + 'j)'
return r + i + ' '
def _formatCharacter(x, format):
return format % x
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From et@appl-math.tu-muenchen.de Sat Feb 10 21:49:25 1996
From: et@appl-math.tu-muenchen.de (Thomas Schwaller)
Date: Sat, 10 Feb 1996 22:49:25 +0100
Subject: [PYTHON MATRIX-SIG] plmodule, openglmodule and delaunaymodule (Version 0.33)
Message-ID: <9602102249.ZM11999@hamster.appl-math.tu-muenchen.de>
Hi all,
just uploaded
plmodule-0.33.c.gz
openglmodule-0.33.c.gz
and the experimental:
delaunaymodule-0.33.c.gz for computing 3D Delaunay Triangulations.
Publish or Perish! :-)
Tom
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Sun Feb 11 15:33:43 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Sun, 11 Feb 1996 10:33:43 -0500
Subject: [PYTHON MATRIX-SIG] Naming Conventions And Practical Experience
In-Reply-To: <9602102037.ZM10034@hamster.appl-math.tu-muenchen.de> (et@appl-math.tu-muenchen.de)
Message-ID: <199602111533.KAA15537@cyclone.ERE.UMontreal.CA>
The Problem is mostly that all of these packages use their own matrix format
Of course; in C it is not at all obvious how to represent a matrix...
which is not that efficient. FORTRAN routines are somehow uniform in style
for matrix stuff, but I would prefer a new clean module
taking the code from there, not just interfacing it.
But that means not profiting from the enormous work others have put
into existing libraries. I'd much prefer using an existing library
which has been tested on many machines and is constantly kept up to
date with new machines, new compilers etc. We are much too small a
group to do that ourselves, even we were determined to try.
How to do a clean exception handling whitout knowing what a procedure does.
Sometimes you just get crashes, which are unacceptable, I want an exception
(not FORTRAN or other style coding.)
I don't see that as a problem. LAPACK, for example, returns an error
code indicating what is going wrong. This can be converted into an
exception by the interface. Of course libraries that print error
messages or even crash on an error should be avoided.
The more I think about it, the more I tend to favour LAPACK. It has
the advantage of being available in a C version as well as in Fortran,
so those who could gain from it (like me ;-) could use the Fortran
version, and yet everyone could use the module without needing a
Fortran compiler. And, of course, it is a very reliable and complete
library.
>Why not an Image SIG, targeted for graphics, publishing, GIS and
>good ole "scientific image processing" users...
I suggest to do that later, at the moment I would like to see
us writing code for the matrixmodule in one group. When we have some
I agree!
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From chris.chase@jhuapl.edu Mon Feb 12 14:48:00 1996
From: chris.chase@jhuapl.edu (Chris Chase S1A)
Date: Mon, 12 Feb 1996 09:48:00 -0500
Subject: [PYTHON MATRIX-SIG] Type Coercion
In-Reply-To: <9602071656.ZM12810@dsdbqvarsa.er.usgs.gov>
References: <9602072018.AA17817@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
<31191D29.D42@llnl.gov>
<dubois1@llnl.gov>
<9602071656.ZM12810@dsdbqvarsa.er.usgs.gov>
Message-ID: <199602121438.JAA18583@python.org>
Regarding the comments on type coercion. I agree with Paul's
comments: it is good not to lose precision and to not get an
exception. However, I would want the capability to turn on exceptions
for those who want to identify unwanted coercions. How could this be
made an option similar to capabilities of numerical packages that can
turn on/off traps on floating point exceptions?
I also agree with those who have voiced opinions to not to impose
deadlines on the module release. I would prefer an alpha release with
flexibility rather than a beta with fixed names/feature set. I do not
think there is a rush on this. Besides, deadlines for this project
can not supercede the main responsibilities that James and most of us
have in our day jobs.
Chris
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Mon Feb 12 19:11:33 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Mon, 12 Feb 1996 14:11:33 -0500
Subject: [PYTHON MATRIX-SIG] Type Coercion
In-Reply-To: <199602121438.JAA18583@python.org> (message from Chris Chase S1A on Mon, 12 Feb 1996 09:48:00 -0500)
Message-ID: <199602121911.OAA10227@cyclone.ERE.UMontreal.CA>
exception. However, I would want the capability to turn on exceptions
for those who want to identify unwanted coercions. How could this be
made an option similar to capabilities of numerical packages that can
turn on/off traps on floating point exceptions?
The problem gets a bit more complicated by the fact that you might
want this check to be applied only to part of the code. For example,
you might want to do "precision debugging" on your code while using a
library that coerces happily between float and double. Such things are
best left to a debugger, which already has provisions for doing things
on specific code regions and under specific conditions. And Python
fortunately has a debugger.
For such an approach, the umath module should provide some hook to
register a function that is called in case of a float->double
coercion. This hook would then be used by an extended debugger.
The main question is: who is willing to look at and modify
the debugger?
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Mon Feb 12 20:08:07 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Mon, 12 Feb 96 15:08:07 EST
Subject: [PYTHON MATRIX-SIG] Release 0.34 now available
Message-ID: <9602122008.AA27148@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
The major news for this release is that Chris Chase's grammar patches
to implement multidimensional slices are included. This means that
you can write:
a[1:, :-1] and other exciting variations now work. The old keywords
All, Reverse, and Slice no longer work in this new version.
I should warn people that I rushed out this version because a few
people were particularly eager to get there hands on it. I haven't
tested this as well as I'd like to, so as usual use at your own risk.
I hope to release something more stable by this Friday, so if you're
not into the cutting edge...
It's reasonably likely that the implementation details of
multi-dimensional slices will change before they become an official
patch to the python core. Nevertheless, it's very likely that the
syntax/semantics of these slices will be exactly as they are in this
distribution.
Personally I find that my own code looks a lot cleaner without all the
Slice(None, -1) stuff in it, thanks Chris!
As usual, this release also includes many minor bug fixes.
(It also includes a small performance optimization, but its unlikely
you'll notice that).
Also, as of this release I officially give in to the will of the sig
on the type coercion issue. Paul DuBois's comment on the relative
performance differences between floats and doubles was probably the
straw that broke my camels back.
floats are now automatically coerced to doubles, and all the other
coercions that you'd expect.
I really like Konrad's idea of adding some hooks into the debugger to
detect automatic type coercions and I'm looking into that.
Also, this release should compile on the Mac (thanks to Steve
Spicklemire for making this happen). He's also got a version working
under windows, but I didn't have the time to make the necessary
changes before this release. Now if I can just get somebody to test
this on a BeBox!
Patches from 0.33 are in
ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumPy.patch-0.33-0.34.gz
The whole thing is in
ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericPython-0.34.tar.gz
Enjoy - Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From da@maigret.cog.brown.edu Tue Feb 13 03:39:01 1996
From: da@maigret.cog.brown.edu (David Ascher)
Date: Mon, 12 Feb 1996 22:39:01 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] RubberIndex
Message-ID: <199602130339.WAA02625@maigret>
I'm still trying to figure out the ... index. If I understand
correctly, it stands for "however many ':' I need depending on the rank
of the object I'm indexing, so that the indices I *do* specify are at the
end of the index list as opposed to the usual beginning.
So, if I have a rank-3 array A, then
A[...,0]
is the same thing as
A[:,:,0]
but if B is rank-4, then
B[...,0]
is the same thing as:
B[:,:,:,0]
This "stretching" only works from the left: In other words, if '...' is
used *after* a non-pseudo index, then it is equivalent to ':', and does
not stretch. [should ... be allowed after a non-pseudo index?]
If C is a rank-5 array, then
C[...,0,...]
is the same thing as
C[...,0,:]
and
C[:,:,:,0,:]
but NOT:
C[:,:,0,:,:]
This seems to work with my test cases. Am I missing some key
functionality?
--david
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Feb 13 13:47:08 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Tue, 13 Feb 1996 08:47:08 -0500
Subject: [PYTHON MATRIX-SIG] RubberIndex
In-Reply-To: <199602130339.WAA02625@maigret> (da@maigret.cog.brown.edu)
Message-ID: <199602131347.IAA19640@cyclone.ERE.UMontreal.CA>
This "stretching" only works from the left: In other words, if '...' is
used *after* a non-pseudo index, then it is equivalent to ':', and does
not stretch. [should ... be allowed after a non-pseudo index?]
'...' at the end of an index expression makes no sense, as you could
just as well leave it out (a[0, ...] is the same as a[0]). Neither
does it make sense to have more than one '...' in an index expression,
since it stands for "all remaining axes", which you can use up only
once. It would be good if this would lead to an exception. But it
does make sense to have '...' between two groups of other indices,
i.e. a[1, ..., 0] or a[2, ..., NewAxis].
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From chris.chase@jhuapl.edu Tue Feb 13 16:36:50 1996
From: chris.chase@jhuapl.edu (Chris Chase S1A)
Date: Tue, 13 Feb 1996 11:36:50 -0500
Subject: [PYTHON MATRIX-SIG] RubberIndex
In-Reply-To: <199602131347.IAA19640@cyclone.ERE.UMontreal.CA>
References: <199602130339.WAA02625@maigret>
<199602131347.IAA19640@cyclone.ERE.UMontreal.CA>
Message-ID: <199602131627.LAA29926@python.org>
The implementation should only allow a single '...'. If multiple
rubber indexes are used then an exception should be raised. I could
change Grammar so that multiple '...' causes a syntax error at compile
time.
Think of the rubber index as justifying/aligning the index list within
the shape of the array. Indexes preceding '...' are left-justified
and succeeding indexes are right-justified.
Let P1,P2,..,PM be indexes preceding and S1,S2,..,SL be indexes
succeeding '...' in an index list. If the shape is D1,D2,D3,..,DN-1,DN
for a rank N array, P1,P2,..,PL are left-justified so they are
associated with dimensions D1,D2,..,DL. S1,S2,..,SM are
right-justified so SM -> DN, SM-1 -> DN-1, etc. Any dimension Di not
associated with one of the actual indexes is assumed to be selected
with a ":".
The rubber index is analogous to negative limits for slices. Just as
a negative slice limit (a[0:-2]) is relative to last element in the
sequence without knowing the length apriori, the rubber index '...'
allows one to specify indexes relative to the end of the shape without
knowing the rank apriori.
Any psuedo indexes (specified by NewAxis) are placed relative to the Pi,
Si indexes next to it. Even when the psuedo index is next to a rubber
index there is no ambiguity as it will be placed relative to the
actual indexes to its left or right.
Chris
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From drh@oasis.unl.edu Tue Feb 13 16:49:41 1996
From: drh@oasis.unl.edu (Doug Heisterkamp)
Date: Tue, 13 Feb 1996 10:49:41 -0600 (CST)
Subject: [PYTHON MATRIX-SIG] LAPACK module questions
Message-ID: <199602131649.KAA29944@oasis.unl.edu>
Hi,
I have completed a first pass at a C module to access the LAPACK library. I
have a few questions as I move on an try to refine it. But first, what I've
done is to write a parser that reads the source code from CLAPACK and writes
an access call for each function. The name and parameters are the same as in
the library with the exception of using the array object for matrices and
vectors.
And now the questions.
What is the proper way to access the data of an array object? I did not pay
attention to the abstract object interface discussion on the main group and
am not sure how to do it through that interface. Currently I'm just directly
accessing it such as (for a double a from array b)
a=(double *)((PyArrayObject *)PyArray_ContiguousFromObject(b,PyArray_DOUBLE,
1,0))->data;
How should I package this C module with a python module? Should it become
a subclass of Matrix or UserArray? Should it be a module with just a bunch
of functions in it?
Are there other people working on accessing math libraries? It would be nice
if we could set up some conventions on function names, calling parameters,
and exceptions for some of the common items. That way code that needs, say
eigenvalues, could import whichever math library that is on the machine and
still use a standard call to get the eigenvectors and values of a matrix.
I'm planning on writing a python wrapper for eigenvectors, determinants,
and svd. What other functions are commonly used and would be nice to
have a clean interface to (as compared to the fortran interface of the
lapack library)?
If people are interested in the module, I could place it on an ftp server
somewhere. It currently uses libraries built from CLapack which are just
the f2c conversion of the original fortran source. Elf binaries of the
lapack and blas libraries are available at the linux sites (if you need
the directory, email me. I have the location written down elsewhere). The
source for lapack, clapack, and f2c are available from netlib.att.com.
Doug Heisterkamp
drh@oasis.unl.edu
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Feb 13 16:48:04 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Tue, 13 Feb 1996 11:48:04 -0500
Subject: [PYTHON MATRIX-SIG] RubberIndex
In-Reply-To: <199602131627.LAA29926@python.org> (message from Chris Chase S1A on Tue, 13 Feb 1996 11:36:50 -0500)
Message-ID: <199602131648.LAA28672@cyclone.ERE.UMontreal.CA>
The implementation should only allow a single '...'. If multiple
rubber indexes are used then an exception should be raised. I could
change Grammar so that multiple '...' causes a syntax error at compile
time.
That is not a good idea, as someone might want to use this syntax
with a somewhat different meaning for other sequence objects.
It would be better to have an exception raised at runtime.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From chris.chase@jhuapl.edu Tue Feb 13 17:14:21 1996
From: chris.chase@jhuapl.edu (Chris Chase S1A)
Date: Tue, 13 Feb 1996 12:14:21 -0500
Subject: [PYTHON MATRIX-SIG] RubberIndex
In-Reply-To: <199602131347.IAA19640@cyclone.ERE.UMontreal.CA>
References: <199602130339.WAA02625@maigret>
<199602131347.IAA19640@cyclone.ERE.UMontreal.CA>
Message-ID: <199602131704.MAA30240@python.org>
>>>>> "Konrad" == Konrad HINSEN <hinsenk@ere.umontreal.ca> writes:
Konrad> '...' at the end of an index expression makes no sense, as you could
Konrad> just as well leave it out (a[0, ...] is the same as a[0]).
It makes perfect sense. Just because there is another way to
accomplish the same thing does not make it bad.
In fact, I would prefer to eliminate a[0] in preference of a[0,...].
I do not like the fact that an error is not generated if I use to few
indexes. For example, suppose 'a' has shape (2,4,5,6). I want the
first element but either I forget that it is rank 4 or incorrectly
enter the index:
scalar = a[0,0,0]
Instead of a scalar, I get an array with shape (6,). This "feature"
requires that I check the rank of every parameter passed into a
function for the desired rank. If I really want the last dimension as
a vector I could specify
vector = a[0,0,0,...]
vector = a[0,0,0,:]
I would prefer an error be produced by a[0,0,0]. An error already
occurs when specifying too many indexes, i.e. a[0,0,0,0,0]. The same
should occur for too few indexes unless a rubber index "..." is
present (although even with a rubber index too indexes is an error).
When using IDL, I have found that an error for too few indexes to be
very useful in quickly finding mistakes in code. Without this error,
I might not find out about the bug until much later in my code and
then I might have a difficult time tracing it back to its origin.
The only purpose I see for a[0] to be valid is to make it look like a
list for indexing so that a[0][1] is valid. I suppose that if we want
an array to be a specialized or subclassed list (so that it works
wherever a list occurs) then this has to be supported. But if we
think of an array as more of a dictionary than a list it should not
support this kind of indexing. I could see that it could lead to
confusion when we have multi-dimensional arrays of other types of
python objects. Suppse 'a' is a rank 2 array containing a
subscriptable python object. a[0,0][1] and a[0][0][1] would be valid.
However, if 'a' had a rank different than 2 then we could have it
produce an error - alerting the user that the object is not what he
thought.
Chris
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Tue Feb 13 18:32:24 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Tue, 13 Feb 96 13:32:24 EST
Subject: [PYTHON MATRIX-SIG] LAPACK module questions
In-Reply-To: <199602131649.KAA29944@oasis.unl.edu> (drh@oasis.unl.edu)
Message-ID: <9602131832.AA12640@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
I'm probably the only one here who can answer these questions (yes I
know that documentation for the C API would be nice).
First a question. Are you building this on top of Guido's bgen tool?
I really think that this is the right way to go. Hopefully bgen can
be eventually turned into a nice generic tool for specifying
interfaces to any C/FORTRAN library. It already handles a large
collection of interface specifications. It has the very nice modular
design of one automatic part that tries to extract the interface
definition which produces interface definition files that can be
easily edited by the user for the cases where it guesses wrong.
Adding array objects to bgen would not be a whole lot of work (so many
things I'd like to do if I just had a free 10 hours or so).
I think that creating a raw C level binding to these libraries is the
right thing to do, and then to write a python module that provides a
nice friendly interface on top of the raw functions created in the
module. The basic point here is that this should be doable without
writing any C code.
I'd be interested in seeing a copy of what you've done so far.
Arrays are not part of the abstract object interface. The right way
to write a function that takes in a single argument and passes it on
to a C function expecting a 1d array of doubles and returning a 1d
array of doubles is as follows:
static PyObject *function(PyObject *ignored, PyObject *args) {
PyArrayObject *ap, *apr;
PyObject *op;
int n;
TRY(PyArg_ParseTuple(args, "O", &op));
TRY(ap = (PyArrayObject *)PyArray_ContiguousFromObject(op, PyArray_DOUBLE, 1, 1));
n = ap->dimensions[0];
TRY(apr = (PyArrayObject *)PyArray_FromDims(1, &n, PyArray_DOUBLE));
c_function((double *)ap->data, (double *)apr->data, n)
Py_DECREF(ap); /* Needed because PyArray_ContiguousFromObject INCREFs it. */
return (PyObject *)apr;
}
In fact, if there is an error in allocating the return array this will
leak memory, but I figure anytime you're getting exceptions during
array allocation, enough things are wrong with your system that a
little memory leak won't be a problem.
Hopefully from the general complexity of this you can see the value in
a bgen style approach.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dubois@kristen.llnl.gov Tue Feb 13 20:15:00 1996
From: dubois@kristen.llnl.gov (P. Dubois)
Date: Tue, 13 Feb 1996 12:15:00 -0800
Subject: [PYTHON MATRIX-SIG] Naming in arrayobject.c
Message-ID: <9602132015.AA25919@kristen.llnl.gov>
SIG'ers,
Here are some thoughts about the names in arrayobject.c; I am now leaving
town until the 20th so that I don't have to listen to the resulting howls
of outrage (:->. My philosophy is close to that used in the ISE Eiffel
libraries and as described in the book Reusable Software by Meyer.
1. These are ok as is:
transpose (ok, imperative verb for something that modifies self)
cast (ok, ditto)
typecode (ok, noun = attribute)
itemsize (ok, noun = attribute)
toString
toFile
choose (this is so weird a function anyway ...)
2. These I think need changing:
Current -> Proposed Reason
------------------------------------
byteswap -> byteswapped (returns result, shouldn't be an imperative verb)
reshape -> shaped (returns result, shouldn't be an imperative verb)
If worried about typos because shape and shaped too similar,
consider shaped_as. But not reshape, which definitely sounds like
self is being modified.
isnonzero -> nonzero_indices (sounds too much like test of not all zeros)
This returns a list of indices where x is not zero.
(Note: maybe a new function x.is_zero() would be useful to mean
true => all elements are 0)
copy -> clone
copy is an ambiguous word as to whether one is the copier or the
copyee. But clone definitely carries more of a feeling that this
is going to produce a copy.
In Eiffel, x.copy(y) means set x's attributes from y, for example.
concat -> append (philosophically equivalent to list.append)
If you don't like append, then change it to concatenate;
abbreviations are not worth the trouble they cause.
contiguous -> is_contiguous
This is a query, not a command to make self contiguous
3. These are not clear to me
take -> gather
m.take(i_list) returns [m[i_list[0]], m[i_list[1]], ...]
Gather is a more traditional name but I won't argue this too hard.
conjugate -> conjugated
Ambiguous as to whether it modifies self.
But, you could make the case "conjugate" is a noun, not a verb
in which case it is ok.
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Feb 13 20:16:05 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Tue, 13 Feb 1996 15:16:05 -0500
Subject: [PYTHON MATRIX-SIG] LAPACK module questions
In-Reply-To: <199602131649.KAA29944@oasis.unl.edu> (drh@oasis.unl.edu)
Message-ID: <199602132016.PAA11799@cyclone.ERE.UMontreal.CA>
I have completed a first pass at a C module to access the LAPACK library. I
Great!
have a few questions as I move on an try to refine it. But first, what I've
done is to write a parser that reads the source code from CLAPACK and writes
an access call for each function. The name and parameters are the same as in
the library with the exception of using the array object for matrices and
vectors.
Not bad for a low-level access module; some people are familiar with
these names and will probably want to use them. I just wonder what you
do with the routines for special storage methods (banded, packed
symmetric, etc.); they don't make any sense in Python.
How should I package this C module with a python module? Should it become
a subclass of Matrix or UserArray? Should it be a module with just a bunch
of functions in it?
I'd say the latter. I have thought a bit about how the current array
operations and extensions such as linear algebra can best be combined
and used with a consistent access scheme. I have come to the conclusion
that the best solution is to have only functions in various modules
for all mathematical and structural operations, and use methods only
for special stuff like typecode inquiries.
This proposal is certainly somewhat surprising for true believers in
object-oriented programming, but I'll argue that it is still the best
compromise. The alternative in a true object-oriented spirit would be
to implement most (if not all) operations as methods on array objects.
For extensions like linear algebra, one would define a subclass of
arrays with added methods. Someone who wants to use several such
extensions, e.g. linear algebra and fft, would then create another
class inheriting both from linear-algebra-arrays and fft-arrays.
There are three problems with this approach, one specific to Python,
and two generic. The Python-specific problem is that one cannot
inherit from C types, and that C types cannot inherit from
anything. That would make a Python wrapper for all array classes
obligatory, with a resulting performance penalty (mostly noticeable
for small arrays).
One of the generic problems (at least in languages without type
checking) is that mixing different array classes causes a lot of
headaches. If some library returns an fft-array and someone want to
use it as a linear-algebra-array, then it has to be converted
first. This would be worst for the standard constructors, like
zeros(), whose results could not be used in any derived module.
The other generic problem is caused by the relatively many operations
that depend symmetrically on more than one object. It doesn't really
make sense to consider an addition as an operation on the first number
with the second as parameter, and this is basically what causes all
the coercion problems for binary operators that Python suffers from.
I am not saying that a more sensible approach is impossible (one could
for example define addition as an operation on a tuple of numbers),
but I am not aware of any language that supports this in a clean way.
The best way out in Python is, in my opinion, not to implement any
operation as a method. That leads to a clean system without confusion,
and Python's module system takes care of name space pollution. It may
be old-fashioned, but it works.
Are there other people working on accessing math libraries? It would be nice
if we could set up some conventions on function names, calling parameters,
and exceptions for some of the common items. That way code that needs, say
Definitely. Still all I personally need is LAPACK...
I'm planning on writing a python wrapper for eigenvectors, determinants,
and svd. What other functions are commonly used and would be nice to
have a clean interface to (as compared to the fortran interface of the
lapack library)?
Inverse, generalized inverse, and solution of linear equations. As a
second step, generalized eigenvalue problems and linear equations with
iterative refinement.
If people are interested in the module, I could place it on an ftp server
somewhere. It currently uses libraries built from CLapack which are just
Yes, please!
the f2c conversion of the original fortran source. Elf binaries of the
Great, so one should be able to substitute the Fortran library
easily.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Feb 13 20:22:43 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Tue, 13 Feb 1996 15:22:43 -0500
Subject: [PYTHON MATRIX-SIG] RubberIndex
In-Reply-To: <199602131702.MAA00480@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Tue, 13 Feb 1996 12:14:21 -0500)
Message-ID: <199602132022.PAA12218@cyclone.ERE.UMontreal.CA>
In fact, I would prefer to eliminate a[0] in preference of a[0,...].
I do not like the fact that an error is not generated if I use to few
indexes. For example, suppose 'a' has shape (2,4,5,6). I want the
You certainly have a point, but it also makes sense to keep
the usability of arrays as sequence objects. For example, you
can now iterate over the first axis of an array with
for x in array([[1,2],[3,4]]): print x*x
Also, most array functions accept nested lists as well as arrays.
I'd like to keep this analogy as far as possible.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From da@maigret.cog.brown.edu Tue Feb 13 20:54:00 1996
From: da@maigret.cog.brown.edu (David Ascher)
Date: Tue, 13 Feb 1996 15:54:00 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] Naming in arrayobject.c
In-Reply-To: <9602132015.AA25919@kristen.llnl.gov> from "P. Dubois" at Feb 13, 96 12:15:00 pm
Message-ID: <199602132054.PAA05052@maigret>
Mostly, I think Paul's suggestions are clear and good.
> copy -> clone
> copy is an ambiguous word as to whether one is the copier or the
> copyee. But clone definitely carries more of a feeling that this
> is going to produce a copy.
> In Eiffel, x.copy(y) means set x's attributes from y, for example.
This is a little odd since python uses copy generally, I think.
Clone is certainly less ambiguous.
> 3. These are not clear to me
>
> take -> gather
> m.take(i_list) returns [m[i_list[0]], m[i_list[1]], ...]
> Gather is a more traditional name but I won't argue this too hard.
Neither is very intuitive, but since the function isn't either...
> conjugate -> conjugated
> Ambiguous as to whether it modifies self.
> But, you could make the case "conjugate" is a noun, not a verb
> in which case it is ok.
Not really, since my first read was that it was a verb. It's not what
was intended which should matter, but what is understood.
I'll ask my students tonight what they think of these, and see if I come
up with any interesting feedback.
--david
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From drh@oasis.unl.edu Tue Feb 13 21:40:07 1996
From: drh@oasis.unl.edu (Doug Heisterkamp)
Date: Tue, 13 Feb 1996 15:40:07 -0600 (CST)
Subject: [PYTHON MATRIX-SIG] LAPACK module questions
In-Reply-To: <9602131832.AA12640@baribal.LCS.MIT.EDU.LCS.MIT.EDU> from "James Hugunin" at Feb 13, 96 01:32:24 pm
Message-ID: <199602132140.PAA00422@oasis.unl.edu>
''
>
>
> I'm probably the only one here who can answer these questions (yes I
> know that documentation for the C API would be nice).
>
> First a question. Are you building this on top of Guido's bgen tool?
> I really think that this is the right way to go. Hopefully bgen can
> be eventually turned into a nice generic tool for specifying
> interfaces to any C/FORTRAN library. It already handles a large
> collection of interface specifications. It has the very nice modular
> design of one automatic part that tries to extract the interface
> definition which produces interface definition files that can be
> easily edited by the user for the cases where it guesses wrong.
> Adding array objects to bgen would not be a whole lot of work (so many
> things I'd like to do if I just had a free 10 hours or so).
Unfortunately, I did not use bgen. I'll take a look at to see what it
can do.
>
> I think that creating a raw C level binding to these libraries is the
> right thing to do, and then to write a python module that provides a
> nice friendly interface on top of the raw functions created in the
> module. The basic point here is that this should be doable without
> writing any C code.
>
> I'd be interested in seeing a copy of what you've done so far.
I'll place a copy on the ftp server here at unl. I'll make some of the
changes from below before placing it there. I'll let the list know when
it's placed on the server and it actual location.
Doug
>
> Arrays are not part of the abstract object interface. The right way
> to write a function that takes in a single argument and passes it on
> to a C function expecting a 1d array of doubles and returning a 1d
> array of doubles is as follows:
>
> static PyObject *function(PyObject *ignored, PyObject *args) {
> PyArrayObject *ap, *apr;
> PyObject *op;
> int n;
>
> TRY(PyArg_ParseTuple(args, "O", &op));
> TRY(ap = (PyArrayObject *)PyArray_ContiguousFromObject(op, PyArray_DOUBLE, 1, 1));
>
> n = ap->dimensions[0];
>
> TRY(apr = (PyArrayObject *)PyArray_FromDims(1, &n, PyArray_DOUBLE));
>
> c_function((double *)ap->data, (double *)apr->data, n)
>
> Py_DECREF(ap); /* Needed because PyArray_ContiguousFromObject INCREFs it. */
>
> return (PyObject *)apr;
> }
>
> In fact, if there is an error in allocating the return array this will
> leak memory, but I figure anytime you're getting exceptions during
> array allocation, enough things are wrong with your system that a
> little memory leak won't be a problem.
>
> Hopefully from the general complexity of this you can see the value in
> a bgen style approach.
>
> -Jim
>
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From drh@oasis.unl.edu Tue Feb 13 22:13:07 1996
From: drh@oasis.unl.edu (Doug Heisterkamp)
Date: Tue, 13 Feb 1996 16:13:07 -0600 (CST)
Subject: [PYTHON MATRIX-SIG] LAPACK module questions
In-Reply-To: <3120DBFE.4C9E@llnl.gov> from "Paul. Dubois" at Feb 13, 96 10:44:14 am
Message-ID: <199602132213.QAA00459@oasis.unl.edu>
''
>
> We'd be interested in hearing about your parser. We face many tasks like
> this.
Actually, parser may be a little to generous to describe it. I wrote it
as a throwaway program target at f2c version of the Lapack source code.
It scans for the first function definition in a file and grabs it. The
parameter list is parsed and depending on the parameter type, different
information is added to the end of a number of different lists. These
lists are then used to generate the C function to access the original
function. The first comment after the function definition is placed as
the __doc__ attribute of the access function. Of course this is written
in python.
I made a number of assumption about the parameter list. The main one
is that I assume that all parameters which are double or complex are
actually pointers to arrays. This was needed since in the C version,
both a double and an array of doubles is referred to as double *a, in
the function definition. I will have to change the cases where this is
the wrong assumption by hand. Until I make the change, the function could
still be used, but it would be necessary to place the double in a python
array object of a single element and then use this array object in the
function call.
Similarly, I made the assumption that all integer parameters are not
arrays. My goal was to get close and then make changes as needed.
Doug
> --
> Paul F. Dubois, L-472 (510)-422-5426
> Lawrence Livermore National Laboratory FAX (510)-423-9969
> Livermore, CA 94550 dubois1@llnl.gov
> Consulting: PaulDubois@aol.com
>
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From drh@oasis.unl.edu Tue Feb 13 22:57:21 1996
From: drh@oasis.unl.edu (Doug Heisterkamp)
Date: Tue, 13 Feb 1996 16:57:21 -0600 (CST)
Subject: [PYTHON MATRIX-SIG] LAPACK module questions
In-Reply-To: <199602132016.PAA11799@cyclone.ERE.UMontreal.CA> from "Konrad HINSEN" at Feb 13, 96 03:16:05 pm
Message-ID: <199602132257.QAA00502@oasis.unl.edu>
>
> Not bad for a low-level access module; some people are familiar with
> these names and will probably want to use them. I just wonder what you
> do with the routines for special storage methods (banded, packed
> symmetric, etc.); they don't make any sense in Python.
I assume that the caller knows the structure of the storage methods and
packs it into an python array object. I have not used these functions
yet (I really just started to use the Lapack library), but they seem to
want one or two dimensional arrays with specific information at certain
locations. The low-level access module assumes this has already been
done for the objects passed to it. A higher level access module would
be useful in setting up these structures.
>
> How should I package this C module with a python module? Should it become
> a subclass of Matrix or UserArray? Should it be a module with just a bunch
> of functions in it?
>
> I'd say the latter. I have thought a bit about how the current array
> operations and extensions such as linear algebra can best be combined
> and used with a consistent access scheme. I have come to the conclusion
> that the best solution is to have only functions in various modules
> for all mathematical and structural operations, and use methods only
> for special stuff like typecode inquiries.
>
<snip>
>
> The best way out in Python is, in my opinion, not to implement any
> operation as a method. That leads to a clean system without confusion,
> and Python's module system takes care of name space pollution. It may
> be old-fashioned, but it works.
I agree.
>
> Are there other people working on accessing math libraries? It would be nice
> if we could set up some conventions on function names, calling parameters,
> and exceptions for some of the common items. That way code that needs, say
>
> Definitely. Still all I personally need is LAPACK...
>
> I'm planning on writing a python wrapper for eigenvectors, determinants,
> and svd. What other functions are commonly used and would be nice to
> have a clean interface to (as compared to the fortran interface of the
> lapack library)?
>
> Inverse, generalized inverse, and solution of linear equations. As a
> second step, generalized eigenvalue problems and linear equations with
> iterative refinement.
What is a "generalized inverse"? If I remember correctly, the generalized
eigenvalue problem is (A - lambda C)x = 0 where C is a matrix. But I don't
remember running across a generalized inverse.
>
> If people are interested in the module, I could place it on an ftp server
> somewhere. It currently uses libraries built from CLapack which are just
>
> Yes, please!
>
> the f2c conversion of the original fortran source. Elf binaries of the
>
> Great, so one should be able to substitute the Fortran library
> easily.
I hope so as the Fortran libraries should be faster.
Doug
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Wed Feb 14 13:45:10 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Wed, 14 Feb 1996 08:45:10 -0500
Subject: [PYTHON MATRIX-SIG] LAPACK module questions
In-Reply-To: <199602132257.QAA00502@oasis.unl.edu> (drh@oasis.unl.edu)
Message-ID: <199602141345.IAA19287@cyclone.ERE.UMontreal.CA>
I assume that the caller knows the structure of the storage methods and
packs it into an python array object. I have not used these functions
Yes, that's a reasonable assumption. After all, it's low level. A
good interface can always be added later by defining a Python class
representing e.g. a packed symmetric matrix.
What is a "generalized inverse"? If I remember correctly, the generalized
eigenvalue problem is (A - lambda C)x = 0 where C is a matrix. But I don't
remember running across a generalized inverse.
It's also known as a pseudoinverse or Moore-Penrose-inverse. There is
a uniquely defined generalized inverse for any matrix, and in the case
of a square non-singular matrix it is equal to the ordinary inverse.
Generalized inverses can be used to express the solution of over- or
underdetermined (or both) systems of linear equations conveniently,
and they are very useful in least-squares problems. The preferred way
to calculate the generalized inverse uses SVD: take the original
matrix, do SVD, invert all the non-zero singular values, multiply the
three matrices again, and return the transpose. Trivial to code in
Python based on low-level SVD calls.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From chris.chase@jhuapl.edu Wed Feb 14 16:56:02 1996
From: chris.chase@jhuapl.edu (Chris Chase S1A)
Date: Wed, 14 Feb 1996 11:56:02 -0500
Subject: [PYTHON MATRIX-SIG] RubberIndex
In-Reply-To: <199602132022.PAA12218@cyclone.ERE.UMontreal.CA>
References: <199602131702.MAA00480@cyclone.ERE.UMontreal.CA>
<199602132022.PAA12218@cyclone.ERE.UMontreal.CA>
Message-ID: <199602141646.LAA05432@python.org>
>>>>> "Konrad" == Konrad HINSEN <hinsenk@ere.umontreal.ca> writes:
Konrad> In fact, I would prefer to eliminate a[0] in preference of
Konrad> a[0,...]. I do not like the fact that an error is not
Konrad> generated if I use to few indexes. For example, suppose
Konrad> 'a' has shape (2,4,5,6). I want the
Konrad> You certainly have a point, but it also makes sense to keep
Konrad> the usability of arrays as sequence objects. For example, you
Konrad> can now iterate over the first axis of an array with
Konrad> for x in array([[1,2],[3,4]]): print x*x
It would be better to have an iterator method that would allow one to
access regular sub-arrays. At the minimum it would allow you to
specify iteration along any chosen dimension (i.e. a.iterator(dim)).
I am not sure how to extend it to allow iterating over other
sub-arrays (e.g. 2x2 sub-matrices).
Konrad> Also, most array functions accept nested lists as well as arrays.
Konrad> I'd like to keep this analogy as far as possible.
Generating an error when using too few indexes when subscripting an
array does not prohibit using nested lists as arguments to array
functions.
Chris
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Wed Feb 14 18:31:31 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Wed, 14 Feb 1996 13:31:31 -0500
Subject: [PYTHON MATRIX-SIG] RubberIndex
In-Reply-To: <199602141643.LAA28371@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Wed, 14 Feb 1996 11:56:02 -0500)
Message-ID: <199602141831.NAA04933@cyclone.ERE.UMontreal.CA>
It would be better to have an iterator method that would allow one to
access regular sub-arrays. At the minimum it would allow you to
specify iteration along any chosen dimension (i.e. a.iterator(dim)).
I am not sure how to extend it to allow iterating over other
sub-arrays (e.g. 2x2 sub-matrices).
That should certainly be available, but that was not my point...
Konrad> Also, most array functions accept nested lists as well as arrays.
Konrad> I'd like to keep this analogy as far as possible.
Generating an error when using too few indexes when subscripting an
array does not prohibit using nested lists as arguments to array
functions.
That wasn't my point either. My point is that since arrays and nested
lists are treated as interchangeable in many situations, it makes
sense to use this analogy as much as possible and allow an indexing
scheme for arrays that matches that for nested arrays. WHich means
that it should always be allowed to have just one index, independent
of the rank of the array.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From da@maigret.cog.brown.edu Wed Feb 14 22:30:16 1996
From: da@maigret.cog.brown.edu (David Ascher)
Date: Wed, 14 Feb 1996 17:30:16 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] RubberIndex
In-Reply-To: <199602141831.NAA04933@cyclone.ERE.UMontreal.CA> from "Konrad HINSEN" at Feb 14, 96 01:31:31 pm
Message-ID: <199602142230.RAA08591@maigret>
> That wasn't my point either. My point is that since arrays and nested
> lists are treated as interchangeable in many situations, it makes
> sense to use this analogy as much as possible and allow an indexing
> scheme for arrays that matches that for nested arrays. WHich means
> that it should always be allowed to have just one index, independent
> of the rank of the array.
This is sort of irrelevant, but it has occured to me, now that I
understand and really like [::-1], I was sort of hoping that I could do
the same with lists. How easy would it be to extend that third argument
to list slices?
--david
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From et@appl-math.tu-muenchen.de Thu Feb 15 21:25:25 1996
From: et@appl-math.tu-muenchen.de (Thomas Schwaller)
Date: Thu, 15 Feb 1996 22:25:25 +0100
Subject: [PYTHON MATRIX-SIG] LAPACK
Message-ID: <9602152225.ZM29681@hamster.appl-math.tu-muenchen.de>
I hope so as the Fortran libraries should be faster.
I have the possiblity to test the FORTRAN libraries with a new commercial
Fortran Compiler on Linux. Will see! BTW g77 is told to be faster than
f2c on Linux e.g.
Tom
Eagerly waiting for the Lapack/Python module....
Sounds great.
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From stoll@atr-sw.atr.co.jp Fri Feb 16 07:55:06 1996
From: stoll@atr-sw.atr.co.jp (Perry A. Stoll)
Date: Fri, 16 Feb 1996 16:55:06 +0900
Subject: [PYTHON MATRIX-SIG] changes to compress
Message-ID: <199602160755.QAA18871@ciris21.atr-sw.atr.co.jp>
Howdy All,
Question: How is the function Numeric.compress supposed to behave if
none of the elements of m meet the condition?
def compress(condition, m, dimension=-1):
"""compress(condition, x, dimension=-1) = those elements of x corresponding
to those elements of condition that are "true". condition must be the
same size as the given dimension of x."""
Right now it raises an exception. I think it should return some
NULL-type object, either None, [] or maybe an empty array object if
such a thing exists.
The problem stems from array methods not accepting empty lists as
arguments. Couldn't it just return an empty Array object, which would
act just like any other empty mapping or sequence object?
For now, I've changed compress to:
def compress(condition, m, dimension=-1):
nonzero = condition.nonzero()
if (nonzero): return m.take(nonzero, dimension)
else: return []
I noticed this while using Konrad's pretty printer - did anyone else
notice this? Konrad?
-Perry
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Fri Feb 16 14:23:01 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Fri, 16 Feb 1996 09:23:01 -0500
Subject: [PYTHON MATRIX-SIG] changes to compress
In-Reply-To: <199602160755.QAA18871@ciris21.atr-sw.atr.co.jp> (stoll@atr-sw.atr.co.jp)
Message-ID: <199602161423.JAA17508@cyclone.ERE.UMontreal.CA>
Right now it raises an exception. I think it should return some
NULL-type object, either None, [] or maybe an empty array object if
such a thing exists.
It should definitely return an empty array object. And all the array
functions should be able to deal with empty arrays without raising
exceptions. As any APL programmer will confirm, empty arrays occur
very often in array manipulations. It is even necessary to keep the
correct shapes for empty arrays, i.e. an array of shape (0,2) must be
distinct from an array of shape (3,0,4), even though both contain no
elements. Therefore a generic "empty" object like None can't be
used as a substitute.
The problem stems from array methods not accepting empty lists as
arguments. Couldn't it just return an empty Array object, which would
act just like any other empty mapping or sequence object?
That's what it should do.
I noticed this while using Konrad's pretty printer - did anyone else
notice this? Konrad?
Yes, I have even pointed it out in one of my messages about the pretty
printer.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From drh@oasis.unl.edu Fri Feb 16 23:27:02 1996
From: drh@oasis.unl.edu (Doug Heisterkamp)
Date: Fri, 16 Feb 1996 17:27:02 -0600 (CST)
Subject: [PYTHON MATRIX-SIG] Python-LAPACK module is available
Message-ID: <199602162327.RAA20173@oasis.unl.edu>
For those who may wish to try my python-LAPACK module, it is now
available (finally).
The ftp location is:
ftp.cs.unl.edu/pub/drh/python
Two versions are available. One with and one without the __doc__
string attributes. The one without the __doc__ is about 1/4 the
size of the other one.
Let me know if it works for your machine. I have built it on an SGI
and an Linux box.
This the first version so please send me the bug reports so we can
make a useful module out of it.
Doug
drh@oasis.unl.edu
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From et@appl-math.tu-muenchen.de Sun Feb 18 23:37:40 1996
From: et@appl-math.tu-muenchen.de (Thomas Schwaller)
Date: Mon, 19 Feb 1996 00:37:40 +0100
Subject: [PYTHON MATRIX-SIG] trimodule.c Anybody interested?
Message-ID: <9602190037.ZM5679@hamster.appl-math.tu-muenchen.de>
I've written some triangulation modules for Python and promised some
people to release them. Most of them use C++-class libraries and are difficult
to handle (some compilable only with special compilers), so to make them public
is probably not so wise.
During the weekend I found some time to write a new one with some code
I found usefull for it. I consists of only one module, adding
tri trimodule.c
to Setup is enough. It is based on a french FEM package and I started to write
some documentation for it. It works in 2 dimensions only, but there it seems to
be a fine and fast tool. (Handling of local refinements, holes, cracks,
subdomains is possible, to name just a few possibilities)
As I got no feedback for my delaunaymodule.c, I'll first ask this time.
Is anybody interested in this module?
If yes, I'll release it next week after writing some more stuff for the
documentation which I wrangled out of the Source-Code (It is mostly in French,
so I have to do that job for this module to be useful for others).
And I'll have to test it a little bit more (perhaps adding some pl-module
stuff displaying it or a Postscript output methods, which is not that
difficult)
Comments?
Tom
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From da@maigret.cog.brown.edu Tue Feb 20 16:34:36 1996
From: da@maigret.cog.brown.edu (David Ascher)
Date: Tue, 20 Feb 1996 11:34:36 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] returning scalars or rank-0 arrays..
Message-ID: <199602201634.LAA21501@maigret>
I'm working on a 'literate' array class, which labels axes and indices.
The point is that one can do slicing based on keyword references to the
axes, and labels for index values. I'll of course make it available
when it's ready for testing.
One issue I've come accross is the "what do I return, a rank-0 entity or
a scalar?". My understanding of the current scheme is that indexing
returns scalars, while slicing returns ranked entities, even if that
rank is 0. Is that correct?
>>> a = arange(9).reshape(3,3)
>>> b = UserArray(a)
>>> b
UserArray([[0,1,2],[3,4,5],[6,7,8]], 'l')
>>> b[0]
UserArray([0,1,2], 'l')
>>> b[0,2]
2
>>> b[0,2:3]
UserArray([2], 'l')
If that's correct, then I guess that's what subclasses of UserArray
should do, although it's not exactly intuitive. Also, should there be
an _as_scalar method in UserArray which returns a scalar in the cases
where self.array is rank-0?
-david
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Feb 20 20:12:08 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Tue, 20 Feb 1996 15:12:08 -0500
Subject: [PYTHON MATRIX-SIG] Functions and names
Message-ID: <199602202012.PAA17939@cyclone.ERE.UMontreal.CA>
I have made an attempt to put some order into the array functions. I
have made a list of functions that should be available and proposed a
name for each function. Until now I have covered constructors
and structural functions; the rest will follow later, assuming
I am not hit by too many 16-ton weights before ;-)
Lines that describe new or modified operations are marked with a
'+'. Comments are at the end of the file.
As I have suggested before, it is useful to have an additional module
defining abbreviations and/or more meaningful names for often-used
combinations. These should not be part of the standard module to
reduce name space pollution and learning overhead, but they should
nevertheless be part of the standard distribution to ensure
uniformity. I'll include recommended abbreviations in the respective
sections.
Comments of any kind are of course welcome.
Constructors
============
1) Construct an array from an arbitrary sequence object or a function:
+ array(sequenceOrFunction, shape=None, type=None)
Returns an array of the given shape and type using the
elements from the flattened sequence. If not all
elements of the sequence are used, the rest is discarded.
If more elements are needed, the sequence is reused from
the beginning.
If a callable object is passed instead of a sequence,
it is called for each element with its indices as
parameters.
Calling the current constructor oldArray(), the new one
behaves like
def array(sequenceOrFunction, shape=None, type=None):
if isSequence(sequenceOrFunction):
a = oldArray(sequence, type)
if shape is not None:
a = copyAndReshape(a, shape)
return a
else:
return fromFunction(map(shape), sequenceOrFunction)
This constructor replaces the current constructors
array(), copyAndReshape(), and fromFunction(), and
makes special cases like zeros() so easy that they
don't have to be standard functions any more.
2) Construct a rank-1 array of equally spaced points:
+ arrayRange(x1, x2=None, x3=None)
Currently: function arange() does exactly the same.
Abbreviations:
zeros(n1,n2,...) equals array(0,(n1,n2,...))
ones(n1,n2,...) equals array(1,(n1,n2,...))
unitMatrix(n) equals array([1]+n*[0], (n,n))
arange equals arrayRange
Structural array operations
===========================
1) Selecting subarrays
1.1) Selecting on a "raster" along each coordinate:
Done by indexing with integers and slices
1.2) Selecting arbitrary elements specified by an array of indices i:
+ take(a, i, axis=0)
Conditions: i must be of an integer array type,
minimum.reduce(ravel(i)) >= 0,
maximum.reduce(ravel(i)) < a.shape[axis]
Currently: method a.take(i, axis=0), i can be of any type, but
must have rank 1.
1.3) Selecting the diagonal, i.e. those values where the indices
along two axes are equal (see comment 1):
+ diagonal(a, axis1=0, axis2=1)
Conditions: axis1 < len(a.shape), axis2 < len(a.shape),
a.shape[axis1] == a.shape[axis2]
2) Rearranging array elements
2.1) Changing the shape
2.1.1) General reshaping:
+ reshape(a,shape)
shape can be of any non-nested sequence type
Condition: multiply.reduce(shape) == multiply.reduce(a.shape)
Currently: method a.reshape(shape), shape must be a tuple
2.1.2) Reshaping to a rank-1 array:
ravel(a)
equivalent to reshape(a,(multiply.reduce(a.shape),))
2.1.3) Combining a group of axes into one axis (see comment 2):
+ a[i1, i2, ......, i3, i4] (see comment 3)
The double ellipsis works similar to the single one, but
contracts all the axes covered into a single axis.
2.1.4) Adding an axis of length 1:
a[..., NewAxis, ...]
2.2) Transposition:
+ transpose(a, axes)
axes is a non-nested sequence of non-negative integers
with maximum.reduce(axes) < len(a.shape)
Currently: method a.transpose(axes) does the same, but
there is a bug that sometimes gives wrong results if
an axis is used more than once in the list. It also
insists that len(axes) == len(a.shape), although there
is no need for this restriction.
3) Replicating and combining array elements
3.1) Replicating elements along an axis:
+ repeat(a, n, axis=0)
n is a non-nested integer sequence with len(n) == a.shape[axis]
Each element in a is repeated as often as indicated by
the corresponding element in n. The length of the result
along the specified axis is add.reduce(n).
Currently: function compress(n, a, axis=0) is a special case
limited to boolean (0/1) elements in n.
3.2) Concatenation of arrays along an axis:
+ concatenate((a1,a2,a3,...), axis=0)
Condition: all arrays must have the same shape for the
remaining axes.
Currently: method a.concat(a1,a2,a3,...) works only along
first axis.
3.3) Concatenation of arrays along a new axis:
Can be done by combining concatenate() and indexing with NewAxis.
(see comment 4)
Abbreviation:
reverse(a) = a[::-1]
Comments
========
1) APL uses a special case of transposition for selecting a
diagonal, but this is very confusing.
2) In J this is done by using ravel() with reduced rank, but
as long as we don't have functions with bounded ranks,
we need a special function.
3) I tried to find a construction that does not require
yet another syntax and this is the best I could think
of. But I am not particularly attached to it, so feel
free to think of something better.
4) APL does this with fractional indices, which is probably
the weirdest feature in APL.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Feb 20 20:15:39 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Tue, 20 Feb 1996 15:15:39 -0500
Subject: [PYTHON MATRIX-SIG] returning scalars or rank-0 arrays..
In-Reply-To: <199602201634.LAA21501@maigret> (da@maigret.cog.brown.edu)
Message-ID: <199602202015.PAA18143@cyclone.ERE.UMontreal.CA>
One issue I've come accross is the "what do I return, a rank-0 entity or
a scalar?". My understanding of the current scheme is that indexing
returns scalars, while slicing returns ranked entities, even if that
rank is 0. Is that correct?
I am not sure what the current implementation does, but the idea was
that rank-0 arrays should always automatically be converted to
scalars.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Tue Feb 20 20:22:09 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Tue, 20 Feb 96 15:22:09 EST
Subject: [PYTHON MATRIX-SIG] returning scalars or rank-0 arrays..
In-Reply-To: <199602201634.LAA21501@maigret> (da@maigret.cog.brown.edu)
Message-ID: <9602202022.AA17263@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
From: da@maigret.cog.brown.edu (David Ascher)
One issue I've come accross is the "what do I return, a rank-0 entity or
a scalar?". My understanding of the current scheme is that indexing
returns scalars, while slicing returns ranked entities, even if that
rank is 0. Is that correct?
>>> a = arange(9).reshape(3,3)
>>> b = UserArray(a)
>>> b
UserArray([[0,1,2],[3,4,5],[6,7,8]], 'l')
>>> b[0]
UserArray([0,1,2], 'l')
>>> b[0,2]
2
>>> b[0,2:3]
UserArray([2], 'l')
If that's correct, then I guess that's what subclasses of UserArray
should do, although it's not exactly intuitive. Also, should there be
an _as_scalar method in UserArray which returns a scalar in the cases
where self.array is rank-0?
Note: UserArray([2], 'l') is a rank 1 array of shape (1,). It is not
a rank 0 array. According to the standard python notation, a slice
does not reduce the dimensionality of an array, even if it selects
only one element. Scalars are ALWAYS returned as python scalars
(unless you explictitly ask for array(3.14)).
In general, a user should never manage to produce a UserArray or a C
array of rank-0, unless they somehow specifically ask for it.
Consider the following using standard python lists:
>>> b = [1,2,3]
>>> b[1:2]
[2]
>>> b[1]
2
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Tue Feb 20 20:29:05 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Tue, 20 Feb 96 15:29:05 EST
Subject: [PYTHON MATRIX-SIG] Functions and names
In-Reply-To: <199602202012.PAA17939@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca)
Message-ID: <9602202029.AA17288@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
I've read both Paul's and Konrad's proposal's for new naming/function
schemes. I just want to make it clear that I would have no objections
to implementing any part of either of these schemes, but I only intend
to go to the effort if I can see that some consensus is developing on
this list for one particular proposal and I'm personally going to stay
out of most of that discussion.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From chris.chase@jhuapl.edu Tue Feb 20 22:33:19 1996
From: chris.chase@jhuapl.edu (Chris Chase S1A)
Date: Tue, 20 Feb 1996 17:33:19 -0500
Subject: [PYTHON MATRIX-SIG] Functions and names
In-Reply-To: <199602202012.PAA17939@cyclone.ERE.UMontreal.CA>
References: <199602202012.PAA17939@cyclone.ERE.UMontreal.CA>
Message-ID: <199602202221.RAA01153@python.org>
>>>>> "Konrad" == Konrad HINSEN <hinsenk@ere.umontreal.ca> writes:
Konrad> Comments of any kind are of course welcome.
I like Konrad's other suggestions along with those already suggested
by Paul. Paul seems to have a well thought-out naming convention.
I would like to make some comments and additions below:
Konrad> Constructors
Konrad> ============
Konrad> 1) Construct an array from an arbitrary sequence object or a function:
Konrad> + array(sequenceOrFunction, shape=None, type=None)
Konrad> Returns an array of the given shape and type using the
Konrad> elements from the flattened sequence.
I must admit that for constructing matrices the nested list argument
seemed natural. This version does not allow it. Otherwise, I like
its additional functionality.
Konrad> Structural array operations
Konrad> ===========================
Konrad> 1) Selecting subarrays
Support for mapped indexing (like the Tela language) would be useful.
Perhaps it should be an attribute so as to allow assignment:
print a.mapped[c1,c2,c3]
a.mapped[c1,c2,c3] = b
c1,c2,c3 index vectors that must have the same shape when converted to
an array. Elements of 'a' are chosen by the indexes c1[i],c2[i],c3[i]
and the result has the same shape as the shape of the index vectors.
Think of the index vectors as being coordinates for some space (I use
this style of indexing for paths in two-dimensional image maps or
three-dimensional volume data). For assignment 'b' must have the same
shape.
Konrad> 1.2) Selecting arbitrary elements specified by an array of indices i:
Konrad> + take(a, i, axis=0)
Konrad> Conditions: i must be of an integer array type,
Konrad> minimum.reduce(ravel(i)) >= 0,
Konrad> maximum.reduce(ravel(i)) < a.shape[axis]
As Paul suggested, perhaps gather() is a better name than take()
(gather is a name used in vector libraries). But it really does not
matter very much to me.
There needs to be a dual capability for assigning the items specified by
an array of indices (a completely general form of product indexing).
Perhaps something like:
a.index[1,i,j] = b
with i and j index sequences applied to the second and third
dimensions. Here, the 'index' attribute permits index vectors that
are scalars, sequences or slices. There is no equivalent to take()
since all dimensions must treated simultaneously. Perhaps a better
name would be 'subarray'. In fact, such an 'subarray' attribute could
also be used for selection, only it would not return a reference but a
copy and always return the same rank as 'a' (i.e. does reduce the rank
for scalar indexes). If it was only being used for assignment it
could be called "scatter" (the dual of gather).
An insertion method for inserting a subarray into a larger array would
also be useful. Something like:
a.insert(b, c1, c2, c3)
or
a.insert[c1,c2,c3] = b
Perhaps someone can come up with a better idea. The concept is:
'b' is inserted into 'a' starting at index c1,c2,c3. The operation
overwrites elements of 'a'. It is more convenient then using slice
notation which would require one to compute the upper limit on the
slices. It could have different behaviors by using a keyword argument
for when 'b' does not fit within the dimensions of 'a' (e.g. truncate,
wrap around or signal error.)
Konrad> 2) Rearranging array elements
Konrad> 2.1.2) Reshaping to a rank-1 array:
Konrad> ravel(a)
Konrad> equivalent to reshape(a,(multiply.reduce(a.shape),))
flatten() could also be a possible name. Why should this be a
function rather than a method or attribute? For example,
a.flat == ravel(a)
but a.flat[s] would allow subscription/assignment of 'a' in flattened
form using a scalar, sequence object, or a slice. Of course a
combination of take() and ravel() and subscripting can accomplish
this. However, flat() and subarray() as methods could completely
replace ravel() and take() functions with the ability to simply support
flattened or product indexing subscription or assignment.
Konrad> 2.1.3) Combining a group of axes into one axis (see comment 2):
Konrad> + a[i1, i2, ......, i3, i4] (see comment 3)
Konrad> The double ellipsis works similar to the single one, but
Konrad> contracts all the axes covered into a single axis.
I do not like this. Perhaps an attribute 'collapse' would work? E.g.
a.collapse[i1, i2, ..., i3, i4]
collapse would interpret '...' to mean contract the axes. Otherwise
it behaves exactly like normal subscripting. This is not necessarily
a good idea either.
Konrad> 2.1.4) Adding an axis of length 1:
Konrad> a[..., NewAxis, ...]
Of course the "..." are notational and have nothing to do with ellipses
indexes. Otherwise, this does not make sense.
Konrad> 2.2) Transposition:
Konrad> + transpose(a, axes)
Konrad> axes is a non-nested sequence of non-negative integers
Konrad> with maximum.reduce(axes) < len(a.shape)
Yorick has a nice general form of the transpose() function. It allows
a variable length argument list where the arguments are cyclic
permutations of the indexes. It is actually a little more general.
Rather than explain here, a description is available in the Yorick
online docs at:
ftp://icf.llnl.gov/pub/Yorick/html/yorick_49.html#SEC49
Chris
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Tue Feb 20 22:38:41 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Tue, 20 Feb 1996 17:38:41 -0500
Subject: [PYTHON MATRIX-SIG] Functions and names
In-Reply-To: <199602202220.RAA23965@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Tue, 20 Feb 1996 17:33:19 -0500)
Message-ID: <199602202238.RAA24811@cyclone.ERE.UMontreal.CA>
I must admit that for constructing matrices the nested list argument
seemed natural. This version does not allow it. Otherwise, I like
its additional functionality.
Of course this is still allowed! That was what I meant with the
somewhat cryptic comment that the shape is inferred from the sequence
if not explicitly specified.
Support for mapped indexing (like the Tela language) would be useful.
Perhaps it should be an attribute so as to allow assignment:
Could you give an example for an application where this would
be useful?
Konrad> 1.2) Selecting arbitrary elements specified by an array of indices i:
Konrad> + take(a, i, axis=0)
Konrad> Conditions: i must be of an integer array type,
Konrad> minimum.reduce(ravel(i)) >= 0,
Konrad> maximum.reduce(ravel(i)) < a.shape[axis]
As Paul suggested, perhaps gather() is a better name than take()
(gather is a name used in vector libraries). But it really does not
matter very much to me.
I explicitly avoided "gather" for two reasons:
1) It is not a particularly clear description of what is happening.
2) What I describe is not the exact equivalent of what you
find in most vector libraries, because the index array can
have a higher rank than 1.
There needs to be a dual capability for assigning the items specified by
an array of indices (a completely general form of product indexing).
I agree that this would be nice, but if I remember correctly there
was some problem with putting this into indexing.
An insertion method for inserting a subarray into a larger array would
also be useful. Something like:
The APL way of doing this is to use what I call repeat() to make
room for the insertion and then assign to this part. Since this
is not a very frequent operation, I don't think we need a special
function for it.
flatten() could also be a possible name. Why should this be a
function rather than a method or attribute? For example,
a.flat == ravel(a)
As I pointed out in an earlier message, we should better avoid methods
for array operations except for very specialized ones (like byteswap
or typecode). There is simply no clean way to decide what should be
a method and what a function, and besides all operations coded in
Python have to be functions anyway. (This, by the way, is the
current distinction: everything implemented in C is a method, everything
else is a function. I think we agree that this is the worst possible
distinction.)
but a.flat[s] would allow subscription/assignment of 'a' in flattened
form using a scalar, sequence object, or a slice. Of course a
So will a function, if it doesn't copy the array but returns a
reference.
Konrad> 2.1.3) Combining a group of axes into one axis (see comment 2):
Konrad> + a[i1, i2, ......, i3, i4] (see comment 3)
Konrad> The double ellipsis works similar to the single one, but
Konrad> contracts all the axes covered into a single axis.
I do not like this. Perhaps an attribute 'collapse' would work? E.g.
a.collapse[i1, i2, ..., i3, i4]
If that can be done, fine. So what should the value of "collapse" be
before subscripting? I can't see a way to make this work.
Konrad> 2.1.4) Adding an axis of length 1:
Konrad> a[..., NewAxis, ...]
Of course the "..." are notational and have nothing to do with ellipses
indexes. Otherwise, this does not make sense.
Of course. I hadn't thought about this possible confusion.
Yorick has a nice general form of the transpose() function. It allows
a variable length argument list where the arguments are cyclic
permutations of the indexes. It is actually a little more general.
I think this is close to what I wanted to describe. I'll have
to look up the Yorick documentation...
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From da@maigret.cog.brown.edu Wed Feb 21 16:12:15 1996
From: da@maigret.cog.brown.edu (David Ascher)
Date: Wed, 21 Feb 1996 11:12:15 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] Functions and names
In-Reply-To: <199602202238.RAA24811@cyclone.ERE.UMontreal.CA> from "Konrad HINSEN" at Feb 20, 96 05:38:41 pm
Message-ID: <199602211612.LAA24573@maigret>
> An insertion method for inserting a subarray into a larger array would
> also be useful. Something like:
>
> The APL way of doing this is to use what I call repeat() to make
> room for the insertion and then assign to this part. Since this
> is not a very frequent operation, I don't think we need a special
> function for it.
Actually, when I saw Chris' description of that I thought "yeah!", since
I can see myself using it quite a bit. But I guess I can always write
my own version.
--da
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dredish@CS.cmu.edu Wed Feb 21 17:59:27 1996
From: dredish@CS.cmu.edu (David Redish)
Date: Wed, 21 Feb 1996 12:59:27 -0500
Subject: [PYTHON MATRIX-SIG] MatLab
Message-ID: <3626.824925567@GS151.SP.CS.CMU.EDU>
I've just downloaded the 0.34 Numeric extension (looks really good so
far -- compiled out of the box for HPUX), but MLab.py does not seem to
work.
e.g.
>>> import MLab
>>> MLab.tri(5)
Traceback (innermost last):
File "<stdin>", line 1, in ?
File "/users/adr/lib/python/numeric/MLab.py", line 55, in tri
m = greaterEqual(subtract.outer(mrange(N), mrange(M)), -k)
NameError: greaterEqual
I tried adding "From Numeric import *" to MLab.py
and then I get
>>> import MLab
>>> MLab.tri(5)
Traceback (innermost last):
File "<stdin>", line 1, in ?
File "/users/adr/lib/python/numeric/MLab.py", line 56, in tri
m = greaterEqual(subtract.outer(mrange(N), mrange(M)), -k)
AttributeError: outer
Any help would be appreciated.
thanx
adr
------------------------------------------------------------
David Redish Computer Science Department CMU
graduate student Neural Processes in Cognition Training Program
Center for the Neural Basis of Cognition (CNBC)
http://www.cs.cmu.edu/Web/People/dredish/home.html
------------------------------------------------------------
maintainer, CNBC website:
http://www.cs.cmu.edu/Web/Groups/CNBC
maintainer, Cog-Neuro Sites on the Internet, courtesy CNBC:
http://www.cs.cmu.edu/Web/Groups/CNBC/other
maintainer, NIPS*96 website:
http://www.cs.cmu.edu/Web/Groups/NIPS
------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From da@maigret.cog.brown.edu Wed Feb 21 20:18:31 1996
From: da@maigret.cog.brown.edu (David Ascher)
Date: Wed, 21 Feb 1996 15:18:31 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] MatLab
In-Reply-To: <3626.824925567@GS151.SP.CS.CMU.EDU> from "David Redish" at Feb 21, 96 12:59:27 pm
Message-ID: <199602212018.PAA25500@maigret>
> I've just downloaded the 0.34 Numeric extension (looks really good so
> far -- compiled out of the box for HPUX), but MLab.py does not seem to
> work.
I've hacked a version of MLab which mostly works -- no guarantees.
--da
"""Matlab(tm) compatibility functions.
This will hopefully become a complete set of the basic functions available in
matlab. The syntax is kept as close to the matlab syntax as possible. One
fundamental change is that the first index in matlab varies the fastest (as in
FORTRAN). That means that it will usually perform reductions over columns,
whereas with this object the most natural reductions are over rows. It's perfectly
possible to make this work the way it does in matlab if that's desired.
"""
from Numeric import *
# Elementary Matrices
# zeros is from matrixmodule in C
# ones is from Numeric.py
## This should be replaced by Paul Dubois' URNG very soon now.
import Ranf
def rand(*args):
"""rand(d1,...,dn, typecode='d') returns a matrix of the given dimensions
which is initialized to random number in the range [0,1).
"""
return Ranf.random_sample(args)
def eye(N, M=None, k=0, typecode=None):
"""eye(N, M=N, k=0, typecode=None) returns a N-by-M matrix where the
k-th diagonal is all ones, and everything else is zeros.
"""
if M == None: M = N
if type(M) == type('d'):
typecode = M
M = N
if (typecode == None):
m = outer(subtract, arange(N), arange(M)).equal(-k)
else:
i = arange(N, typecode=typecode)
m = outer(subtract, i, range(M)).equal(-k)
return m
def tri(N, M=None, k=0, typecode=None):
if M == None: M = N
if type(M) == type('d'):
typecode = M
M = N
if (typecode == None):
m = outer(subtract, arange(N), arange(M)).greaterEqual(-k)
else:
i = arange(N, typecode=typecode)
m = outer(subtract, i, arange(M)).greaterEqual(-k)
return m
# Matrix manipulation
def diag(v, k=0):
s = v.shape
if len(s)==1:
n = s[0]+abs(k)
if k > 0:
v = v.concat(zeros(k, v.typecode))
elif k < 0:
v = zeros(-k, v.typecode).concat(v)
return multiply[[1,0]](eye(n, k=k), v)
elif len(s)==2:
v = add.reduce(eye(s[0], s[1], k=k)*v)
if k > 0: return v[:-k]
elif k < 0: return v[-k:]
else: return v
def fliplr(m):
return m[:, ::-1]
def flipud(m):
return m[::-1]
# reshape(x, m, n) is not used, instead use reshape(x, (m, n))
def rot90(m, k=1):
k = k % 4
if k == 0: return m
elif k == 1: return m.transpose()[::-1,::-1]
elif k == 2: return fliplr(m)[::-1,::-1]
elif k == 3: return fliplr(m.transpose())
def tril(m, k=0):
return tri(m.shape[0], m.shape[1], k=k, typecode=m.typecode())*m
def triu(m, k=0):
return (1-tri(m.shape[0], m.shape[1], k-1, m.typecode()))*m
# Data analysis
# Basic operations
def max(m):
return maximum.reduce(m)
def min(m):
return minimum.reduce(m)
# Actually from BASIS, but it fits in so naturally here...
def ptp(m):
return max(m)-min(m)
def mean(m):
return add.reduce(m)/len(m)
# sort is done in C but is done row-wise rather than column-wise
def msort(m):
return sort(m.transpose()).transpose()
def median(m):
return msort(m)[m.shape[0]/2]
def std(m):
mu = mean(m)
return sqrt(add.reduce(pow(m-mu,2)))/sqrt(len(m)-1)
def sum(m):
return add.reduce(m)
def cumsum(m):
return add.accumulate(m)
def prod(m):
return multiply.reduce(m)
def cumprod(m):
return multiply.accumulate(m)
def trapz(y, x=None):
"""Integrate f using the trapezoidal rule, where y is f(x).
"""
if x == None: d = 1
else: d = diff(x)
return sum(d * (y[1:]+y[0:-1])/2)
def diff(x, n=1):
"""Discrete difference approximation to the derivative
"""
if n > 1:
return diff(x[1:]-x[:-1], n-1)
else:
return x[1:]-x[:-1]
def dot(x, y):
return add.reduce(x*y)
def corrcoef(x, y=None):
"""The correlation coefficients
"""
c = cov(x, y)
d = diag(c)
return c/sqrt(outer(multiply, d,d))
def cov(m,y=None):
if y != None: m = array([m,y], m.typecode())
mu = mean(m)
sum_cov = 0.0
for v in m:
sum_cov = sum_cov+outer(multiply, v,v)
return (sum_cov-len(m)*outer(multiply,mu,mu))/(len(m)-1)
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Wed Feb 21 20:26:29 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Wed, 21 Feb 96 15:26:29 EST
Subject: [PYTHON MATRIX-SIG] Version 0.35
Message-ID: <9602212026.AA22323@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Well, I warned people that v0.34 would be buggy and people did a great
job of finding bugs. Thanks to Konrad Hinsen, Perry Stoll and David
Ascher in particular, v0.35 is now a little bit more solid.
It's in the usual place ftp://sls-ftp.lcs.mit.edu/pub/jjh/patch-0.34-0.35
or
ftp://sls-ftp.lcs.mit.edu/pub/jjh/NumericPython-0.35.tar.gz
Warning: I did something to break the TestSuite fairly recently, so
don't expect it to work for you. I thought it was more important to
get these bug fixes out to people quickly than to dive into the
TestSuite right now.
Enjoy - Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From et@appl-math.tu-muenchen.de Wed Feb 21 23:47:09 1996
From: et@appl-math.tu-muenchen.de (Thomas Schwaller)
Date: Thu, 22 Feb 1996 00:47:09 +0100
Subject: [PYTHON MATRIX-SIG] Announement: trimodule.tgz
Message-ID: <9602220047.ZM18768@hamster.appl-math.tu-muenchen.de>
Hi all,
Here's a first release of trimodule, a standalone module for triangulation
of 2-dimensional domains.You can get it at:
ftp://ftp.appl-math.tu-muenchen.de/pub/et/trimodule.tgz
I also sent it to Jim by mail and it should appear in the usual place at
the matrixmodule site.
Read trimodule.html (With Netscape it looks nice) to get a first impression,
(look especially at the last pictures)
Hope you enjoy it. I'm sorry for not responding immediatly to incoming mail,
I read them just every second or third day at the moment, so I'm not that
reactive! :-)
I tried to do my best documenting the stuff I did, but be prepared
for crashes when doing your first tests. The Input has to be correct!
BTW. The Lapack Module of Doug works very fine. On My Linux box
I use it with a static double Lapack library only (also built statically
into the Interpreter, not as dynamically loadable module for maximal speed)
When using it dynamically the import time is not as good, as usual.
The big documentation modules are are a problem. One should put the docstrings
in *.py modules and load them when needed, I think that would be the best
solution (one *.py documentation module for each function).
So we have a small c-module, speed and the documentation)
Tom
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dredish@CS.cmu.edu Thu Feb 22 17:13:40 1996
From: dredish@CS.cmu.edu (David Redish)
Date: Thu, 22 Feb 1996 12:13:40 -0500
Subject: [PYTHON MATRIX-SIG] Applying a general function to an array
Message-ID: <4504.825009220@GS151.SP.CS.CMU.EDU>
Is it possible to apply a general python function (that takes one
input and produces one output to an array? For example, I'd like to
have a python function (implemented in C), "y = f(x)" and then be able
to say Y = f(X) and get Y set to an array where each element in Y is f
applied to the corresponding element in X.
I guess I could get the shape, ravel the array, apply the function
using map, convert the list to an array, and reshape it, but that
seems like it would be very slow.
In a similar vein, is there documentation for
1. the Array C API
2. ofuncs
While, I'm at it, I'd like to complement Paul Dubois and David Ascher
for their documentation efforts. It's been incredibly helpful. (And
of course to Jim and Konrad for all their work putting this together.)
adr
------------------------------------------------------------
David Redish Computer Science Department CMU
graduate student Neural Processes in Cognition Training Program
Center for the Neural Basis of Cognition (CNBC)
http://www.cs.cmu.edu/Web/People/dredish/home.html
------------------------------------------------------------
maintainer, CNBC website:
http://www.cs.cmu.edu/Web/Groups/CNBC
maintainer, Cog-Neuro Sites on the Internet, courtesy CNBC:
http://www.cs.cmu.edu/Web/Groups/CNBC/other
maintainer, NIPS*96 website:
http://www.cs.cmu.edu/Web/Groups/NIPS
------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Thu Feb 22 17:35:06 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Thu, 22 Feb 96 12:35:06 EST
Subject: [PYTHON MATRIX-SIG] Applying a general function to an array
In-Reply-To: <4504.825009220@GS151.SP.CS.CMU.EDU> (message from David Redish on Thu, 22 Feb 1996 12:13:40 -0500)
Message-ID: <9602221735.AA28770@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
An example of the sort of function you want to do this with would be
very helpful in answering this. Many python functions already work as
you describe.
ie. (This is from my memory of how RBF NN's work)
Say I have a Radial Basis Function Neural Net and I want the user to
be able to give a python function the input RBF (actually this would
be nice thing to have):
The standard RBF function is (from memory, I last worked on these
things years ago):
def gaussian(x, A, B, C):
return A*exp(B*(x-C)**2)
This function wil normally just work for any reasonable combination of
1d, 2d arrays and scalars.
> In a similar vein, is there documentation for
> 1. the Array C API
arrayobject.h provides some minimal documentation on this. I'd look
at the various modules that people have produced for examples.
> 2. ofuncs
I doubt that there will be much need for additional ofuncs in the
future. They are currently only really useful for binary functions
that need to operate over a large variety of array types, and need to
be applied in numerous interesting ways (pseudo indices, reduction,
etc.) If you have something that you really think should be an ofunc,
I'd like to know some more of the details.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dredish@CS.cmu.edu Thu Feb 22 18:02:50 1996
From: dredish@CS.cmu.edu (David Redish)
Date: Thu, 22 Feb 1996 13:02:50 -0500
Subject: [PYTHON MATRIX-SIG] Applying a general function to an array
In-Reply-To: Your message of "Thu, 22 Feb 1996 12:35:06 EST."
<9602221735.AA28770@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <368.825012170@GS151.SP.CS.CMU.EDU>
>An example of the sort of function you want to do this with would be
>very helpful in answering this. Many python functions already work as
>you describe.
This is fine for already-coded python functions (like exp), but what I
really want to know how to add new C-coded functions.
For example, I have a gaussian function coded up in C that takes care
of normalization, random-sampling with a gaussian distribution, etc.
So I'd like to be able to apply that function to each element of an
array. I could recode the function in python, but that would be much
slower. Also, it's already nicely coded-up as a C-coded python object
and software-reuse is the word here, right?
e.g. I can currently do:
x = 10
G = Gaussian(wt, mean, variance)
y = G(x)
A = arange(10)
G = Gaussian(wt, mean, variance)
Y = G(A)
where G is a C-coded python object that defines a __call__ function.
thanx
adr
PS. I am using these for NN's similar to RBFs. (:
------------------------------------------------------------
David Redish Computer Science Department CMU
graduate student Neural Processes in Cognition Training Program
Center for the Neural Basis of Cognition (CNBC)
http://www.cs.cmu.edu/Web/People/dredish/home.html
------------------------------------------------------------
maintainer, CNBC website:
http://www.cs.cmu.edu/Web/Groups/CNBC
maintainer, Cog-Neuro Sites on the Internet, courtesy CNBC:
http://www.cs.cmu.edu/Web/Groups/CNBC/other
maintainer, NIPS*96 website:
http://www.cs.cmu.edu/Web/Groups/NIPS
------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Thu Feb 22 18:25:04 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Thu, 22 Feb 96 13:25:04 EST
Subject: [PYTHON MATRIX-SIG] Applying a general function to an array
In-Reply-To: <368.825012170@GS151.SP.CS.CMU.EDU> (message from David Redish on Thu, 22 Feb 1996 13:02:50 -0500)
Message-ID: <9602221825.AA29237@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
I assume that you have a python function of the form:
PyObject *gaussian_call(PyObject *self, PyObject *args) {
double x, r;
PyArg_ParseTuple("d", &x);
r = calculate(self, x);
return Py_BuildValue("d", r);
}
if you convert this to the following, it will work on arbitrary arrays
PyObject *gaussian_call(PyObject *self, PyObject *args) {
int n, i;
PyObject *op;
PyArrayObject *ap, *rp;
PyArg_ParseTuple("O", &op);
ap = PyArray_ContiguousFromObject(op, PyArray_DOUBLE, 0, 0);
rp = PyArray_FromDims(ap->nd, ap->dimensions, PyArray_DOUBLE);
n = PyArray_Size(ap);
for(i=0; i<n; i++) {
((double *)rp->data)[i] =
calculate(self, ((double *)ap->data)[i]);
}
return PyArray_Return(rp);
}
I admit this looks a little bit more complicated, but this new version
will work on any size/shape of array as well as on python scalars.
Any other solution will involve converting the elements of the array
into and out of python objects, and the overhead of doing this would
probably totally swap the computation.
I don't know if this is what you're looking for, but I hope it's helpful.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Thu Feb 22 18:24:27 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Thu, 22 Feb 1996 13:24:27 -0500
Subject: [PYTHON MATRIX-SIG] Applying a general function to an array
In-Reply-To: <4504.825009220@GS151.SP.CS.CMU.EDU> (message from David Redish on Thu, 22 Feb 1996 12:13:40 -0500)
Message-ID: <199602221824.NAA05399@cyclone.ERE.UMontreal.CA>
Is it possible to apply a general python function (that takes one
input and produces one output to an array? For example, I'd like to
have a python function (implemented in C), "y = f(x)" and then be able
to say Y = f(X) and get Y set to an array where each element in Y is f
applied to the corresponding element in X.
There is no real support for this right now. It has to be added, and I
am trying to make such things part of my function set proposal
(progressingly slowly due to lots of other work). If your function f
is written in terms of functions from umath, then it should work
automatically, but in the case of a C function or a more complicated
Python function (with tests, loops, etc.), you have no other way than
to write explicit for-loops or some messy map() application.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dubois@kristen.llnl.gov Thu Feb 22 21:03:58 1996
From: dubois@kristen.llnl.gov (P. Dubois)
Date: Thu, 22 Feb 1996 13:03:58 -0800
Subject: [PYTHON MATRIX-SIG] URNG-2.2
Message-ID: <9602222103.AA11572@kristen.llnl.gov>
URNG-2.2 is available at ftp-icf.llnl.gov/pub/basis/URNG-2.2.tar.gz.
This fixes an error in memory management. This was the first C extension
I wrote. I'd like to describe the error to make sure I have it right this
time.
I have a routine that allocates a new "urngobject", and I was using it
like this:
static PyObject *
URNG_CreateGenerator(self, args)
PyObject *self; /* Not used */
PyObject *args;
{
int seed;
PyObject *result;
if (!PyArg_ParseTuple(args, "i", &seed))
return NULL;
result = (PyObject *) newurngobject(seed);
return result;
}
I was getting a core dump when the program ended if I created such an object.
As a newbie, I assumed I had the reference count stuff messed up, so
I tried returning Py_BuildValue("O", result) instead of just result. I
stopped dumping core and went on. But this is wrong, of course, since
I now have a reference count of two on the object and it is never
deleted.
I had made the skeleton using modulator and had not checked the "dealloc"
option. When I add such a routine:
static void
urng_dealloc(self)
urngobject *self;
{
PyMem_DEL(self);
}
and put it in the object's table, all is well.
So is this right, you *have* to supply a dealloc method when you write an
extension object? If so, maybe modulator shouldn't give you a choice.
Or, am I still confused?
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Thu Feb 22 22:29:40 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Thu, 22 Feb 1996 17:29:40 -0500
Subject: [PYTHON MATRIX-SIG] URNG-2.2
In-Reply-To: <9602222103.AA11572@kristen.llnl.gov> (dubois@kristen.llnl.gov)
Message-ID: <199602222229.RAA27584@cyclone.ERE.UMontreal.CA>
So is this right, you *have* to supply a dealloc method when you write an
extension object? If so, maybe modulator shouldn't give you a choice.
That is also my understanding. It also makes sense, because Python
can't know how to deallocate your object. It can only keep track
of the reference count and call the dealloc method when it reaches
zero. So even if it were allowed not to have a dealloc method,
this would mean a permanent memory leak.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From chris.chase@jhuapl.edu Fri Feb 23 22:34:13 1996
From: chris.chase@jhuapl.edu (Chris Chase S1A)
Date: Fri, 23 Feb 1996 17:34:13 -0500
Subject: [PYTHON MATRIX-SIG] Functions and names
In-Reply-To: <199602202238.RAA24811@cyclone.ERE.UMontreal.CA>
References: <199602202220.RAA23965@cyclone.ERE.UMontreal.CA>
<199602202238.RAA24811@cyclone.ERE.UMontreal.CA>
Message-ID: <199602232227.RAA20187@python.org>
I apologize that this response is delayed. My regular work takes
priority over my attention to the SIG mailing list.
>>>>> "Konrad" == Konrad HINSEN <hinsenk@ere.umontreal.ca> writes:
Chase> Support for mapped indexing (like the Tela language) would
Chase> be useful. Perhaps it should be an attribute so as to
Chase> allow assignment:
Konrad> Could you give an example for an application where this would
Konrad> be useful?
One application where I use this:
I often use map projections (e.g. azimuthal) to represent regularly
sampled data on a sphere (latitude/longitude or azimuth/elevation
coordinates), i.e. data is a 2D array on a regularly sampled lat/lon
grid. One way to map data onto the map projection is to compute the
lat/lon coordinate of each pixel in the map. Suppose the map is to be
an NxM image for a particular projection and that the NxM arrays 'lat'
and 'lon' contain the latitude and longitude coordinates of each pixel
in map. 'lat' and 'lon' then are converted to indexes according to the
grid sampling.
Then mapped indexing produces the desired map image:
map = data.mapped(lat,lon)
As I mentioned previously, I use this kind of indexing when dealing
with coordinates for 2D image maps or 3D volume data. The coordinates
are often used as part of a transformation (as above) or a path
through the data (i.e. a line-of-sight).
For more info, mapped indexing is supported in both IDL and Tela.
See http://www.geo.fmi.fi/prog/tela/telahelp-5.html#ss5.18.
Chase> There needs to be a dual capability for assigning the items
Chase> specified by an array of indices (a completely general form
Chase> of product indexing).
Konrad> I agree that this would be nice, but if I remember correctly
Konrad> there was some problem with putting this into indexing.
Originally, James Hugunin had implemented general index vectors.
However, he stated that they were removed when he decided to have
array indexing return by reference. Index vectors do not fit well
into this implementation. Returning by reference would require
keeping a copy of the index vector and would require a rewrite of the
current array representation. Besides, I think it is more natural to
return a copy of the indexed elements rather than a reference when
using a general index vector.
When the operation is assignment there is no reason to not support
index vectors since array references are not being created.
Just because general product indexing does not completely fit into
the current implementation is not a good reason to not support it.
Rather than mess around with the current indexing that returns by
reference, I suggested a 'subarray' attribute that would support
general product indexing for assignment and selection (returning a
copy).
Chase> An insertion method for inserting a subarray into a larger
Chase> array would also be useful. Something like:
Konrad> The APL way of doing this is to use what I call repeat() to make
Konrad> room for the insertion and then assign to this part. Since this
Konrad> is not a very frequent operation, I don't think we need a special
Konrad> function for it.
I specifically said that the function I was describing overwrites
elements in the target array rather than making room. True it is not
as frequent of an operation as some others, but it is one that I use
other with matrix linear algebra (manipulating block matrices) and for
cut-away's and slices in 3D volume data. It is a general operation
supported by other languages and not application specific.
Konrad> 2.1.3) Combining a group of axes into one axis (see comment 2):
Konrad> + a[i1, i2, ......, i3, i4] (see comment 3)
Konrad> The double ellipsis works similar to the single one, but
Konrad> contracts all the axes covered into a single axis.
Chase> I do not like this. Perhaps an attribute 'collapse' would
Chase> work? E.g.
Chase> a.collapse[i1, i2, ..., i3, i4]
Konrad> If that can be done, fine. So what should the value of "collapse" be
Konrad> before subscripting? I can't see a way to make this work.
It would have to be an object containing a reference to the array 'a'.
It would have the special property that the ellipses index, '...',
collapses missing dimensions. Otherwise it would act like a normal
array. Since this is a special kind of access to the array object, a
'collapse' method would be better, but then it could not support the
indexing syntax for slices and ellipses. I personally think that the
best solution would be a syntactical symbol like "*", but that idea
was shot down.
Chris
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From chris.chase@jhuapl.edu Sat Feb 24 00:06:19 1996
From: chris.chase@jhuapl.edu (Chris Chase S1A)
Date: Fri, 23 Feb 1996 19:06:19 -0500
Subject: [PYTHON MATRIX-SIG] RubberIndex
In-Reply-To: <199602141831.NAA04933@cyclone.ERE.UMontreal.CA>
References: <199602141643.LAA28371@cyclone.ERE.UMontreal.CA>
<199602141831.NAA04933@cyclone.ERE.UMontreal.CA>
Message-ID: <199602240000.TAA20621@python.org>
I made a suggestion about having array indexing generate an error when
too few indexes are used. Konrad felt that the analogy to lists was
more important:
>>>>> "Konrad" == Konrad HINSEN <hinsenk@ere.umontreal.ca> writes:
Konrad> That wasn't my point either. My point is that since arrays and nested
Konrad> lists are treated as interchangeable in many situations, it makes
Konrad> sense to use this analogy as much as possible and allow an indexing
Konrad> scheme for arrays that matches that for nested arrays. WHich means
Konrad> that it should always be allowed to have just one index, independent
Konrad> of the rank of the array.
For an array 'a' (rank > 1), the current implementation of a[0] adds
no functionality since it is equivalent to a[0,...].
I would like to make another suggestion along the lines my original
one that also handles the purpose of ravel() for both assignment and
selection.
In IDL, Tela and Yorick using a single index or slice defaults to
indexing an array in flattened form. Otherwise, the number of actual
indexes used must equal the rank.
A similar implementation could be supported for the array object.
This obviates 'flat' attribute that I suggested earlier.
a[i] indexes 'a' in flattened form. 'i' can be a scalar or a slice. Then,
a[:] == ravel(a)
and assignment to 'a' in flattened form would also be supported:
a[i] = 2
which eliminates the need for a "flattened" assignment function or for
ravel().
Like other array languages, if more than one index is used without a
rubber index and there are fewer indexes than the array rank then an
error should result.
If 'a' has rank 3, then
a[0] = 1
b = a[0,...]
b = a[0,0,0]
would be valid. But a[0,0] would generate an error. This would be
consistent with the fact that a[0,0,0,0] also generates an error.
This implementation still allows an array to be used anywhere where a
sequence object (e.g. list) is used.
Chris
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Sat Feb 24 14:38:58 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Sat, 24 Feb 1996 09:38:58 -0500
Subject: [PYTHON MATRIX-SIG] Functions and names
In-Reply-To: <199602232226.RAA24062@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Fri, 23 Feb 1996 17:34:13 -0500)
Message-ID: <199602241438.JAA20228@cyclone.ERE.UMontreal.CA>
I often use map projections (e.g. azimuthal) to represent regularly
sampled data on a sphere (latitude/longitude or azimuth/elevation
coordinates), i.e. data is a 2D array on a regularly sampled lat/lon
grid. One way to map data onto the map projection is to compute the
lat/lon coordinate of each pixel in the map. Suppose the map is to be
an NxM image for a particular projection and that the NxM arrays 'lat'
and 'lon' contain the latitude and longitude coordinates of each pixel
in map. 'lat' and 'lon' then are converted to indexes according to the
grid sampling.
Then mapped indexing produces the desired map image:
map = data.mapped(lat,lon)
I think I understand this now. In fact, I was planning to include this
feature in my function proposal, but under "mapping functions" (yet
to be written) rather than "indexing", because it is just a special
case of array mapping with simple indexing as the function to be
applied.
current array representation. Besides, I think it is more natural to
return a copy of the indexed elements rather than a reference when
using a general index vector.
Indeed, but this creates the unpleasant situation that an expression
like a[i,j] returns either a copy or a reference, depending on the
values of i and j. Having references at all is confusing enough, but
this would certainly cause some headaches.
Personally I'd prefer to have standard indexing always return a copy
(that is the safer option) and provide an alternative syntax for
obtaining a reference. Then standard indexing could again allow
general index vectors.
I specifically said that the function I was describing overwrites
elements in the target array rather than making room. True it is not
Then allowing general index vectors in indexed assignments should
be all you need, right?
Chase> I do not like this. Perhaps an attribute 'collapse' would
Chase> work? E.g.
Chase> a.collapse[i1, i2, ..., i3, i4]
Konrad> If that can be done, fine. So what should the value of "collapse" be
Konrad> before subscripting? I can't see a way to make this work.
It would have to be an object containing a reference to the array 'a'.
It would have the special property that the ellipses index, '...',
collapses missing dimensions. Otherwise it would act like a normal
array. Since this is a special kind of access to the array object, a
So effectively you want to introduce a second array type with a slightly
different behaviour. I don't think this is a good idea. There would
be no way to stop users from just writing a.collapse and assign the
result to variables, return it from library functions etc. Suddenly
there are two array types in free circulation that behave almost, but
not quite, identically.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Sat Feb 24 14:50:39 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Sat, 24 Feb 1996 09:50:39 -0500
Subject: [PYTHON MATRIX-SIG] RubberIndex
In-Reply-To: <199602232358.SAA29109@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Fri, 23 Feb 1996 19:06:19 -0500)
Message-ID: <199602241450.JAA20452@cyclone.ERE.UMontreal.CA>
In IDL, Tela and Yorick using a single index or slice defaults to
indexing an array in flattened form. Otherwise, the number of actual
indexes used must equal the rank.
I won't deny that this has some practical advantages, but it violates
the principle that an array is first of all a structured collection
of data with a definite shape. Changing this structure should always
be an explicit operation, not hidden in a special rule for indexing.
For me it still makes more sense to consider an array equivalent
to a nested list with the same structure.
Like other array languages, if more than one index is used without a
rubber index and there are fewer indexes than the array rank then an
error should result.
I certainly agree on that.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Sat Feb 24 18:41:40 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Sat, 24 Feb 1996 13:41:40 -0500
Subject: [PYTHON MATRIX-SIG] Two weeks' experience
Message-ID: <199602241841.NAA28353@cyclone.ERE.UMontreal.CA>
During the last two weeks I have been very busy with my real work, but
most of this has been done in Python, with lots of arrays. I was
helping a colleague to locate a bug in his Fortran code that is very
important for both of us. Python has been incredibly useful for this
task, but I have also discovered a few serious problems, which I would
like to report here. My point is mostly to show how important
practical applications are to find weak points in the current design
and implementation.
My first and most frustrating experience was that it is very difficult
to write code that works for both arrays and scalars (or rank-0
arrays). Sure, a one-line expression using operators and mathematical
formulas is no problem. But as soon as there are conditionals, loops,
etc., the easiest way is often to have two separate functions. All
this is caused mostly by conditional expressions. To check whether all
elements of a are greater than the corresponding elements of b you can
write andLogical.reduce(ravel(a.greater(b))). But if a is a scalar,
this has to be andLogical.reduce(ravel(b.less(a))), and if both a and
b are scalars it must be a > b. I ended up writing a special
comparison function that uses one of these expressions depending on
the type of the arguments.
This shows that no array operations that make sense for rank-0 arrays
and scalars (i.e. most) should be implemented as a method of the array
type. I had reached this conclusion before on the basis of other
arguments, but this practical frustration was much more convincing!
Another problem that cost me two hours to solve was matrix multiplication.
I had a three-dimensional wave function evaluated on a grid, and therefore
stored in the array psi of rank 3. I wanted to apply the operator for kinetic
energy in one direction, which is a rank-2 array, t. My first attempt was
t.matrixMultiply(psi),
and I expected this to do a dot product using the last axis of tx and the
first of psi. But no, all I got was an exception. So I had to write
this function myself, which I still considered a minor exercise. More
than an hour later, I had arrived at the following code:
def dot(a1, a2):
r1 = len(a1.shape)-1
r2 = len(a2.shape)-1
axes = [r1] + range(r1)
a1 = a1.transpose(axes).copy()
f = a1.reshape
a1 = apply(f, a1.shape + r2*(1,))
a2 = a2.copy()
f = a2.reshape
a2 = apply(f, a2.shape[0:1] + r1*(1,) + a2.shape[1:])
return add.reduce(a1*a2)
If you think that this ought to be easier, I agree. The basic idea of
this function is to reshape both arrays to make the axis to be summed
over the first, and to add new axes of length one to make things
match. All the trouble is caused by two unpleasant features of
the method reshape():
1) It works only on contiguous arrays, which makes it necessary
to copy the arguments first.
2) It wants the new shape in the form of several integer arguments,
which causes the weird construction used above to obtain a
rank that is calculated rather than a known constant.
The final operation I want to complain about is fromFunction(). I had
always thought that it calls the supplied function once for each
element, supplying integer indices as arguments. But no, it tries to
be smarter, by calling the function only once with suitably shaped
arrays as arguments. That is of course much more efficient, but it
limits the function to a straightforward sequence of ofuncs, in which
case there is hardly any need to use the fromFunction() constructor.
Most cases I can think of where this constructor makes sense involve
functions with conditional expressions, like my kinetic energy operator:
def tij(i,j):
if i == j:
return pi**2/3
else:
d = i-j
return 2.*(-1.)**d/d**2
So instead of the straightforward and readable expression
t = (hbar**2/(2.*mass*delta**2)) * fromFunction([ngrid,ngrid], tij)
I had to write the very Fortran-like code
t = zeros(ngrid,ngrid)
for i in range(ngrid):
for j in range(ngrid):
t[i,j] = tij(i,j)
t = (hbar**2/(2.*mass*delta**2)) * t
There were a few more minor problems, but I think I can stop now. The
direct consequences of the problems described here will appear in my
proposal for reorganising the array functions, but most of all I'd
like to urge all of you to use what we have for as many applications
you can think of. And we should seriously consider a public alpha
release to get more testers.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From steve@estel.uindy.edu Sat Feb 24 22:40:21 1996
From: steve@estel.uindy.edu (Steve Spicklemire)
Date: Sat, 24 Feb 96 17:40:21 -0500
Subject: [PYTHON MATRIX-SIG] Pretty Printer
Message-ID: <9602242240.AA26596@estel.uindy.edu>
I'm probably doing something wrong but I couldn't
get PrettyPrinter to work with shape (1,) float
arrays. I made this change to mine... (there's
probably a much better way to do this...
but at least I'm back up and running...)
RCS file: PrettyPrinter.py,v
retrieving revision 1.1
retrieving revision 1.2
diff -c5 -r1.1 -r1.2
*** /tmp/,RCSt1026586 Sat Feb 24 17:36:04 1996
--- /tmp/,RCSt2026586 Sat Feb 24 17:36:05 1996
***************
*** 86,96 ****
format = format + str(max_str_len) + '.' + str(precision) + 'e'
if large_exponent: format = format + '3'
item_length = max_str_len + 1
else:
format = '%.' + str(precision) + 'f'
! precision = min(precision, apply(max, tuple(map(lambda x, p=precision,
f=format: _digits(x,p,f),
data))))
max_str_len = len(str(int(max_val))) + precision + 2
if sign: format = '%#+'
else: format = '%#'
--- 86,99 ----
format = format + str(max_str_len) + '.' + str(precision) + 'e'
if large_exponent: format = format + '3'
item_length = max_str_len + 1
else:
format = '%.' + str(precision) + 'f'
! if len(data.shape) == 1 and data.shape[0] == 1:
! precision = min(precision, _digits(data[0],precision,format))
! else:
! precision = min(precision, apply(max, tuple(map(lambda x, p=precision,
f=format: _digits(x,p,f),
data))))
max_str_len = len(str(int(max_val))) + precision + 2
if sign: format = '%#+'
else: format = '%#'
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Sun Feb 25 14:35:58 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Sun, 25 Feb 1996 09:35:58 -0500
Subject: [PYTHON MATRIX-SIG] Pretty Printer
In-Reply-To: <9602242240.AA26596@estel.uindy.edu> (message from Steve Spicklemire on Sat, 24 Feb 96 17:40:21 -0500)
Message-ID: <199602251435.JAA24211@cyclone.ERE.UMontreal.CA>
I'm probably doing something wrong but I couldn't
get PrettyPrinter to work with shape (1,) float
arrays. I made this change to mine... (there's
A better patch is
precision = min(precision, max(tuple(map(lambda x, p=precision,
f=format: _digits(x,p,f),
data))))
There seem to be too many versions of the pretty printer around. I
had assumed that everyone by now should have the latest one, but
appearantly that's not true.
I am even more surprised that there are no more serious complaints.
When I installed version 0.35 of the numerics package, I would
always get error messages when typing "from Numeric import *"
due to recursive imports of Numeric and PrettyPrinter. Am I the
only one with this problem? Or has everyone silently fixed this?
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Sun Feb 25 19:22:17 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Sun, 25 Feb 1996 14:22:17 -0500
Subject: [PYTHON MATRIX-SIG] Updated function list
Message-ID: <199602251922.OAA02089@cyclone.ERE.UMontreal.CA>
Thanks to a rainy weekend, I have completed the proposed function list.
It now covers all operations that I can think of (which doesn't mean
that nothing is missing!). The descriptions of array() and reshape()
have changed with respect to the first draft. Enjoy and ciriticize!
---------------------------------------------------------------------------
This list is an attempt to put some order into the array functions. I
have made a list of functions that should be available and proposed a
name for each function. Lines that describe new or modified operations
are marked with a '+'. Comments are at the end of the file.
As I have suggested before, it is useful to have an additional module
defining abbreviations and/or more meaningful names for often-used
combinations. These should not be part of the standard module to
reduce name space pollution and learning overhead, but they should
nevertheless be part of the standard distribution to ensure
uniformity. I'll include recommended abbreviations in the respective
sections.
The following principles have been applied:
1) Every operation that is not closely related to the implementation
details of the array object is implemented as a function, not
as a method. This is necessary because methods don't work on
scalars, which also serve as rank-0 arrays. It is also necessary
to allow the implementation of every operation in C or Python
without changing the interface. And this decision also ensures
consistency with additional modules (linear algebra etc.); such
modules can only add functions, not methods.
2) There are no functions with a variable number of array arguments.
Such functions are difficult to apply to a number of arrays that
is calculated in the code rather than constant. Instead, a sequence
of arrays is passed. This has the added advantage that a higher-dimensional
array can be passed, implying an iteration over the first axis.
Constructors
============
1) Construct an array from an arbitrary sequence object or a function:
+ array(object, shape=None, type=None)
Creates an array from the given object with a given
shape and type. If the shape and/or the type are not
specified, they are inferred from the data.
The object providing the data can be of the following
types:
1) An object with a method "toArray". This method
is called, and the resulting array is converted to
the given shape and type (if not None).
2) A nested list or tuple, or an array. The shape is inferred
from the nesting, the type from the data, unless
space and/or type are specified.
3) A file object. The file is read as a rank-2
array. If a shape and/or type is specified, the result is
changed to that shape/type.
4) A callable object. This object is called once for
each array element with the indices as arguments.
In reshaping to an explicitly given shape, the input
is reused from the beginning if necessary. It is also
allowed not to use all the input data.
This constructor replaces the current constructors
array(), copyAndReshape(), and fromFunction(), and
makes special cases like zeros() so easy that they
don't have to be standard functions any more.
Note: the current fromFunction() behaves in a subtly
different way: it calls the function only once with
index arrays as arguments. This is equivalent if
the function consists only of operators and ofuncs,
and of course much more efficient, but for most
interesting functions it doesn't work at all. In
this case I prefer generality to efficiency.
2) Construct a rank-1 array of equally spaced points:
+ arrayRange(x1, x2=None, x3=None)
Currently: function arange() does exactly the same.
Abbreviations:
zeros(n1,n2,...) equals array(0,(n1,n2,...))
ones(n1,n2,...) equals array(1,(n1,n2,...))
unitMatrix(n) equals array([1]+n*[0], (n,n))
arange equals arrayRange
Structural array operations
===========================
1) Selecting subarrays
1.1) Selecting on a "raster" along each coordinate:
Done by indexing with integers and slices
1.2) Selecting arbitrary elements specified by an array of indices i:
+ take(a, i, axis=0)
Conditions: i must be of an integer array type,
minimum.reduce(ravel(i)) >= 0,
maximum.reduce(ravel(i)) < a.shape[axis]
Currently: method a.take(i, axis=0), i can be of any type, but
must have rank 1.
1.3) Selecting the diagonal, i.e. those values where the indices
along two axes are equal (see comment 1):
+ diagonal(a, axis1=0, axis2=1)
Conditions: axis1 < len(a.shape), axis2 < len(a.shape),
a.shape[axis1] == a.shape[axis2]
2) Rearranging array elements
2.1) Changing the shape
2.1.1) General reshaping:
+ reshape(a,shape, copy=1)
shape can be of any non-nested sequence type
Condition: multiply.reduce(shape) == multiply.reduce(a.shape)
If copy==1, the returned array is a new copy, otherwise
a reference to a.
Currently: method a.reshape(length1, length2, ...)
This syntax does not provide an easy way to specify
a non-constant shape. Also, the current behaviour is
like copy=0, which can lead to unpleasant surprises
(since otherwise only indexing returns references)
and limits reshaping to contiguous arrays.
2.1.2) Reshaping to a rank-1 array:
+ ravel(a, copy=1)
equivalent to reshape(a,(multiply.reduce(a.shape),), copy)
2.1.3) Combining a group of axes into one axis (see comment 2):
+ a[i1, i2, ......, i3, i4] (see comment 3)
The double ellipsis works similar to the single one, but
contracts all the axes covered into a single axis.
2.1.4) Adding an axis of length 1:
a[..., NewAxis, ...]
2.2) Transposition:
+ transpose(a, axes)
axes is a non-nested sequence of non-negative integers
with maximum.reduce(axes) < len(a.shape)
Currently: method a.transpose(axes) does the same, but
there is a bug that sometimes gives wrong results if
an axis is used more than once in the list. It also
insists that len(axes) == len(a.shape), although there
is no need for this restriction.
3) Replicating and combining array elements
3.1) Replicating elements along an axis:
+ repeat(a, n, axis=0)
n is a non-nested integer sequence with len(n) == a.shape[axis]
Each element in a is repeated as often as indicated by
the corresponding element in n. The length of the result
along the specified axis is add.reduce(n).
Currently: function compress(n, a, axis=0) is a special case
limited to boolean (0/1) elements in n.
3.2) Concatenation of arrays along an axis:
+ concatenate((a1,a2,a3,...), axis=0)
Condition: all arrays must have the same shape for the
remaining axes.
Currently: method a.concat(a1,a2,a3,...) works only along
first axis and is difficult to apply to a variable number
of arguments.
3.3) Concatenation of arrays along a new axis:
Can be done by combining concatenate() and indexing with NewAxis.
(see comment 4)
Abbreviation:
reverse(a) = a[::-1]
Arithmetic and logical operations
=================================
1) Binary elementwise arithmetic operators
+ - * / % **
as functions: add, subtract, multiply, divide, remainder, power
2) Standard "mathematical" functions
arccos, arccosh, arcsin, arcsinh, arctan, arctanh, cos, cosh, exp,
log, log10, sin, sinh, sqrt, tan, tanh
3) Other elementwise arithmetic functions
maximum, minimum, clip
+ conjugate
Currently, conjugate() is implemented as a method both on
complex objects and array objects. It should be available
as a function, because then it can be applied to other
number objects as well. There are many algorithms that work
for both real and complex numbers providing that conjugates
are used here and there. It ought to be possible to implement
these algorithms and make them work for float and complex
objects as well as array objects. The method we have now
doesn't work on real and integer objects.
4) Comparison functions
+ equal(a,b), notEqual(a,b), greater(a,b), greaterEqual(a,b),
+ less(a,b), lessEqual(a,b)
Currently all these are methods. Apart from introducing
an arbitrary asymmetry, this makes it impossible to write
code that works both for scalars and arrays.
5) Logical and boolean operators
+ logicalAnd(a,b), logicalOr(a,b), logicalXor(a,b), logicalNot(a),
+ booleanAnd(a,b), booleanOr(a,b), booleanXor(a,b), booleanNot(a,b)
Currently there are equivalent methods with "inversed" names.
I think these names are more intuitive. About the problem of
methods, see above. For some strange reason there is currently
no logical xor operation.
6) Non-elementwise operations
6.1) Attribute functions of binary operator functions (see 1 and 5)
reduce, accumulate,
+ outer
+ Note: reduce and accumulate should return the neutral
+ element of their base operation, reshaped to agree with the
+ remaining axes, if used along an axis of length zero.
+ Example:
+ add.reduce(array(42, (2,0,1)), 1)
+ should be equal to
+ array(0, (2,1))
6.2) Dot/matrix product
+ dot(a, b, axis_a = -1, axis_b = 0)
equivalent to:
if axis_a < 0: axis_a = len(a.shape)+axis_a
if axis_b >= 0: axis_b = axis_b + len(a.shape)
add.reduce(diagonal(multiply.outer(a,b), axis_a, axis_b))
(but of course done more efficiently)
Currently: method matrixMultiply, restricted to rank-2
arrays.
Redundant method: nonzero
I can't see what this operation is good for. It seems
to be equivalent to ravel(notEqual(a,0)), which is not
an extremely frequent combination.
Abbreviations:
sum = add.reduce
cumsum = add.accumulate
product = multiply.reduce
cumproduct = multiply.accumulate
allTrue = logicalAnd.reduce
someTrue = logicalOr.reduce
ptp(a) = maximum.reduce(a)-minimum.reduce(a)
Looping, mapping, filtering, etc.
=================================
1) Looping over array elements
for element in a: ...
Looping over anything else than the first axis can be achieved
by indexing.
2) Mapping a function on one or more arrays
+ arrayMap(function, (a1, a2, a3, ..., an), ranks = None)
function must be a callable object. If ranks is specified, it
must be a sequence of integers of length n; if ranks is None,
all ranks are assumed to be one.
The frames of all arrays with respect to the rank specification
must match.
3) Filtering
+ arrayFilter(function, a, axis=0)
collects all elements along the given axis for which the
supplied function returns non-0.
See also repeat() in section "structural operations".
4) Conditional selection
+ choose(a, (b0, b2, b3, ..., bn))
a must be an array of integers between 0 and n. The result
has the same shape as a with elements selected from b0...bn
according to the value of the corresponding element of a.
Currently: method a.choose(b0, ..., bn)
Passing the arrays b0...bn as a tuple has the advantage that
this tuple can be constructed arbitrarily in the code.
5) Sorting
5.1) Sort an array
+ sortArray(a, function=lambda x:x, axis=0)
Sorts the elements along the specified axis according to the return
value of the supplied function.
Note: the default function makes sense only for rank-1 arrays.
5.2) Sort index
+ sortIndex(a, function=lambda x:x, axis=0)
returns an index array i such that take(a,i) == sort(a, function, axis)
Redundant function:
where(condition, x, y) equals choose(condition, (x,y))
Other operations
================
1) Type casts
1.1) General cast
+ a.asType(typecode)
1.2) Special cast functions that also work for scalars
+ integerArray(a)
+ floatArray(a)
+ complexArray(a)
2) Implementation dependent functions:
2.1) Obtain the amount of bytes occupied by each element
a.itemsize()
2.2) Return a byte-swapped copy
+ a.byteswapped()
2.3) Return the element type code
a.typecode()
2.4) Return the element type
+ a.elementType()
returns the type object of the Python scalar object that results
if an element is extracted. Returns None for a general object
(typecode 'O').
2.5) Checks if an array is contiguous
+ a.isContiguous()
2.1) Return data as a Python string
a.toString()
Redundant function:
a.copy()
Copying should be done with copy.copy(), just as for any other object.
Comments
========
1) APL uses a special case of transposition for selecting a
diagonal, but this is often confusing.
2) In J this is done by using ravel() with reduced rank, but
as long as we don't have functions with bounded ranks,
we need a special function.
3) I tried to find a construction that does not require
yet another syntax and this is the best I could think
of. But I am not particularly attached to it, so feel
free to think of something better.
4) APL does this with fractional indices, which is probably
the weirdest feature in APL.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Mon Feb 26 16:28:48 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Mon, 26 Feb 96 11:28:48 EST
Subject: [PYTHON MATRIX-SIG] Two weeks' experience
In-Reply-To: <199602241841.NAA28353@cyclone.ERE.UMontreal.CA> (hinsenk@ere.umontreal.ca)
Message-ID: <9602261628.AA26473@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
As the only person here with several months of experience writing
pratical code, I'd like to comment on a few of Konrad's points.
1) You can in fact write greater(a, b) already, today, using the
current package. In fact, I personally do this all this time. Once
again, I wish that people would at least try things before complaining
that they can't be done.
2) reshape does clearly need work, no arguments here
3) fromFunction was only written to prove to Tom Schwaller that it
could be done. It is in fact very useful in it's current form. I
have made every efort to avoid having to loop over python functions in
the array code as this is ridiculously slow. If you want these
functions they can always be easily written in python.
I really feel that once this gets released beyond this list it's going
to get difficult to make the kinds of major changes that Konrad is
advocating, so I'm reluctant to go forward on a public alpha release
until these issues are at least somewhat resolved.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From maurice@blueskyprod.com Mon Feb 26 15:11:13 1996
From: maurice@blueskyprod.com (Maurice Van Swaaij)
Date: Mon, 26 Feb 1996 10:11:13 -0500
Subject: [PYTHON MATRIX-SIG] openGL
Message-ID: <3131CD91.1CFB@blueskyprod.com>
I've downloaded the alpha version of openGL from
http://maigret.cog.brown.edu/python/opengl/.
It says that it needs the python matrix module. Is there an alpha
available to test this openGL module with ?
--
Maurice Van Swaaij
System Architect
Blue Sky Productions
maurice@blueskyprod.com
1 914 941 5260
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From jjh@Goldilocks.LCS.MIT.EDU Mon Feb 26 16:38:14 1996
From: jjh@Goldilocks.LCS.MIT.EDU (James Hugunin)
Date: Mon, 26 Feb 96 11:38:14 EST
Subject: [PYTHON MATRIX-SIG] RubberIndex
In-Reply-To: <199602240000.TAA20621@python.org> (message from Chris Chase S1A on Fri, 23 Feb 1996 19:06:19 -0500)
Message-ID: <9602261638.AA26505@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Just a quick point:
If 'a' has rank 3, then
a[0] = 1
This implementation still allows an array to be used anywhere where a
sequence object (e.g. list) is used.
This is not true. In order to be able to use an array anywhere a list
is used, a[0] for a rank 3 array must return a rank 2 array, because
this is how it works for multidimensional lists currently.
-Jim
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From chris.chase@jhuapl.edu Mon Feb 26 17:41:00 1996
From: chris.chase@jhuapl.edu (Chris Chase S1A)
Date: Mon, 26 Feb 1996 12:41:00 -0500
Subject: [PYTHON MATRIX-SIG] RubberIndex
In-Reply-To: <9602261638.AA26505@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
References: <199602240000.TAA20621@python.org>
<9602261638.AA26505@baribal.LCS.MIT.EDU.LCS.MIT.EDU>
Message-ID: <199602261735.MAA06305@python.org>
>>>>> "James" == James Hugunin <jjh@Goldilocks.LCS.MIT.EDU> writes:
James> Just a quick point:
James> If 'a' has rank 3, then
James> a[0] = 1
James> This implementation still allows an array to be used anywhere where a
James> sequence object (e.g. list) is used.
James> This is not true.
My statement was that it could be used anywhere a sequence object is
used, a list being an example of a sequence object.
James> In order to be able to use an array anywhere a list is used,
You will never be able to use the current array implementation
everywhere that a list is used because array has different definitions
for operators (e.g. *, +) and array can not support 'del' without
having to copy all of its elements. If you want to use an array as a
list then use a.toList() (the toList() method would not be needed if
arrays were a subclass of lists).
James> a[0] for a rank 3 array must return a rank 2 array, because
James> this is how it works for multidimensional lists currently.
"must"? Why? As far as I can tell the hierarchical indexing for
arrays was a convenience for those used to working lists, because it
duplicates behavior already provided by a[0,...] indexing.
I was suggesting a different behavior for arrays as sequence objects.
My suggestion regarded supporting the use of flattened indexing for
both selection and assignment. This specific solution for flattened
indexing is supported in all the array languages that I have had
exposure to (e.g. Matlab, IDL, Tela, Yorick). To me this is more
important than the hierarchical indexing scheme so that an array looks
like a list (but does not act like a list for other purposes).
As an alternative, a 'flat' (or 'flattened') array attribute could be
supported.
An aside: I know that Konrad Hinsen prefers functions to methods and
attributes so that the same code will work for both scalars and
arrays. But this is similar to the list analogy above. scalars will
never work in all array code because scalars are neither sequences nor
dictionary types. The first time one attempts to subscript a scalar
an error will be produced. Certainly we don't want subscription to be
a function rather than a method. Rather than jump onto the everything
is a function bandwagon we may need to rethink the relationship
between scalars and arrays. What is the entire scope of code where
scalars and arrays need to be interchangeable?
Chris
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From chris.chase@jhuapl.edu Mon Feb 26 19:30:43 1996
From: chris.chase@jhuapl.edu (Chris Chase S1A)
Date: Mon, 26 Feb 1996 14:30:43 -0500
Subject: [PYTHON MATRIX-SIG] Functions and names
In-Reply-To: <199602241438.JAA20228@cyclone.ERE.UMontreal.CA>
References: <199602232226.RAA24062@cyclone.ERE.UMontreal.CA>
<199602241438.JAA20228@cyclone.ERE.UMontreal.CA>
Message-ID: <199602261925.OAA07208@python.org>
>>>>> "Konrad" == Konrad HINSEN <hinsenk@ere.umontreal.ca> writes:
Chase> current array representation. Besides, I think it is more
Chase> natural to return a copy of the indexed elements rather
Chase> than a reference when using a general index vector.
Konrad> Indeed, but this creates the unpleasant situation that an expression
Konrad> like a[i,j] returns either a copy or a reference, depending on the
Konrad> values of i and j. Having references at all is confusing enough, but
Konrad> this would certainly cause some headaches.
This was not my suggestion. My suggestion kept the current behavior.
I suggested a different access method or attribute called 'subarray'
allowing the use of a general index vector that would return a copy.
Konrad> Personally I'd prefer to have standard indexing always return a copy
Konrad> (that is the safer option) and provide an alternative syntax for
Konrad> obtaining a reference. Then standard indexing could again allow
Konrad> general index vectors.
Personally, I currently only care that general index vectors be
supported for both subscription and assignment.
Chase> I specifically said that the function I was describing overwrites
Chase> elements in the target array rather than making room. True it is not
Konrad> Then allowing general index vectors in indexed assignments should
Konrad> be all you need, right?
One only needs slices. The insertion functionality I suggested only
requires one to specify the starting position. Normally, the user
would have to compute the slice ending position and check to see if
the input array will fit (if not the input array could be sliced to
fit). I was suggesting a insertion function that would handle options
of wrap-around versus truncation versus just error signaling. It is
very convenient for manipulating block regions of arrays (like placing
one image inside another image). Perhaps a better name than "insert"
would be "place"?
Chase> I do not like this. Perhaps an attribute 'collapse' would
Chase> work? E.g.
Chase> a.collapse[i1, i2, ..., i3, i4]
Konrad> If that can be done, fine. So what should the value of "collapse" be
Konrad> before subscripting? I can't see a way to make this work.
Chase> It would have to be an object containing a reference to the
Chase> array 'a'. It would have the special property that the
Chase> ellipses index, '...', collapses missing dimensions.
Chase> Otherwise it would act like a normal array. Since this is a
Chase> special kind of access to the array object, a
Konrad> So effectively you want to introduce a second array type with
Konrad> a slightly different behaviour.
Konrad> I don't think this is a good idea. There would be no way to
Konrad> stop users from just writing a.collapse and assign the result
Konrad> to variables, return it from library functions etc. Suddenly
Konrad> there are two array types in free circulation that behave
Konrad> almost, but not quite, identically.
I was not sure that it was a good solution for this reason. I
suggested an attribute because attributes can support the indexing
syntax whereas functions can not.
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Mon Feb 26 21:09:38 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Mon, 26 Feb 1996 16:09:38 -0500
Subject: [PYTHON MATRIX-SIG] Two weeks' experience
In-Reply-To: <9602261628.AA26473@baribal.LCS.MIT.EDU.LCS.MIT.EDU> (jjh@Goldilocks.LCS.MIT.EDU)
Message-ID: <199602262109.QAA23356@cyclone.ERE.UMontreal.CA>
3) fromFunction was only written to prove to Tom Schwaller that it
could be done. It is in fact very useful in it's current form. I
have made every efort to avoid having to loop over python functions in
the array code as this is ridiculously slow. If you want these
functions they can always be easily written in python.
Of course, but something straightforward should be in the
standard function set. The reason why I think the current
fromFunction() is not so important is that it can easily
be replaced:
fromFunction([1,2], f)
is the same as
apply(f, indices(1,2))
I really feel that once this gets released beyond this list it's going
to get difficult to make the kinds of major changes that Konrad is
advocating, so I'm reluctant to go forward on a public alpha release
until these issues are at least somewhat resolved.
Of course it would be necessary to warn people that things are
going to change. It would even make sense to include a list of
proposals and ask for comments.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Mon Feb 26 21:27:05 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Mon, 26 Feb 1996 16:27:05 -0500
Subject: [PYTHON MATRIX-SIG] RubberIndex
In-Reply-To: <199602261735.MAA06305@python.org> (message from Chris Chase S1A on Mon, 26 Feb 1996 12:41:00 -0500)
Message-ID: <199602262127.QAA24193@cyclone.ERE.UMontreal.CA>
You will never be able to use the current array implementation
everywhere that a list is used because array has different definitions
for operators (e.g. *, +) and array can not support 'del' without
True, but that is no reason to introduce additional incompatibilities.
"must"? Why? As far as I can tell the hierarchical indexing for
arrays was a convenience for those used to working lists, because it
duplicates behavior already provided by a[0,...] indexing.
Historically it was the other way round, but all that doesn't
matter. The question is what behaviour is the most useful.
both selection and assignment. This specific solution for flattened
indexing is supported in all the array languages that I have had
exposure to (e.g. Matlab, IDL, Tela, Yorick). To me this is more
important than the hierarchical indexing scheme so that an array looks
like a list (but does not act like a list for other purposes).
The array languages I am familiar with (APL and J) do not permit this.
I suppose that the languages you mention support flattened access
because Fortran programmers are used to it, not because it is
important in real applications. I still think that a structural
modification like flattening should be explicit, not implied.
And let's not forget that our goal is not to write a Matlab/IDL/whatever
clone, but to provide arrays for Python. We should stick as close
as possible to established Python habits, which is why I insist
so much on the analogy with nested lists.
As an alternative, a 'flat' (or 'flattened') array attribute could be
supported.
Why? What's wrong with ravel(a)? You can index its result if you want.
An aside: I know that Konrad Hinsen prefers functions to methods and
attributes so that the same code will work for both scalars and
arrays. But this is similar to the list analogy above. scalars will
never work in all array code because scalars are neither sequences nor
dictionary types. The first time one attempts to subscript a scalar
an error will be produced. Certainly we don't want subscription to be
True, but that makes sense. Scalars are considered rank-0 arrays,
and indexing rank-0 arrays makes no sense. So the behaviour is
consistent. And again, the fact that something can't be done
perfectly is no argument for not doing it as good as possible.
My point about functions vs. methods is that many algorithms
can reasonably applied to arrays of all ranks, including
rank 0. Of course such algorithms cannot include indexing.
But all the operations that are rank-independent should be
implemented in such a way that they accept both arrays and
scalars.
a function rather than a method. Rather than jump onto the everything
is a function bandwagon we may need to rethink the relationship
between scalars and arrays. What is the entire scope of code where
scalars and arrays need to be interchangeable?
Everything except indexing. Some care must be taken with functions
that require an axis specification (like add.reduce). Obviously
a rank-0 array does not have axes, but add.reduce(x) still makes
sense if x is a scalar. Ideally, a call to add.reduce with an
explicit axis specification should raise an exception if given
a scalar, but do the right thing with the default axis.
BTW, there are more arguments not to implement array operations as
methods, as I have outlined in my proposal.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Mon Feb 26 21:34:55 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Mon, 26 Feb 1996 16:34:55 -0500
Subject: [PYTHON MATRIX-SIG] Functions and names
In-Reply-To: <199602261922.OAA18677@cyclone.ERE.UMontreal.CA> (message from Chris Chase S1A on Mon, 26 Feb 1996 14:30:43 -0500)
Message-ID: <199602262134.QAA24610@cyclone.ERE.UMontreal.CA>
One only needs slices. The insertion functionality I suggested only
requires one to specify the starting position. Normally, the user
would have to compute the slice ending position and check to see if
the input array will fit (if not the input array could be sliced to
fit). I was suggesting a insertion function that would handle options
of wrap-around versus truncation versus just error signaling. It is
very convenient for manipulating block regions of arrays (like placing
one image inside another image). Perhaps a better name than "insert"
would be "place"?
I see. That looks indeed useful; I remember having once written an
APL function to do more or less the same.
It is however not so easy to find a nice syntax for this. Right now
all operations that change array elements are assignments, and I'd
like to keep it that way. One possibility would be to allow somthing
like this:
a = array([1,2,3,4,5,6])
b = array([0,0])
a[2] = b
with the result
a == array([1,2,0,0,5,6])
I just don't like the error checking that gets lost in the process.
Maybe the alternative
a[2::] = b
(assignment of a vector that is too short) would be a lesser evil.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From da@maigret.cog.brown.edu Wed Feb 28 15:54:59 1996
From: da@maigret.cog.brown.edu (David Ascher)
Date: Wed, 28 Feb 1996 10:54:59 -0500 (EST)
Subject: [PYTHON MATRIX-SIG] fft's
Message-ID: <199602281554.KAA12674@maigret>
I've been working on python interface to the FFT routines in the
complib.sgimath library (since those are the ones I have optimized
binaries for).
This has been educational, but it has raised several questions which I
can't answer. First I'll talk about a specific one related to 2-d and
3-d FFT's, then ask a few questions about interfacing to fortran in
general.
The complib.sgimath (based on stuff from netlib) has a set of fft
calls, 1, 2 and 3-d. Things are working fine in 1-D, but I have to
admit that my grokking of 2-D and 3-D FFT's is rather limited.
When doing a 1-d fft, the output array is bigger than the input array by
enough to fit an aligned complex. Fine.
The 2-d fft calls in the fft library I'm using have the following syntax:
int dfft2du( int job, int n1, int n2, double *sequence,
int lda, double *workspace)
job specifies direction (forward vs. backward)
sequence is where the data is stored (both before and after the fft)
workspace is the sines & cosines and the factorization of n1 and n2.
All that is fine.
n1 and n2 are defined as follows:
N1 - INTEGER.
On entry, N1 specifies the number of elements in the
first dimension of the sequence.
Unchanged on exit.
N2 - INTEGER.
On entry, N2 specifies the number of elements in the
second dimension of the sequence.
Unchanged on exit.
and LDA is defined as:
LDA - INTEGER.
On entry, LDA specifies the first dimension of the
array SEQUENCE as declared in the calling (sub)program.
LDA must be at least max( 1, 2*((N1+2)/2) ).
Unchanged on exit.
This is where I get lost. I assumed that if I had a n1 x n2 array to
start with, I'd have an array which would be slightly bigger in, say, n1
and n2 (enough to store a complex?). But this LDA parameter is
completely confusing me. I have similar problems in 3d of course, where
LDA is replaced by ld1 and ld2. I suspect if I grok LDA i'll grok LD1
and LD2. =)
I realize that fortran and C/Python index into arrays differently (row
vs. column first). What is a good way of dealing with this?
This brings to mind a different issue regarding interfaces to existing
libraries. The library I'm using returns rank 1 arrays, which are
really rank 1, 2, or 3 but designed for antiquated languages. Should I
do the conversion to the appropriate rank (e.g. the assignment of shape)
in a .py wrapper, so that the library module itself is as compatible w/
the fortran and C interfaces, or should I make the library
python-friendly, and have it return appropriately sized arrays?
Thanks for reading this far.
--david
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From dubois1@llnl.gov Wed Feb 28 17:19:27 1996
From: dubois1@llnl.gov (Paul. Dubois)
Date: Wed, 28 Feb 1996 09:19:27 -0800
Subject: [PYTHON MATRIX-SIG] fft's
References: <199602281554.KAA12674@maigret>
Message-ID: <31348E9F.8EC@llnl.gov>
Most routines of this type written in C or Fortran allow for the
possibility that an array has been declared of a larger size than it
actually is. For example
#define N 32
#define M 20
int n, m, c[N]{M];
n = something
m = something
...now you want to do something with c and as far as you are
concerned it is n by m.
In C, the arrays are stored so that in considering
c[i][j] and c[i][j+1], they are 1 storage unit apart, but
c[i][j] and c[i+1][j] are M apart. For Fortran, it works the
other way, and it is c(i,j) and c(i, j+1) that are N apart.
When you pass such a beast to a general routine, therefore, it is
necessary to tell it N in Fortran or M in C. So in your fft what they
want for lda is M.
However, when working with an object-oriented language, this kind of
partially full matrix just doesn't arise very often, and in most of
your work the python array x with shape (n,m) will be passed to the
library fft routine with lda = m.
Hope this helps.
--
Paul F. Dubois, L-472 (510)-422-5426
Lawrence Livermore National Laboratory FAX (510)-423-9969
Livermore, CA 94550 dubois1@llnl.gov
Consulting: PaulDubois@aol.com
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Wed Feb 28 22:02:55 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Wed, 28 Feb 1996 17:02:55 -0500
Subject: [PYTHON MATRIX-SIG] fft's
In-Reply-To: <199602281554.KAA12674@maigret> (da@maigret.cog.brown.edu)
Message-ID: <199602282202.RAA24236@cyclone.ERE.UMontreal.CA>
> n1 and n2 are defined as follows:
>
> N1 - INTEGER.
> On entry, N1 specifies the number of elements in the
> first dimension of the sequence.
> Unchanged on exit.
> N2 - INTEGER.
> On entry, N2 specifies the number of elements in the
> second dimension of the sequence.
> Unchanged on exit.
So these describe the number of data points...
> and LDA is defined as:
>
> LDA - INTEGER.
> On entry, LDA specifies the first dimension of the
> array SEQUENCE as declared in the calling (sub)program.
> LDA must be at least max( 1, 2*((N1+2)/2) ).
> Unchanged on exit.
and this the physical size of the grid. The reason for having both
is that you might want to dimension some arrays larger than necessary
to avoid recompilation for every different value of N1/N2. Remember,
we are talking about a library designed for a language without
dynamic allocation! In the case of Python, you would most certainly
allocate an array of the correct size and LDA would be equal
to N1. You will find these "LDA" parameters everywhere in Fortran
libraries.
> I realize that fortran and C/Python index into arrays differently (row
> vs. column first). What is a good way of dealing with this?
Your interface should make sense to Python users. They won't
see the Fortran source code anyway and probably don't even care
whether it's Fortran.
> really rank 1, 2, or 3 but designed for antiquated languages. Should I
> do the conversion to the appropriate rank (e.g. the assignment of shape)
> in a .py wrapper, so that the library module itself is as compatible w/
> the fortran and C interfaces, or should I make the library
> python-friendly, and have it return appropriately sized arrays?
It matters only that the function that end users call uses correctly
shaped arrays. Personally, I'd use the correct shape also for the
low-level interface, because there is no possible use for 1d access.
But if you prefer otherwise, I don't care.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From guido@CNRI.Reston.VA.US Wed Feb 28 22:36:03 1996
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Wed, 28 Feb 1996 17:36:03 -0500
Subject: [PYTHON MATRIX-SIG] fft's
In-Reply-To: Your message of "Wed, 28 Feb 1996 17:02:55 EST."
<199602282202.RAA24236@cyclone.ERE.UMontreal.CA>
References: <199602282202.RAA24236@cyclone.ERE.UMontreal.CA>
Message-ID: <199602282236.RAA23873@monty>
> > LDA - INTEGER.
> > On entry, LDA specifies the first dimension of the
> > array SEQUENCE as declared in the calling (sub)program.
> > LDA must be at least max( 1, 2*((N1+2)/2) ).
> > Unchanged on exit.
>
> and this the physical size of the grid. The reason for having both
> is that you might want to dimension some arrays larger than necessary
> to avoid recompilation for every different value of N1/N2.
This is what I thought, except that the formula for LDA's minimum size
seems to imply that it must be at least *1 larger* than N1? Perhaps
the library module uses this as temp space?
--Guido van Rossum <guido@CNRI.Reston.VA.US>
URL: <http://www.python.org/~guido/>
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================
From hinsenk@ere.umontreal.ca Wed Feb 28 22:41:18 1996
From: hinsenk@ere.umontreal.ca (Konrad HINSEN)
Date: Wed, 28 Feb 1996 17:41:18 -0500
Subject: [PYTHON MATRIX-SIG] fft's
In-Reply-To: <199602282236.RAA23873@monty> (message from Guido van Rossum on Wed, 28 Feb 1996 17:36:03 -0500)
Message-ID: <199602282241.RAA26182@cyclone.ERE.UMontreal.CA>
> > > LDA - INTEGER.
> > > On entry, LDA specifies the first dimension of the
> > > array SEQUENCE as declared in the calling (sub)program.
> > > LDA must be at least max( 1, 2*((N1+2)/2) ).
> > > Unchanged on exit.
> >
> > and this the physical size of the grid. The reason for having both
> > is that you might want to dimension some arrays larger than necessary
> > to avoid recompilation for every different value of N1/N2.
>
> This is what I thought, except that the formula for LDA's minimum size
> seems to imply that it must be at least *1 larger* than N1? Perhaps
> the library module uses this as temp space?
The minimum value is nonsense. The smallest reasonable value for N1 is
1, which makes 2*((N1+2)/2) = 2, which is always larger than 1.
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsenk@ere.umontreal.ca
Departement de chimie | Tel.: +1-514-343-6111 ext. 3953
Universite de Montreal | Fax: +1-514-343-7586
C.P. 6128, succ. Centre-Ville | Deutsch/Esperanto/English/Nederlands/
Montreal (QC) H3C 3J7 | Francais (phase experimentale)
-------------------------------------------------------------------------------
=================
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================