From da@skivs.ski.org Sun Feb 1 01:18:16 1998
From: da@skivs.ski.org (David Ascher)
Date: Sat, 31 Jan 1998 17:18:16 -0800 (PST)
Subject: [MATRIX-SIG] compiling NumPy for Windows
In-Reply-To: <199801312054.VAA16626@lmspc1.ibs.fr>
Message-ID: <Pine.SUN.3.96.980131170939.978A-100000@skivs.ski.org>
On Sat, 31 Jan 1998, Konrad Hinsen wrote:
> It would be nice if it were possible to distribute a single source
> package for all operating systems that could be compiled and installed
> by anyone with the necessary tools (compiler etc.) *automatically*,
> i.e. with no manual intervention. For Unix this is already reality
> (via Misc/Makefile.pre.in and a Setup file). For Windows we seem to be
> far away from such a situation, if it is at all doable.
Here's the issue -- with the latest Microsoft Visual compiler suite, the
GUI is the predominant way to manipulate the build process. While it can
(somewhat) read old makefiles, and it can export makefiles, I think there
will be less and less support for that over the years -- for example, I
haven't bothered to learn MS's Makefile format, since it's just easier to
click here and there. Not impossible, but sort of like maintaining a .el
file -- the talent is rare. =)
> Except for the "shipping back" part, this looks like what I just
> described for some particular compiler. Is that really such a big
> problem?
Well, there is no equivalent to Makefile.pre.in on PC's. There's the
registry, which tells you about things like version number & where the
binaries & Lib directory are, but there is no way to find out
automatically where the source tree is.
I noticed that Tcl/Tk's "full installation" includes the Include
subdirectory and the .LIB files (equivalent to libpython15.a). Maybe
that's something worth considering. The person compiling extensions
shouldn't have to worry about having the source tree, and if the .h files
and the .lib file were installed by the binary installer package, we could
find out about it from the registry (making a Makefile.pre.in tool much
easier to write).
Such an extension compilation tool would be easier to write if one could
assume that Mark Hammond's Registry and COM tools were installed, as
they'd make talking to the registry and compiler much easier.
> Another question: is it necessary to consider more than one compiler
> for the Windows platform? With several different compilers, it would
> already be a problem to provide installation instructions for
> everyone.
I haven't heard anyone on this list mention anything but Microsoft's
compiler.
--david
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Mon Feb 2 14:58:28 1998
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Mon, 2 Feb 1998 15:58:28 +0100
Subject: [MATRIX-SIG] Using Fortran-LAPACK on a DEC Alpha
Message-ID: <199802021458.PAA26636@lmspc1.ibs.fr>
Has anyone succeeded in interfacing a Fortran version of LAPACK
to Python on a DEC Alpha? Whatever I do, I always get
the error message "unresolved symbols" when I import a module
that uses Fortran code, and it won't even tell me which symbols
are undefined.
Konrad.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From krodgers@tdyryan.com Mon Feb 2 17:08:00 1998
From: krodgers@tdyryan.com (Kevin Rodgers)
Date: Mon, 02 Feb 1998 09:08:00 -0800
Subject: [MATRIX-SIG] Data import into Numeric arrays
Message-ID: <3.0.1.32.19980202090800.005321f0@gate.tdyryan.com>
I have the need to process large amounts of time history data with NumPy.
The data are stored as ordinary ASCII files with each line being one
timepoint and its associated data. Currently, the only ways I know to get
this data into NumPy all involve reading the data into a Python list (say,
via readline or readlines), splitting each line into fields, converting the
string representations of numbers to actual numbers, then using the array()
function to build an array. This seems really inefficient. What I'd like
to do is just read the data directly into NumPy from disk as a Numeric
array! Similarly with writing arrays; it would be really nice to easily
write this kind of data out to disk. Any ideas on this? Currently, about
80% of my scripts processing time is in the reading and writing of the
datafiles. Thanks in advance . . .
(BTW, if you're only suggestion is for me to store the data in a different
format, thanks but no thanks; this data has to be used by lots of different
people, and I am pretty much stuck with this format, as much as I'd like to
move it to, say netCDF or HDF or even straight binary.)
----------------------------------------------------------------------
Kevin Rodgers Teledyne Ryan Aeronautical krodgers@tdyryan.com
"This one goes up to eleven." -- Nigel Tufnel
----------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From Paul F. Dubois" <dubois1@llnl.gov Mon Feb 2 17:29:00 1998
From: Paul F. Dubois" <dubois1@llnl.gov (Paul F. Dubois)
Date: Mon, 2 Feb 1998 09:29:00 -0800
Subject: [MATRIX-SIG] Data import into Numeric arrays
Message-ID: <01bd3000$0cd96930$6c357380@xfiles.llnl.gov>
The input function can be used for breaking up lines if you put commas in
the file.
Better is to write your own extension in C or C++ that knows how to read
your files. You can use the C API to create the Python numerical arrays and
then fill them up. Or, you can use the C API to Python "list" and then at
the end convert to numerical arrays. Doing the read in C avoids having the
loop in Python, for one thing.
I do have some help for making the file in the first place. Check out
PyHistory at http://xfiles.llnl.gov/pyhistory. Two different text file
formats are available as output choices, along with Pact/PDB (which is like
netCDF or HDF, only better).
-----Original Message-----
From: Kevin Rodgers <krodgers@tdyryan.com>
To: matrix-sig@python.org <matrix-sig@python.org>
Date: Monday, February 02, 1998 9:13 AM
Subject: [MATRIX-SIG] Data import into Numeric arrays
>I have the need to process large amounts of time history data with NumPy.
>The data are stored as ordinary ASCII files with each line being one
>timepoint and its associated data. Currently, the only ways I know to get
>this data into NumPy all involve reading the data into a Python list (say,
>via readline or readlines), splitting each line into fields, converting the
>string representations of numbers to actual numbers, then using the array()
>function to build an array. This seems really inefficient. What I'd like
>to do is just read the data directly into NumPy from disk as a Numeric
>array! Similarly with writing arrays; it would be really nice to easily
>write this kind of data out to disk. Any ideas on this? Currently, about
>80% of my scripts processing time is in the reading and writing of the
>datafiles. Thanks in advance . . .
>
>(BTW, if you're only suggestion is for me to store the data in a different
>format, thanks but no thanks; this data has to be used by lots of different
>people, and I am pretty much stuck with this format, as much as I'd like to
>move it to, say netCDF or HDF or even straight binary.)
>----------------------------------------------------------------------
>Kevin Rodgers Teledyne Ryan Aeronautical krodgers@tdyryan.com
>"This one goes up to eleven." -- Nigel Tufnel
>----------------------------------------------------------------------
>
>_______________
>MATRIX-SIG - SIG on Matrix Math for Python
>
>send messages to: matrix-sig@python.org
>administrivia to: matrix-sig-request@python.org
>_______________
>
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Mon Feb 2 19:53:20 1998
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Mon, 2 Feb 1998 20:53:20 +0100
Subject: [MATRIX-SIG] Data import into Numeric arrays
In-Reply-To: <3.0.1.32.19980202090800.005321f0@gate.tdyryan.com> (message from
Kevin Rodgers on Mon, 02 Feb 1998 09:08:00 -0800)
Message-ID: <199802021953.UAA27894@lmspc1.ibs.fr>
> The data are stored as ordinary ASCII files with each line being one
> timepoint and its associated data. Currently, the only ways I know to get
> this data into NumPy all involve reading the data into a Python list (say,
> via readline or readlines), splitting each line into fields, converting the
> string representations of numbers to actual numbers, then using the array()
> function to build an array. This seems really inefficient. What I'd like
Right; someone should write a nice C module for ASCII I/O of arrays.
In the meantime, I can offer a ready and working "slow" version, so
you can at least check how bad it is for you:
---------------------------------------------------------------------------
# Array I/O to text files
#
# Written by Konrad Hinsen <hinsen@ibs.ibs.fr>
# last revision: 1998-1-29
#
from TextFile import TextFile
import string, Numeric
def readArray(filename):
data = []
for line in TextFile(filename):
data.append(map(eval, string.split(line)))
a = Numeric.array(data)
if a.shape[0] == 1 or a.shape[1] == 1:
a = Numeric.ravel(a)
return a
def writeArray(a, filename):
file = TextFile(filename, 'w')
if len(a.shape) == 1:
a = a[:, Numeric.NewAxis]
for line in a:
for element in line:
file.write(`element` + ' ')
file.write('\n')
file.close()
---------------------------------------------------------------------------
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Mon Feb 2 19:57:04 1998
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Mon, 2 Feb 1998 20:57:04 +0100
Subject: [MATRIX-SIG] Reading pickle files from Alpha
Message-ID: <199802021957.UAA27911@lmspc1.ibs.fr>
Has anyone done something that allows pickle files containing arrays
from a DEC Alpha to be read on lowly 32-bit machines? It doesn't look
like much work, but currently I'd be happy for any minute I needn't
spend on such minor details...
Konrad.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From busby@icf.llnl.gov Mon Feb 2 18:40:19 1998
From: busby@icf.llnl.gov (L. Busby)
Date: Mon, 2 Feb 98 10:40:19 PST
Subject: [MATRIX-SIG] ArrayPrinter problem
Message-ID: <9802021840.AA24444@icf.llnl.gov.llnl.gov>
[ Jeffery Collins says ]
>There seems to be a problem in the ArrayPrinter module. Consider
>the following:
[ deleted problem demonstration ]
>I was able to fix the problem by replacing (line 101)
>
> if max_val >= 1.e12:
> exp_format = 1
>
>with
>
> if max_val >= 1.e9:
> exp_format = 1
We've got this fixed in our LLNL snapshot, although we used 1.e8
instead of 1.e9. Thanks for the report.
Lee Busby
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From jhauser@ifm.uni-kiel.de Tue Feb 3 10:09:53 1998
From: jhauser@ifm.uni-kiel.de (Janko Hauser)
Date: Tue, 3 Feb 1998 11:09:53 +0100 (CET)
Subject: [MATRIX-SIG] Data import into Numeric arrays
In-Reply-To: <3.0.1.32.19980202090800.005321f0@gate.tdyryan.com>
References: <3.0.1.32.19980202090800.005321f0@gate.tdyryan.com>
Message-ID: <m0xzfIf-000t4KC@lisboa.ifm.uni-kiel.de>
Hi, attached is a much faster version for reading, than the one of
Konrad. It also seems to me, that the most inefficient part of data
reading is the conversion from strings to numbers.
I think the time for str=file.read() is reasonable fast. The
conversion took most of the time. This is where sscanf is really good
at. It is also nice to to use sscanf as an filter like in
def scan(self, lineformat, typecode = Float):
saved = []
line = 1
while line:
try:
data = sscanf(line, lineformat)
if data:
saved.append(data)
line = self.file.readline()
except:
line = self.file.readline()
pass
return asarray(saved, typecode)
with a lineformat of:
a = self.file.scan(" Day = %f at iic = %f avgKE = %f avgPE = %f")
This skips all lines, which are not of this format, good for report
creation. You see, I have build a class for this, but this is rather
clumsy at the moment, but I can mail it to you if you want. The bad
part at the moment is, that I don't have something equivilant for
writing.
If something more general should be implemented, then the question
arises, what we want.
o One 1d or 2d array per file
o More of these but all with the same length in the last dimension.
o Can text be inserted? Also between the numbers? TeX-tables ...
o Should this be a new file object, or some new methods for the
arraytype?
Besides Yoricks read is only ca. 2 times faster than read_table. And
Matlab very similar to read_table. Not so bad at all.
__Janko
def read_table(fname,header=0):
"""read_table(filename,header=number_of_header_lines)
Reads from filename after number_of_header_lines
"""
from sscanf import sscanf
import string
try:
fp=open(fname,'r')
except:
print "Can't open file or wrong filename in function read_table"
return
scanline='%f'
lines = fp.readlines()
fp.close()
n=len(string.split(lines[header]))
new_table=ones((len(lines)-header,n),typecode = Float)
while n-1:
n=n-1
scanline=scanline+' %f'
for item in xrange(len(lines)-header):
a=array(sscanf(lines[item],scanline))
new_table[item,:] = a
return(new_table)
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Tue Feb 3 10:50:06 1998
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Tue, 3 Feb 1998 11:50:06 +0100
Subject: [MATRIX-SIG] Pickling arrays on a DEC Alpha
Message-ID: <199802031050.LAA31073@lmspc1.ibs.fr>
Here's a quick hack for those who want to exchange pickle files between
DEC Alphas and other machines (with 32-bit longs). If you replace the
functions LoadArray() and DumpArray() in Numeric.py, arrays of longs
will always be saved and retrieved in 32-bit format, even on Alphas.
The modification doesn't make any difference on non-Alpha machines.
---------------------------------------------------------------------------
def DumpArray(m, fp):
s = m.shape
if LittleEndian: endian = "L"
else: endian = "B"
typecode = m.typecode()
# Save 64-int arrays (DEC Alpha) as 32 bit for compatibility
if typecode == 'l' and m.itemsize() == 8:
m = m.astype('i')
fp.write("A%s%s%d " % (typecode, endian, m.itemsize()))
for d in s:
fp.write("%d "% d)
fp.write('\n')
fp.write(m.tostring())
SizeOfLong = zeros((1,), 'l').itemsize()
def LoadArray(fp):
ln = string.split(fp.readline())
if ln[0][0] == 'A': ln[0] = ln[0][1:] # Nasty hack showing my ignorance of pickle
typecode = ln[0][0]
endian = ln[0][1]
shape = map(lambda x: string.atoi(x), ln[1:])
itemsize = string.atoi(ln[0][2:])
sz = reduce(multiply, shape)*itemsize
data = fp.read(sz)
# Read 32-bit arrays as int (not long) on DEC Alphas
if typecode == 'l' and itemsize == 4 and SizeOfLong == 8:
m = fromstring(data, 'i').astype('l')
else:
m = fromstring(data, typecode)
m = reshape(m, shape)
if (LittleEndian and endian == 'B') or (not LittleEndian and endian == 'L'):
return m.byteswapped()
else:
return m
---------------------------------------------------------------------------
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Tue Feb 3 16:26:06 1998
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Tue, 3 Feb 1998 17:26:06 +0100
Subject: [MATRIX-SIG] compiling NumPy for Windows
In-Reply-To: <Pine.SUN.3.96.980131170939.978A-100000@skivs.ski.org> (message
from David Ascher on Sat, 31 Jan 1998 17:18:16 -0800 (PST))
Message-ID: <199802031626.RAA32584@lmspc1.ibs.fr>
> Here's the issue -- with the latest Microsoft Visual compiler suite, the
> GUI is the predominant way to manipulate the build process. While it can
> (somewhat) read old makefiles, and it can export makefiles, I think there
> will be less and less support for that over the years -- for example, I
I'd call that a serious problem for scientific applications, where
distribution of source code is nothing unusual. Even if you adopt the
Microsoft point of view that non-Windows systems don't matter, you
still need executables for different chips (currently Intel and
Alpha), and you can't expect every program author to have all systems
available for producing binaries.
> Well, there is no equivalent to Makefile.pre.in on PC's. There's the
> registry, which tells you about things like version number & where the
> binaries & Lib directory are, but there is no way to find out
> automatically where the source tree is.
You don't need the source tree for compiling extensions. If you can
locate the libraries and header files, that's sufficient.
> Such an extension compilation tool would be easier to write if one could
> assume that Mark Hammond's Registry and COM tools were installed, as
> they'd make talking to the registry and compiler much easier.
That doesn't sound like an unrealistic assumption.
> I haven't heard anyone on this list mention anything but Microsoft's
> compiler.
But there's a port of gcc plus some Unix tools to Windows, right?
Would it make sense to recommend this to Windows users? It should
make porting Unix stuff much easier.
Konrad.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From Paul F. Dubois" <dubois1@llnl.gov Tue Feb 3 17:07:27 1998
From: Paul F. Dubois" <dubois1@llnl.gov (Paul F. Dubois)
Date: Tue, 3 Feb 1998 09:07:27 -0800
Subject: [MATRIX-SIG] compiling NumPy for Windows
Message-ID: <01bd30c6$349ffd00$998a7380@pduboispc>
No, we need to supply project files for Windows and your stuff for Unix. I'm
in the process of doing that. In the process I'm trying to rationalize the
arrangement of the source directory. I have the Wise installer software and
am learning how to use it for making the Windows installer.
I'm going to proceed on the premise that the installation on each platform
includes the library and include files because such a high percentage of
users are also going to be extenders. I will have a proposal for you
shortly about what I think the "installed" and "source" sides should look
like. Suggestions are welcome, of course.
Too bad I don't have more experience on Windows...here are some questions I
could stand some help with.
a. I believe an "extender" would need the .lib and the .pyd both. What about
the .exp file? The .def? Anything else?
b. How do you tell Wise to adjust Python's path in the registry?
I will eventually digest all of this but any pointers would help me get this
out faster.
-----Original Message-----
From: Konrad Hinsen <hinsen@ibs.ibs.fr>
To: da@skivs.ski.org <da@skivs.ski.org>
Cc: matrix-sig@python.org <matrix-sig@python.org>
Date: Tuesday, February 03, 1998 8:38 AM
Subject: Re: [MATRIX-SIG] compiling NumPy for Windows
>> Here's the issue -- with the latest Microsoft Visual compiler suite, the
>> GUI is the predominant way to manipulate the build process. While it can
>> (somewhat) read old makefiles, and it can export makefiles, I think there
>> will be less and less support for that over the years -- for example, I
>
>I'd call that a serious problem for scientific applications, where
>distribution of source code is nothing unusual. Even if you adopt the
>Microsoft point of view that non-Windows systems don't matter, you
>still need executables for different chips (currently Intel and
>Alpha), and you can't expect every program author to have all systems
>available for producing binaries.
>
>> Well, there is no equivalent to Makefile.pre.in on PC's. There's the
>> registry, which tells you about things like version number & where the
>> binaries & Lib directory are, but there is no way to find out
>> automatically where the source tree is.
>
>You don't need the source tree for compiling extensions. If you can
>locate the libraries and header files, that's sufficient.
>
>> Such an extension compilation tool would be easier to write if one could
>> assume that Mark Hammond's Registry and COM tools were installed, as
>> they'd make talking to the registry and compiler much easier.
>
>That doesn't sound like an unrealistic assumption.
>
>> I haven't heard anyone on this list mention anything but Microsoft's
>> compiler.
>
>But there's a port of gcc plus some Unix tools to Windows, right?
>Would it make sense to recommend this to Windows users? It should
>make porting Unix stuff much easier.
>
>Konrad.
>--
>---------------------------------------------------------------------------
----
>Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
>Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
>Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
>41, av. des Martyrs | Deutsch/Esperanto/English/
>38027 Grenoble Cedex 1, France | Nederlands/Francais
>---------------------------------------------------------------------------
----
>
>_______________
>MATRIX-SIG - SIG on Matrix Math for Python
>
>send messages to: matrix-sig@python.org
>administrivia to: matrix-sig-request@python.org
>_______________
>
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From da@skivs.ski.org Tue Feb 3 17:37:22 1998
From: da@skivs.ski.org (David Ascher)
Date: Tue, 3 Feb 1998 09:37:22 -0800 (PST)
Subject: [MATRIX-SIG] compiling NumPy for Windows
In-Reply-To: <01bd30c6$349ffd00$998a7380@pduboispc>
Message-ID: <Pine.SUN.3.96.980203093231.29660A-100000@skivs.ski.org>
> No, we need to supply project files for Windows and your stuff for Unix.
Agreed.
> I'm going to proceed on the premise that the installation on each platform
> includes the library and include files because such a high percentage of
> users are also going to be extenders.
Again, agreed -- let's get this to Guido ASAP -- the sooner we convince
him, the sooner it'll happen.
> a. I believe an "extender" would need the .lib and the .pyd both. What
> about the .exp file? The .def? Anything else?
The .LIB is created on the way to the .PYD file. The .DEF is simply there
to tell the compiler which symbols to export in the .LIB. The .EXP is an
intermediate file. So .PYD for Python-only runtime use, .PYD + .LIB for
those cases which need access to the C API.
> b. How do you tell Wise to adjust Python's path in the registry?
item: Edit Registry
Total Keys=1
Key=Software\Python\PythonCore\%PY_REG_VERSION%\PythonPath\%PY_APP_NAME%
New Value=%INSTDIR%;%INSTDIR%\Lib
Root=2
end
This sort of thing.
NB: 1) WISE has an upgrade which you should get -- apparently makes things
much more robust.
2) We can take this offline -- I think Mark, Harri, you and myself are
the only ones who've played with WISE (except for Guido, but he's
busy =)
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From da@skivs.ski.org Tue Feb 3 17:56:18 1998
From: da@skivs.ski.org (David Ascher)
Date: Tue, 3 Feb 1998 09:56:18 -0800 (PST)
Subject: [MATRIX-SIG] compiling NumPy for Windows
In-Reply-To: <199802031626.RAA32584@lmspc1.ibs.fr>
Message-ID: <Pine.SUN.3.96.980203093727.29660B-100000@skivs.ski.org>
> I'd call that a serious problem for scientific applications, where
> distribution of source code is nothing unusual. Even if you adopt the
> Microsoft point of view that non-Windows systems don't matter, you
> still need executables for different chips (currently Intel and
> Alpha), and you can't expect every program author to have all systems
> available for producing binaries.
No, but I think you're being pessimistic. There's really three platforms.
Unix (source distribution works fine), Windows, Mac. Even if we can't
come up with an automatic way of you (a non-Windows user) generating a
Windows makefile, I'm pretty sure I (or others) can take your Unix
makefile (provided it's not that complicated) and make a windows binary in
a day at most. When there are so many extensions announced that we can't
keep up, we'll deal with that bridge. I haven't coded on a mac in a while
(since the days of Think C =), so I don't know what the compilation
configuration issues are there.
Just last weekend I've SWIGged a couple of libraries, and I have been
thinking about the right way to package extension *source* distributions
so that they work well with both Unix and Windows. Probably not too
surprisingly, it looks a fair bit like Tcl's distribution model.
> You don't need the source tree for compiling extensions. If you can
> locate the libraries and header files, that's sufficient.
That's my point exactly. All I meant was that we need to get them in
Guido's installer so that they are there. And they need to be locatable
via the registry.
>> [we need COM and registry support]
> That doesn't sound like an unrealistic assumption.
Well, the logistics are somewhat tricky, since Mark is the author of
those, and Guido releases the base install.
> > I haven't heard anyone on this list mention anything but Microsoft's
> > compiler.
>
> But there's a port of gcc plus some Unix tools to Windows, right?
> Would it make sense to recommend this to Windows users? It should
> make porting Unix stuff much easier.
There's a couple of ports, and they are all much much too incomplete or
expensive or hard to setup to recommend to the generic Windows user. It
took me weeks to get my gnuwin32 setup "just right", and I have a fair bit
of experience with Unix. Also, it's not at all trivial to get
interoperability between gnuwin/gcc-compiled DLLs and "straight" Windows
DLL's. In other words I'd recommend heartily *against* setting up a unix
lookalike on Windows, except if, like me, you can't do without the shell,
grep, find, etc.
opinionatedly yours,
--david
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From jhauser@ifm.uni-kiel.de Wed Feb 4 13:13:14 1998
From: jhauser@ifm.uni-kiel.de (Janko Hauser)
Date: Wed, 4 Feb 1998 14:13:14 +0100 (CET)
Subject: [MATRIX-SIG] HDF reader
Message-ID: <m0y04de-000t4KC@lisboa.ifm.uni-kiel.de>
As we are at this subject, has someone a module to read HDF files,
more precisely bitmap information from a HDF file?
__Janko
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From npat@ariadne.di.uoa.gr Fri Feb 6 14:23:10 1998
From: npat@ariadne.di.uoa.gr (Nick Patavalis)
Date: Fri, 6 Feb 1998 16:23:10 +0200 (EET)
Subject: [MATRIX-SIG] The Python interface to PGPLOT graphics library
Message-ID: <199802061423.QAA09424@ariadne.di.uoa.gr>
A PYTHON / NUMERIC-PYTHON INTERFACE TO THE PGPLOT
GRAPHICS LIBRARY.
One thing I found missing from Python and Numeric Python was a good
library-of-functions (module) specially designed for scientific
visualization (as opposed to imaging or GUI implementation). I know
there is at least one interface to Gnuplot, and other interfaces for
plotting through programs like Mathematica. They are a good
solution for simple tasks, and maybe even more. One has to admit,
though, that they lack the power (performance) and versatility
required for several more complex applications. This is true for a
number of reasons:
1. They provide their services through a different task,
usually communicating via temporary files. This is not
the most efficient architecture, and its also not free of other
problems.
2. Some of them are implemented as python scripts. Python
is nice, powerful and easy to use, but it lacks the
performance of C.
3. The underlying programs that actually provide the graphics
capabilities are usually not general-purpose visualization
tools but programs primarily designed for interactive use
(e.g. Gnuplot)
In order to circumvent these limitations I decided to implement an
interface module that enables Numeric Python to call the functions of
the (really remarkable) PGPLOT graphics library. The interface module
is written in C and provides (or at least will do so soon) the full
set of PGPLOT facilities, wrapped in a python interface that resembles
as much as possible the original PGPLOT semantics and calling
conventions. I decided to do so in order to make life easier for those
who are already familiar with PGPLOT. I have also coded an
alternative interface for *some* PGPLOT functions that had a rather
"complicated" interface. These simplified interfaces are more close to
python's idiosyncrasies, and were designed to take advantage of some
of its special features (like optional arguments or keyword arguments,
etc).
After the original code is proven stable enough, I plan to implement
an even simpler interface. It will be designed for "interactive" use
(directly at the python prompt), but not only. I think of making it
like Gnuplot's interface, (with a lot of functionality added). Anyway
I believe its too soon to say anything more on this subject.
I have NOT released the source code yet, but I plan to do so soon
(maybe sometime in February). You can find some example screen-shots
produced by PPGPLOT in:
http://ariadne.di.uoa.gr/ppgplot ,
along with the python code required to produce them.
For those of you who are not familiar with PGPLOT here's a short
description:
PGPLOT is written by Tim Pearson
The PGPLOT Graphics Subroutine Library is a Fortran- or C-callable,
device-independent graphics package for making simple scientific
graphs. It is intended for making graphical images of publication
quality with minimum effort on the part of the user. For most
applications, the program can be device-independent, and the output
can be directed to the appropriate device at run time.
The PGPLOT library consists of two major parts: a device-independent
part and a set of device-dependent ``device handler'' subroutines for
output on various terminals, image displays, dot-matrix printers,
laser printers, and pen plotters. Common file formats supported
include PostScript and GIF.
The home-page of pgplot is:
http://astro.caltech.edu/~tjp/pgplot
The main thing missing from ppgplot is a decent build-system, usable
under many platforms and configurations. Any help on this would be
greatly appreciated.
Please send any comments/remarks or questions about the python interface
to pgplot (ppgplot) to:
ppgplot@ariadne.di.uoa.gr
The amount of feedback I receive, will help me find out how many
people need this module, and adjust the development effort.
Regards
Nick Patavalis
University of Athens
npat@ariadne.di.uoa.gr
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From holz@zib.de Fri Feb 6 17:05:03 1998
From: holz@zib.de (Martin Holz)
Date: 06 Feb 1998 18:05:03 +0100
Subject: [MATRIX-SIG] Re: The Python interface to PGPLOT graphics library
In-Reply-To: Nick Patavalis's message of Fri, 6 Feb 1998 16:23:10 +0200 (EET)
References: <199802061423.QAA09424@ariadne.di.uoa.gr>
Message-ID: <wi7ra5gvf40.fsf@mailhost.zib.de>
There is a interface file for SWIG, which creates a PGPLOT module
at <http://www.geog.ubc.ca/~phil/ubc_python.html>. It is quite
useful, but a simpler, more python-like interface would be
interesting.
/Martin
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Fri Feb 6 18:32:43 1998
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Fri, 6 Feb 1998 19:32:43 +0100
Subject: [MATRIX-SIG] New version of netCDF module
Message-ID: <199802061832.TAA17475@lmspc1.ibs.fr>
I have just uploaded a new version of my netCDF module to the Starship:
http://starship.skyport.net/crew/hinsen/netcdf.html
There are several small bug fixes, and the new version allows access
to variables without dimensions, works on DEC Alphas, and should be
compilable under Windows.
Another innovation is a version number; the current version has
the official number 1.0, and future versions will have different
numbers. That should eliminate the confusion that past updates
have caused.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From jhauser@ifm.uni-kiel.de Mon Feb 9 12:08:50 1998
From: jhauser@ifm.uni-kiel.de (Janko Hauser)
Date: Mon, 9 Feb 1998 13:08:50 +0100 (CET)
Subject: [MATRIX-SIG] Matrix to Fortran
Message-ID: <m0y1s14-000t4KC@lisboa.ifm.uni-kiel.de>
At the weekend I have tried to interface NumPy with some Fortran
routines. I use for this the Cfortran package to prototype the
interface between C and Fortran, in the hope that this will be
portable. I have success with a routine wich accepts 1D arrays.
The interface looks like this (I use the LLNL defines for ease):
PROTOCCALLSFSUB11(RGRD1, rgrd1, INT, FLOATV, FLOATV, INT, FLOATV,
FLOATV, INT, INT, FLOATV, INT, INT)
#define RGRID1(nx,x,p,mx,xx,q,intpol,lw,w,lwmin,ier) \
CCALLSFSUB11(RGRD1, rgrd1, INT, FLOATV, FLOATV, INT, FLOATV, \
FLOATV, INT, INT, FLOATV, INT, INT, nx,x,p,mx,xx,q,intpol,lw,w, \
lwmin,ier)
Py_Try(PyArg_ParseTuple(args, "iOOiOOiiOii", &nx, &ox, &op,
&mx, &oxx, &oq, &intpol, &lw, &ow, &lwmin, &ier));
GET_ARR(ax, ox, PyArray_FLOAT, 1);
.
.
.
x = (float *) A_DATA (ax);
.
.
.
RGRID1(nx,x,p,mx,xx,q,intpol,lw,w,lwmin,ier);
This is ok, but if I want to interface the 2D version of regrid I
can't pass the 2D fields correctly. I pass
GET_ARR(ax, ox, PyArray_FLOAT, 2);
.
.
.
x = (float *) A_DATA (ax);
.
.
.
But in the Fortran routine I can't access the field. I have looked
into the result of f2c, where also a simple pointer is passed, and
every index calculated from the two indices.
Then, is this the right way? I hope that this will be portable. Is
there another way? This are only four subroutines, so I haven't looked
into SWIG yet, but I think I should.
__Janko
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From da@skivs.ski.org Mon Feb 9 18:52:45 1998
From: da@skivs.ski.org (David Ascher)
Date: Mon, 9 Feb 1998 10:52:45 -0800 (PST)
Subject: [MATRIX-SIG] matrix storage using zlib
Message-ID: <Pine.SUN.3.96.980209105216.8417C-100000@skivs.ski.org>
Has anyone tried to use the zlib module for small storage of arrays?
--da
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From miller5@uiuc.edu Tue Feb 10 20:33:09 1998
From: miller5@uiuc.edu (Mike Miller)
Date: 10 Feb 1998 14:33:09 -0600
Subject: [MATRIX-SIG] Question about Numeric.convolve
Message-ID: <t967mnryii.fsf@uinpluxc.npl.uiuc.edu>
I'm (trying) to use convolve to convolute a calculation with
Gaussian resolution and am having some trouble sorting things
out. I have two arrays, x and f = f(x) and I'd like to convolute
f(x) with a Gaussian. So I calculate an array g like this:
g = Numeric.zeros( x.shape,'Float').real
for i in range(len(g)):
g[i] = math.exp(-(math.pow((x[i]-mu)/sigma,2)))
and convolute the two arrays like this:
fg = Numeric.convolve( f, g, 0 )
I've been fiddling with this for a while now, adjusting the
location of the Gaussian, zero extending f and g at either end,
but I can't seem to sort out where convolve puts the result in
fg. I'd like to be able to plot both f and fg versus x to see
the result.
So, the question is - is there a way to convolute two arrays with
x, rather than the array index, as the index. Or am I just
totally clueless? Or both?
Mike
--
Michael A. Miller miller5@uiuc.edu
Department of Physics, University of Illinois, Urbana-Champaign
PGP public key available on request
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Wed Feb 11 18:42:16 1998
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Wed, 11 Feb 1998 19:42:16 +0100
Subject: [MATRIX-SIG] Question about Numeric.convolve
In-Reply-To: <t967mnryii.fsf@uinpluxc.npl.uiuc.edu> (message from Mike Miller
on 10 Feb 1998 14:33:09 -0600)
Message-ID: <199802111842.TAA12183@lmspc1.ibs.fr>
> I'm (trying) to use convolve to convolute a calculation with
> Gaussian resolution and am having some trouble sorting things
> out. I have two arrays, x and f = f(x) and I'd like to convolute
> f(x) with a Gaussian. So I calculate an array g like this:
> g = Numeric.zeros( x.shape,'Float').real
> for i in range(len(g)):
> g[i] = math.exp(-(math.pow((x[i]-mu)/sigma,2)))
First of all, that's complicated and inefficient. NumPy can
do much better:
g = Numeric.exp(((x-mu)/sigma)**2)
> So, the question is - is there a way to convolute two arrays with
> x, rather than the array index, as the index. Or am I just
> totally clueless? Or both?
Numeric.convolve is not what you are looking for, as indeed it
convolutes according to the array index (which is equivalent for
equally-spaced data and much more efficient). A convolution for
unevenly spaced data amounts to a explicit numerical integration. If
that's what you want, well, Simpson's rule is easy enough to implement
in Python.
Konrad.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From pas@xis.xerox.com Wed Feb 11 20:11:41 1998
From: pas@xis.xerox.com (Perry Stoll)
Date: Wed, 11 Feb 1998 12:11:41 PST
Subject: [MATRIX-SIG] Question about Numeric.convolve
Message-ID: <006201bd3729$446e0650$4a4df60d@stoll-pc.xis.xerox.com>
>> f(x) with a Gaussian.
>> for i in range(len(g)):
>> g[i] = math.exp(-(math.pow((x[i]-mu)/sigma,2)))
> NumPy can do much better:
> g = Numeric.exp(((x-mu)/sigma)**2)
but only with the right formula ;)
g = Numeric.exp( -.5 * ((x-mu)/sigma)**2)
And for a normalized Gaussian, don't forget the scaling term
1.0/(sigma * Numeric.sqrt(2 * Numeric.pi))
-Perry
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From Dirk.Engelmann@iwr.uni-heidelberg.de Wed Feb 11 21:32:50 1998
From: Dirk.Engelmann@iwr.uni-heidelberg.de (Dirk Engelmann)
Date: Wed, 11 Feb 1998 22:32:50 +0100 (CET)
Subject: [MATRIX-SIG] 2-D convolution ?
Message-ID: <Pine.LNX.3.95.980211222827.24752B-100000@klimt.iwr.uni-heidelberg.de>
Hi!
Is there a method for a 2 dimnesional convolution ?
It could be done by fft
inverse_fft2d(fft2d(mask) * fft2d(two_d_array))
but only for square matrix (mask and two_d_array).
Thanks !
Cheers,
Dirk Engelmann
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From Dirk.Engelmann@iwr.uni-heidelberg.de Sun Feb 15 15:22:21 1998
From: Dirk.Engelmann@iwr.uni-heidelberg.de (Dirk Engelmann)
Date: Sun, 15 Feb 1998 16:22:21 +0100 (CET)
Subject: [MATRIX-SIG] umathmodule and python-1.5
Message-ID: <Pine.LNX.3.95.980215161606.11616A-100000@klimt.iwr.uni-heidelberg.de>
Hi!
Compiling NumPy-1.0b3 for python-1.5 gives the error message:
ImportError: ./umathmodule.so: undefined symbol: PyUFunc_OO_O
if Setup in Python-Modules is set to shared.
How can I get it running for shared modules ?
Thanks for any help!
Cheers,
Dirk
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From Dirk.Engelmann@iwr.uni-heidelberg.de Tue Feb 17 16:58:29 1998
From: Dirk.Engelmann@iwr.uni-heidelberg.de (Dirk Engelmann)
Date: Tue, 17 Feb 1998 17:58:29 +0100 (CET)
Subject: [MATRIX-SIG] umathmodule and python-1.5
Message-ID: <Pine.LNX.3.95.980217175629.26408B-100000@klimt.iwr.uni-heidelberg.de>
Hi!
Compiling NumPy-1.0b3 for python-1.5 gives the error message:
ImportError: ./umathmodule.so: undefined symbol: PyUFunc_OO_O
if Setup in Python-Modules is set to shared.
I'm running gcc-2.8.0 on linux (redhat 5.0, kernel 2.0.33).
How can I get it running for shared modules ?
Thanks for any help!
Cheers,
Dirk
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From dars@soton.ac.uk Wed Feb 18 11:03:18 1998
From: dars@soton.ac.uk (Dave Stinchcombe)
Date: Wed, 18 Feb 1998 11:03:18 +0000 (GMT)
Subject: [MATRIX-SIG] PyGist
Message-ID: <199802181103.LAA10247@oak.sucs.soton.ac.uk>
Hi ,
I'm trying to compile Python 1.5 with NumPy-1.0b3, NumPyLib-1.0b2,
and pygist-1.2, having first installed yorick-1.4. I have made sure that I
don't compile for Readline.
I get compilation failure with gistCmodule.c. It now finds all the headers
it needs, but I get warning messages like:
./gistCmodule.c: In function `initgistC':
./gistCmodule.c:412: `Pyv_fgets' undeclared (first use this function)
and
./gistCmodule.c: In function `pldefault':
./gistCmodule.c:2109: structure has no member named `path'
./gistCmodule.c:2109: `T_DOWN' undeclared (first use this function)
./gistCmodule.c:2109: `T_RIGHT' undeclared (first use this function)
I have probably got the line in Python-1.5/Modules/Setup in the wrong
place.
gistC gistCmodule.c -I$(YINC) -I./../NumPy/Include ...
Does anyone know were it belongs??
Also, I've been away for well over six months, and have missed the
consequence of the discussion about a drawing package associated with
NumPy, I couldn't find any references in the NumPy FAQ, or Home page.
(sorry matrix-sig faq/homepage).
Thanks for your help
Yours
Dave Stinchcombe
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From jhauser@ifm.uni-kiel.de Wed Feb 18 12:38:08 1998
From: jhauser@ifm.uni-kiel.de (Janko Hauser)
Date: Wed, 18 Feb 1998 13:38:08 +0100 (CET)
Subject: [MATRIX-SIG] PyGist
In-Reply-To: <199802181103.LAA10247@oak.sucs.soton.ac.uk>
References: <199802181103.LAA10247@oak.sucs.soton.ac.uk>
Message-ID: <m0y58lM-000szwC@lisboa.ifm.uni-kiel.de>
This is a very old version of the pygist module. Better you use the
code from the CVS tree snapshot which resides in the same directory as
pygist.
Or ask the developers at LLNL to release a new standalone version
... (hint,hint:-)
__Janko
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From Paul F. Dubois" <dubois1@llnl.gov Wed Feb 18 15:29:13 1998
From: Paul F. Dubois" <dubois1@llnl.gov (Paul F. Dubois)
Date: Wed, 18 Feb 1998 07:29:13 -0800
Subject: [MATRIX-SIG] PyGist
Message-ID: <000d01bd3c81$f803faa0$998a7380@pduboispc>
I don't need any hints. I'm working on this. We are going to try to release
all our modules in a separate format, not as part of the source tree as they
are now. My first goal is NumPy 1.0, which I expect to release the first of
next week including a Windows binary.
The original reason we made our stuff a part of the source tree was that
some of our stuff needed modifications to the source tree. They were small
changes, and the need for most of them disappeared with Python 1.5. So it is
now possible to make things into stand-alone packages. I'm sorry I have an
actual job that is getting in the way of all this (:->. Let me thank in
advance Harri Pasanen, David Ascher, Konrad Hinsen, and Guido, who have
spent considerable time on my education, especially with the Windows side.
-----Original Message-----
From: Janko Hauser <jhauser@ifm.uni-kiel.de>
To: Matrix-Sig <Matrix-sig@python.org>
Date: Wednesday, February 18, 1998 4:41 AM
Subject: [MATRIX-SIG] PyGist
>This is a very old version of the pygist module. Better you use the
>code from the CVS tree snapshot which resides in the same directory as
>pygist.
>
>Or ask the developers at LLNL to release a new standalone version
>... (hint,hint:-)
>
>
>__Janko
>
>
>_______________
>MATRIX-SIG - SIG on Matrix Math for Python
>
>send messages to: matrix-sig@python.org
>administrivia to: matrix-sig-request@python.org
>_______________
>
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Wed Feb 18 16:18:39 1998
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Wed, 18 Feb 1998 17:18:39 +0100
Subject: [MATRIX-SIG] umathmodule and python-1.5
In-Reply-To: <Pine.LNX.3.95.980217175629.26408B-100000@klimt.iwr.uni-heidelberg.de>
(message from Dirk Engelmann on Tue, 17 Feb 1998 17:58:29 +0100 (CET))
Message-ID: <199802181618.RAA07725@lmspc1.ibs.fr>
> Compiling NumPy-1.0b3 for python-1.5 gives the error message:
> ImportError: ./umathmodule.so: undefined symbol: PyUFunc_OO_O
> if Setup in Python-Modules is set to shared.
>
> I'm running gcc-2.8.0 on linux (redhat 5.0, kernel 2.0.33).
>
> How can I get it running for shared modules ?
I don't know what exactly goes wrong there, but my standard recommendation
for treating shared library problems is to use my modified NumPy
distribution at
http://starship.skyport.net/crew/hinsen/numpy.tar.gz
It handles the relation between the various dynamic libraries in a much
more portable way than the standard distribution. If you are not in a
hurry, you might also wait for the next official NumPy release, which
Paul Dubois just promised for the near future.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From phil@geog.ubc.ca Wed Feb 18 16:47:59 1998
From: phil@geog.ubc.ca (Phil Austin)
Date: Wed, 18 Feb 1998 08:47:59 -0800
Subject: [MATRIX-SIG] PyGist
In-Reply-To: <199802181103.LAA10247@oak.sucs.soton.ac.uk>
References: <199802181103.LAA10247@oak.sucs.soton.ac.uk>
Message-ID: <199802181647.IAA18349@curlew.geog.ubc.ca>
>>>>> "DS" == Dave Stinchcombe <dars@soton.ac.uk> writes:
DS> Hi , I'm trying to compile Python 1.5 with NumPy-1.0b3,
DS> NumPyLib-1.0b2, and pygist-1.2, having first installed
DS> yorick-1.4. I have made sure that I don't compile for
DS> Readline.
and (much context deleted):
>>>>> "Paul" == Paul F Dubois <dubois1@llnl.gov> writes:
Paul> I don't need any hints. I'm working on this. We are going to
Paul> try to release all our modules in a separate format, not as
Paul> part of the source tree as they are now. My first goal is
Paul> NumPy 1.0, which I expect to release the first of next week
Paul> including a Windows binary.
as an interim measure, people are invited to try my standalone install
of pygist, which I've tested on Solaris and Linux. It uses Konrad
Hinsen's compile.py and install.py to winkle out your --prefix and
--exec directories and fetch Makefile.pre.in and build gistCmodule.so
and arrayfns.so. It is missing some of
the demos, which I need to tweak to remove PDB dependencies. I'm up
against a grant proposal deadline, so this won't happen this week, but
feel free to try it and report your success/failure.
Here's the Readme, to be found with the rest of the package at:
http://www.geog.ubc.ca/~phil/python/gist.html
_________________________________________________________
This README explains how to install gist as
a dynamically loadable library on Linux and Solaris
(and presumably other Unices).
You should have received it as part of the
tar file:
http://www.geog.ubc.ca/~phil/python/ubcgist.tar.gz
Installation requires that you have installed Python 1.5,
Konrad Hinsen's distribution of Numerical Python and gist/yorick. See
http://www.geog.ubc.ca/~phil/python/gist.html for the location of these
packages.
To make the library:
1) uncompress and untar ubcgist.tar.gz. The resulting gist directory
contains this README file, C code for gistCmodule and arrayfnsmodule,
gistdemo.py (to test the plotting package) and a subdirectory
called graphics, which contains gist.py among other things.
2) cd gist
3) create the file Setup, using either Setup.linux or Setup.solaris
as a template. You need to supply the location of the Yorick
include files (-I/YORPATH/yorick/1.4/h) the library libgist.a
(-L/YORPATH/yorick/1.4/lib) and libX11.a
4) type:
python compile.py
python install.py
with luck the files:
gistCmodule.so
arrayfnsmodule.so
have been copied into EXECDIR/lib/python1.5/lib-dynload/,
where EXECDIR is the directory given to python by
./configure --prefix=INSTALLDIR --exec=EXECDIR
5) copy the graphics directory to:
INSTALLDIR/lib/python1.5/
6) Edit your .cshrc file so that PYTHONPATH includes
INSTALLDIR/lib/python1.5/graphics and
INSTALLDIR/lib/python1.5/graphics/gist i.e:
setenv PYTHONPATH .:/home/phil/lib/python:/home/phil/lib/linux/python:/home/phil/src.dir/python.dir/python-1.5/install/lib/python1.5/graphics:/home/phil/src.dir/python.dir/python-1.5/install/lib/python1.5/graphics/gist
7) Test with:
python gistdemo.py
You should be able to step through about 20 plots demonstrating some
of gist's features. For more documentation see:
http://xfiles.llnl.gov/PyGraphics/gist_frame.htm
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From Paul F. Dubois" <dubois1@llnl.gov Wed Feb 18 16:28:18 1998
From: Paul F. Dubois" <dubois1@llnl.gov (Paul F. Dubois)
Date: Wed, 18 Feb 1998 08:28:18 -0800
Subject: [MATRIX-SIG] umathmodule and python-1.5
Message-ID: <001f01bd3c8a$38d6da40$998a7380@pduboispc>
I'm sorry, I have no access to this system. I've forwarded your message to
someone who just got his Linux set up. I hope someone else on the SIG can
help.
-----Original Message-----
From: Dirk Engelmann <Dirk.Engelmann@iwr.uni-heidelberg.de>
To: sig matrix / python <matrix-sig@python.org>
Date: Tuesday, February 17, 1998 9:18 AM
Subject: [MATRIX-SIG] umathmodule and python-1.5
>Hi!
>
>Compiling NumPy-1.0b3 for python-1.5 gives the error message:
>ImportError: ./umathmodule.so: undefined symbol: PyUFunc_OO_O
>if Setup in Python-Modules is set to shared.
>
>I'm running gcc-2.8.0 on linux (redhat 5.0, kernel 2.0.33).
>
>How can I get it running for shared modules ?
>
>Thanks for any help!
>
>
>Cheers,
>
>Dirk
>
>
>
>_______________
>MATRIX-SIG - SIG on Matrix Math for Python
>
>send messages to: matrix-sig@python.org
>administrivia to: matrix-sig-request@python.org
>_______________
>
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From Paul F. Dubois" <dubois1@llnl.gov Wed Feb 18 17:23:07 1998
From: Paul F. Dubois" <dubois1@llnl.gov (Paul F. Dubois)
Date: Wed, 18 Feb 1998 09:23:07 -0800
Subject: [MATRIX-SIG] umathmodule and python-1.5
Message-ID: <004801bd3c91$e13ae620$998a7380@pduboispc>
Konrad wrote:
>I don't know what exactly goes wrong there, but my standard recommendation
>for treating shared library problems is to use my modified NumPy
>distribution at
>
> http://starship.skyport.net/crew/hinsen/numpy.tar.gz
>
>It handles the relation between the various dynamic libraries in a much
>more portable way than the standard distribution. If you are not in a
>hurry, you might also wait for the next official NumPy release, which
>Paul Dubois just promised for the near future.
>--
The diffference between what I will release and Konrad's version is very
small, consisting of one minor bug fix and the addition of stuff to make it
easy to do Windows, some readmes, etc. I would try his version with every
confidence.
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From npat@ariadne.di.uoa.gr Wed Feb 18 18:25:39 1998
From: npat@ariadne.di.uoa.gr (Nick Patavalis)
Date: Wed, 18 Feb 1998 20:25:39 +0200
Subject: [MATRIX-SIG] The Python interface to PGPLOT graphics library
In-Reply-To: <199802181751.TAA02271@ariadne.di.uoa.gr>; from Holger St. John on Wed, Feb 18, 1998 at 09:55:06AM -0800
References: <199802181751.TAA02271@ariadne.di.uoa.gr>
Message-ID: <19980218202539.52678@ariadne.di.uoa.gr>
Quoting Holger St. John (stjohn@gav.gat.com)
Subject was: Re: [MATRIX-SIG] The Python interface to PGPLOT graphics library
Date: Wed, Feb 18, 1998 at 09:55:06AM -0800
> I am trying to get your pgplot stuff working with Python. A minor problem
> that I ran into is that my (Numeric)Python distribution doesnt have a
> libnumpymodule.
> This library is apparently required by your Makefile. Can you point out
> to me what you did with NumPy to create this library?
I didn't do anything special, as I remeber. I just built the NumPy module
following the intructions in "Install.txt". BTW I used NumPy version 1.0b3,
what version are you using? Also: is your NumPy module working? I mean
can you do: "from Numeric import *", and then use the Numeric
facilities?
> I suppose I could build the library easily enough if I knew what it
> is supposed to contain.
The "libnumpymodule.so" that was created when I built my NumPy
distribution contains: "arrayobject.o", and "ufuncobject.o"
Nick
P.S. I' ll try getting the most recent version of NumPy,
build it again, and tell you if something goes wrong...
--
------------------------------------------------
Nick Patavalis Snail-mail:
(npat@ariadne.di.uoa.gr) Thironos 11,
University of Athens, 161 21 Athens,
Computer Science Dept. Hellas.
------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From npat@ariadne.di.uoa.gr Wed Feb 18 19:45:41 1998
From: npat@ariadne.di.uoa.gr (Nick Patavalis)
Date: Wed, 18 Feb 1998 21:45:41 +0200
Subject: [MATRIX-SIG] The Python interface to PGPLOT graphics library
In-Reply-To: <19980218202539.52678@ariadne.di.uoa.gr>; from Nick Patavalis on Wed, Feb 18, 1998 at 08:25:39PM +0200
References: <199802181751.TAA02271@ariadne.di.uoa.gr> <19980218202539.52678@ariadne.di.uoa.gr>
Message-ID: <19980218214541.61337@ariadne.di.uoa.gr>
Quoting Nick Patavalis (npat@ariadne.di.uoa.gr)
Subject was: Re: [MATRIX-SIG] The Python interface to PGPLOT graphics library
Date: Wed, Feb 18, 1998 at 08:25:39PM +0200
> P.S. I' ll try getting the most recent version of NumPy,
> build it again, and tell you if something goes wrong...
Here's EXACTLY what I did in order to build NumPy version 1.0b3:
1. Get NumPy-1.0b3.tar.gz, and NumPyLib-1.0b2.tar.gz from:
http://www.sls.lcs.mit.edu/jjh/numpy/NumPy-1.0b3.tar.gz
http://www.sls.lcs.mit.edu/jjh/numpy/NumPyLib-1.0b2.tar.gz
2. Unpack them in any directory:
> cd /tmp
> gzip -dc ~/downloads/NumPy-1.0b3.tar.gz | tar -xvf -
> gzip -dc ~/downloads/NumPyLib-1.0b2.tar.gz | tar -xvf -
3. Copy Makefile.pre.in from <PYTHON 1.4 SOURCE>/Misc to the Numpy
build directory:
> cd NumPy
> cp Makefile.pre.in Makefile.pre.in.orig
> cp /opt/python/src-1.4/Misc/Makefile.pre.in .
4. Create the Makefile
> make -f Makefile.pre.in boot
5. Build NumPy:
> make
I got several innocent warnings, but everything built fine.
6. Check for the existence of "libnumpymodule.so":
> ls lib*.so
libnumpymodule.so*
NOTE: In order to see the command that _actually_ created
"libnumpymodule.so" I also did:
> rm libnumpymodule.so
> make
gcc -shared arrayobject.o ufuncobject.o -o libnumpymodule.so
Hope this helps
Nick
--
------------------------------------------------
Nick Patavalis Snail-mail:
(npat@ariadne.di.uoa.gr) Thironos 11,
University of Athens, 161 21 Athens,
Computer Science Dept. Hellas.
------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From mallick@hks.com Thu Feb 19 17:22:45 1998
From: mallick@hks.com (Shashank Mallick)
Date: Thu, 19 Feb 1998 12:22:45 -0500
Subject: [MATRIX-SIG] umathmodule and exceptions
Message-ID: <9802191222.ZM22558@hks.com>
Hello,
I'm using the umath module to perform numerical computations on user-defined
types. I ran into the following behavior, which I feel is a bug in the umath
module.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
$python
Python 1.5 (#5, Feb 11 1998, 17:58:28) [C] on irix6
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> class test:
... pass
...
>>> import math
>>> math.cos(test)
Traceback (innermost last):
File "<stdin>", line 1, in ?
TypeError: illegal argument type for built-in operation
>>> import umath
WARNING: Python C API version mismatch for module umath:
This Python has API version 1007, module umath has version 1006.
>>> umath.cos(test)
Traceback (innermost last):
File "<stdin>", line 1, in ?
AttributeError: cos
>>>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is clearly a TypeError and I see no reason why an AttributeError is raised
when I perform this operation.
Any comments?
Thanks a lot,
Shashank
--
Shashank
Hibbitt, Karlsson & Sorensen, Inc. e-mail : mallick@hks.com
1080 Main Street tel : (401) 727 4200
Pawtucket, RI 02860 fax : (401) 727 4208
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From da@skivs.ski.org Thu Feb 19 17:40:03 1998
From: da@skivs.ski.org (David Ascher)
Date: Thu, 19 Feb 1998 09:40:03 -0800 (PST)
Subject: [MATRIX-SIG] umathmodule and exceptions
In-Reply-To: <9802191222.ZM22558@hks.com>
Message-ID: <Pine.SUN.3.96.980219093759.15063A-100000@skivs.ski.org>
On Thu, 19 Feb 1998, Shashank Mallick wrote:
> I'm using the umath module to perform numerical computations on user-defined
> types. I ran into the following behavior, which I feel is a bug in the umath
> module.
Hi Shashank (remember me?).
> >>> import math
> >>> math.cos(test)
> Traceback (innermost last):
> File "<stdin>", line 1, in ?
> TypeError: illegal argument type for built-in operation
> >>> import umath
> WARNING: Python C API version mismatch for module umath:
> This Python has API version 1007, module umath has version 1006.
> >>> umath.cos(test)
> Traceback (innermost last):
> File "<stdin>", line 1, in ?
> AttributeError: cos
> >>>
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> This is clearly a TypeError and I see no reason why an AttributeError is
> raised when I perform this operation.
I'm not so sure it's a TypeError -- what it looks like to me is that
there's something wrong with your umath configuration. The AttributeError
is saying that there is no 'cos' attribute in the umath module. Do a
dir(umath) and see what comes back...
But regardless of that, I'm very suspicious of the API version mismatch,
which you shouldn't be getting. Sounds to me like a recompile is needed.
--david
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From mallick@hks.com Thu Feb 19 17:48:09 1998
From: mallick@hks.com (Shashank Mallick)
Date: Thu, 19 Feb 1998 12:48:09 -0500
Subject: [MATRIX-SIG] umathmodule and exceptions
In-Reply-To: David Ascher <da@skivs.ski.org>
"Re: [MATRIX-SIG] umathmodule and exceptions" (Feb 19, 9:40am)
References: <Pine.SUN.3.96.980219093759.15063A-100000@skivs.ski.org>
Message-ID: <9802191248.ZM23029@hks.com>
Hi David,
>
> Hi Shashank (remember me?).
>
Sure do !
> I'm not so sure it's a TypeError -- what it looks like to me is that
> there's something wrong with your umath configuration. The AttributeError
> is saying that there is no 'cos' attribute in the umath module. Do a
> dir(umath) and see what comes back...
>
I'm pretty sure that cos is an attribute of the umath module:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
$python
import umaPython 1.5 (#5, Feb 11 1998, 17:58:28) [C] on irix6
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> import umath
WARNING: Python C API version mismatch for module umath:
This Python has API version 1007, module umath has version 1006.
>>> umath.cos(umath.pi)
-1.0
>>> dir(umath)
['__doc__', '__file__', '__name__', '__version__', 'absolute', 'add', 'arccos',
'arcsin', 'arctan', 'arctan2', 'boolean_and', 'boolean_or', 'boolean_xor',
'ceil', 'conjugate', 'cos', 'cosh', 'divide', 'divide_safe', 'e', 'equal',
'exp', 'fabs', 'floor', 'fmod', 'greater', 'greater_equal', 'hypot', 'invert',
'left_shift', 'less', 'less_equal', 'log', 'log10', 'logical_and',
'logical_not', 'logical_or', 'logical_xor', 'maximum', 'minimum', 'multiply',
'negative', 'not_equal', 'pi', 'power', 'remainder', 'right_shift', 'sin',
'sinh', 'sqrt', 'subtract', 'tan', 'tanh']
>>>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> But regardless of that, I'm very suspicious of the API version mismatch,
> which you shouldn't be getting. Sounds to me like a recompile is needed.
>
Let me try the recompile and see what happens.
Thanks a lot,
Shashank
--
Shashank
Hibbitt, Karlsson & Sorensen, Inc. e-mail : mallick@hks.com
1080 Main Street tel : (401) 727 4200
Pawtucket, RI 02860 fax : (401) 727 4208
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From da@skivs.ski.org Thu Feb 19 17:52:11 1998
From: da@skivs.ski.org (David Ascher)
Date: Thu, 19 Feb 1998 09:52:11 -0800 (PST)
Subject: [MATRIX-SIG] umathmodule and exceptions
In-Reply-To: <9802191248.ZM23029@hks.com>
Message-ID: <Pine.SUN.3.96.980219095106.15063B-100000@skivs.ski.org>
> I'm pretty sure that cos is an attribute of the umath module:
Found the problem. umath.cos is smarter than I realized, and will call a
method of a class instance if found. Witness:
>>> class A:
... def cos(self):
... return 1.0
...
>>> umath.cos(A())
1.0
So it is an attribute error (your class had no 'cos' method defined), but
I agree the error message could be more informative.
Hope this helps.
--da
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From pas@xis.xerox.com Thu Feb 19 18:14:38 1998
From: pas@xis.xerox.com (Perry Stoll)
Date: Thu, 19 Feb 1998 10:14:38 PST
Subject: [MATRIX-SIG] umathmodule and exceptions
Message-ID: <008401bd3d62$3dc7de40$4a4df60d@stoll-pc.xis.xerox.com>
>> This is clearly a TypeError and I see no reason why an AttributeError is
>> raised when I perform this operation.
>
>I'm not so sure it's a TypeError -- what it looks like to me is that
>there's something wrong with your umath configuration. The AttributeError
>is saying that there is no 'cos' attribute in the umath module. Do a
>dir(umath) and see what comes back...
Dave, me thinks you answered before trying it yourself...
I agree with Shashank that it should not be an AttributeError:
>>> import umath
>>> c = umath.cos # note: no error, so it's there
>>> class test:
... pass
...
>>> c(test)
Traceback (innermost last):
File "<stdin>", line 1, in ?
AttributeError: cos
BTW, it generalizes to umath.sin. I'm considering testing other functions,
but I think it would be easier to write a large automated test procedure for
an exhaustive study of the subject ;)
>But regardless of that, I'm very suspicious of the API version mismatch,
>which you shouldn't be getting
Definitely. Get this resolved before doing anything else.
-Perry
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Fri Feb 20 16:48:54 1998
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Fri, 20 Feb 1998 17:48:54 +0100
Subject: [MATRIX-SIG] LinearAlgebra
Message-ID: <199802201648.RAA21689@lmspc1.ibs.fr>
Some you may have noticed that the routines in LinearAlgebra.py
are not the most efficient ones. I wrote most of it with small
systems in mind, leaving optimization for later. As usual, "later"
didn't arrive for a while...
One of the worst offenders are singular_value_decomposition and
generalized_inverse. Using these on big matrices can quickly eat up
all memory and more.
The good news is that I have better versions, and they work. The bad
news is that the new singular_value_decomposition is not fully
compatible with the old one (generalized_inverse is). I'd like to
know if the incompatibility is a problem for anyone or not; all
my applications turned out to be unaffected.
Here's what happens: The result of a singular value decomposition
consists of two orthonormal matrices and one diagonal matrix. The
diagonal matrix is returned as a vector of the diagonal elements. For
an NxM input matrix, one of the orthonormal matrices has size NxN and
the other MxM; both represent complete bases for their respective
vector spaces. However, for all applications I can think of, the
matrices need to be of size NxK and MxK only, where K = min(N, M),
because the singular vectors corresponding to singular values that are
exactly zero are not relevant. When M and N are very different, the
memory saving is important. So the optimization is to calculate and
return only the essential singular vectors instead of a complete
basis.
Does anyone have an application that would break with such a change?
One could always introduce a second singular value function with a
different name, of course, but it would probably create a bit of
confusion for new users.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From Oliver Gathmann <gathmann@scar.utoronto.ca> Fri Feb 20 17:58:48 1998
From: Oliver Gathmann <gathmann@scar.utoronto.ca> (Oliver Gathmann)
Date: Fri, 20 Feb 1998 12:58:48 -0500 (EST)
Subject: [MATRIX-SIG] LinearAlgebra
In-Reply-To: <199802201648.RAA21689@lmspc1.ibs.fr>
Message-ID: <Pine.GSO.3.95.980220124333.4804D-100000@banks.scar>
Konrad,
how did you achieve this optimization of singular_value_decomposition? To
me, the simplest way seems to be to change the two first arguments passed
on to the dgesvd/zgesvd routines of lapack_lite from "A" ('compute all
columns') to "S" ('compute only min(n,m) columns'). If that is the
approach you have chosen, why not turn this into keyword arguments with a
default value like this:
def singular_value_decomposition(a,left='S',right='S')
and pass these keyword values into lapack_lite? This works fine for me;
and, yes, I actually did make use of the "A" option in some of my code
(although I don't recall why).
Oliver
Oliver Gathmann (gathmann@scar.utoronto.ca)
Surface and Groundwater Ecology Research Group
University of Toronto
phone: (416) - 287 7420 ; fax: (416) - 287 7423
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Fri Feb 20 18:30:22 1998
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Fri, 20 Feb 1998 19:30:22 +0100
Subject: [MATRIX-SIG] LinearAlgebra
In-Reply-To: <Pine.GSO.3.95.980220124333.4804D-100000@banks.scar> (message
from Oliver Gathmann on Fri, 20 Feb 1998 12:58:48 -0500 (EST))
Message-ID: <199802201830.TAA22188@lmspc1.ibs.fr>
> how did you achieve this optimization of singular_value_decomposition? To
> me, the simplest way seems to be to change the two first arguments passed
> on to the dgesvd/zgesvd routines of lapack_lite from "A" ('compute all
> columns') to "S" ('compute only min(n,m) columns'). If that is the
Right. The array allocations have to be changed as well, and the
array dimensions passed to the LAPACK routine, but that's it.
> approach you have chosen, why not turn this into keyword arguments with a
> default value like this:
>
> def singular_value_decomposition(a,left='S',right='S')
No problem, but that could again break old code. Or the defaults are
'A', and then hardly anyone will profit.
In practice I'd use logical variables for the optional arguments to
prevent people from passing nonsense, but that's of course a minor
detail.
Konrad.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From bsd@scripps.edu Sat Feb 21 21:34:48 1998
From: bsd@scripps.edu (Bruce Duncan)
Date: Sat, 21 Feb 1998 13:34:48 -0800 (PST)
Subject: [MATRIX-SIG] Suggestion: alias __deepcopy__ as __copy__ for arrays ??
Message-ID: <199802212134.NAA24785@solomon.scripps.edu>
Greetings,
The standard module copy will not do a deepcopy of class instances
if they contain Numeric arrays. This seems to be because the
array type does not define a __deepcopy__ function.
(Numeric does define a __copy__ function.)
However, it seems that for most cases of Numeric arrays
(float, int, byte, etc.) a __deepcopy__ is equivalent to a __copy__.
[How could a deepcopy be different than a copy for an array??]
IF, (repeat IF), this is true, then it seems that a
__deepcopy__ method can be defined as an alias to the __copy__ method.
That way, a deepcopy of objects containing arrays should work.
Comments?
-bsd-
Bruce Duncan
The Scripps Research Institute
bsd@scripps.edu
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From bsd@scripps.edu Sat Feb 21 21:51:54 1998
From: bsd@scripps.edu (Bruce Duncan)
Date: Sat, 21 Feb 1998 13:51:54 -0800 (PST)
Subject: [MATRIX-SIG] Regression tests
Message-ID: <199802212151.NAA24812@solomon.scripps.edu>
Greetings,
The python interpreter has an extensive set of regression tests.
Unfortunately, Numeric and related modules
(UserArray, LinearAlgebra, etc.) do not.
I have found several limitations, (but, as yet, no bugs) in Numeric,
however, I have found LinearAlgebra to have problems.
(The fact that I could not compute the eigenvalues of an
identity matrix (on a DEC alpha) was particularly troubling.)
I hope that in the next release of Numeric there are more extensive
regression tests and more warnings if (when) things go awry.
Liberal use of the python 1.5 assert statement should help.
In the interim, I've used SWIG to wrap some routines from
Numerical Recipes in C. These routines may not be as fast, but
may be more reliable. For scientific computations, the reliability
of the core numeric programs is CRITICAL.
-bsd-
Bruce Duncan
The Scripps Research Institute
bsd@scripps.edu
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From miller5@uiuc.edu Sun Feb 22 17:06:21 1998
From: miller5@uiuc.edu (Mike Miller)
Date: 22 Feb 1998 11:06:21 -0600
Subject: [MATRIX-SIG] a faster array io module
Message-ID: <t9yaz3po1e.fsf@uinpluxb.npl.uiuc.edu>
I decided that it was time to learn to extend python so I could
take a stab at a high speed (well at least fairly quick) module
to read tables of ASCII data in a file into NumPy arrays. So I
wrote arrayio [1].
At the moment, it contains only one function, arrayio.readASCII.
An arrayio.writeASCII is forthcoming.
readASCII does the following:
- fgets and sscanf's through the file and stores stuff that looks
like columns of numbers in a C array.
- To allow for comments in the input file, it skips any line that
contains a "#" of "!".
- makes a NumPy array from the data and returns it.
- the shape of the returned array is M x N where M is the number
of rows in the file and N is the number of columns in the input
line with the /smallest/ number of columns (that sscanf
finds).
- No attempt is made to deal with non-numeric data, other than the
commenting scheme.
I've tested this with a 6 x 9818 array that I had lying around
and found that it's fairly fast - about 18 times quicker than
Konrad Hinsen's all-python ArrayIO, which was the inspiration.
But I have some questions that maybe folks here can answer.
1) The module uses a static C array that is 40 x 10000. Can
anyone teach me to dynamically extend this as needed?
2) Is it possible to use .append somehow on Python arrays in the
C module? If I could to that, I'd just append each line as I
read it from the file.
3) Any suggestions on fancier parsing schemes?
Any other comments or suggestions are welcome.
Regards, Mike
[1] <URL:http://www.npl.uiuc.edu/~miller/python>
--
Michael A. Miller miller5@uiuc.edu
Department of Physics, University of Illinois, Urbana-Champaign
PGP public key available on request
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From krodgers@tdyryan.com Mon Feb 23 19:27:19 1998
From: krodgers@tdyryan.com (Kevin Rodgers)
Date: Mon, 23 Feb 1998 11:27:19 -0800
Subject: [MATRIX-SIG] Regression tests
In-Reply-To: <199802212151.NAA24812@solomon.scripps.edu>
Message-ID: <3.0.1.32.19980223112719.0062e4a0@gate.tdyryan.com>
At 01:51 PM 2/21/98 -0800, Bruce Duncan wrote:
>
>I hope that in the next release of Numeric there are more extensive
>regression tests and more warnings if (when) things go awry.
This would indeed be nice.
>In the interim, I've used SWIG to wrap some routines from
>Numerical Recipes in C. These routines may not be as fast, but
>may be more reliable. For scientific computations, the reliability
>of the core numeric programs is CRITICAL.
It is generally known, but apparently not as well known as it should, that
most of the exposition in NR is acceptable, but usually the specific
routines are at best naive and at worst egregiously wrong. There's an
excellent page out there (see http://math.jpl.nasa.gov/nr/ for details)
that lists many of the problems with NR. Bottom line: don't use NR for
anything that approaches mission-critical, and always check the NR
algorithm with another source to verify correctness.
----------------------------------------------------------------------
Kevin Rodgers Teledyne Ryan Aeronautical krodgers@tdyryan.com
"This one goes up to eleven." -- Nigel Tufnel
----------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From bsd@scripps.edu Mon Feb 23 20:54:09 1998
From: bsd@scripps.edu (Bruce Duncan)
Date: Mon, 23 Feb 1998 12:54:09 -0800 (PST)
Subject: [MATRIX-SIG] Follow up: Numeric/Numerical Recipes/Regression tests
Message-ID: <199802232054.MAA09307@sheba.scripps.edu>
Greetings,
Many thanks to those who reminded me of the problems
with Numerical Recipes. We are aware of many of them.
However, don't let the fact that Numerical Recipes is not
the best code in the world detract from the suggestion
that Numeric could use additional regression tests.
This is not a flame. We have found python/Numeric to
be in general quite robust, elegant, and extensible.
In fact, we are using it as the base for just about all of our new work.
However, I would like to have better "peace of mind" in knowing that
there is a set of tests for Numeric python (and associated packages),
that are as extensive as those for the python interpreter itself.
Cheers,
-bsd-
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From Paul F. Dubois" <dubois1@llnl.gov Mon Feb 23 21:32:02 1998
From: Paul F. Dubois" <dubois1@llnl.gov (Paul F. Dubois)
Date: Mon, 23 Feb 1998 13:32:02 -0800
Subject: [MATRIX-SIG] Follow up: Numeric/Numerical Recipes/Regression tests
Message-ID: <01bd40a2$7ac25e40$6c357380@xfiles.llnl.gov>
LLNL has committed to supporting NumPy; that includes trying to shape up the
source and provide a better package structure, including test routines. We
agree with your sentiments completely. It only is a matter of having enough
time to get some of this done.
I have the view that the most important reason to strive for reusable
components is not the time it saves you to write them but the reliability of
reusing a well-tested component. Especially in scientific computation, bugs
and non-linear effects are easily confounded.
-----Original Message-----
From: Bruce Duncan <bsd@scripps.edu>
To: matrix-sig@python.org <matrix-sig@python.org>
Date: Monday, February 23, 1998 12:59 PM
Subject: [MATRIX-SIG] Follow up: Numeric/Numerical Recipes/Regression tests
>Greetings,
>
>Many thanks to those who reminded me of the problems
>with Numerical Recipes. We are aware of many of them.
>
>However, don't let the fact that Numerical Recipes is not
>the best code in the world detract from the suggestion
>that Numeric could use additional regression tests.
>
>This is not a flame. We have found python/Numeric to
>be in general quite robust, elegant, and extensible.
>In fact, we are using it as the base for just about all of our new work.
>
>However, I would like to have better "peace of mind" in knowing that
>there is a set of tests for Numeric python (and associated packages),
>that are as extensive as those for the python interpreter itself.
>
>Cheers,
>
>-bsd-
>
>_______________
>MATRIX-SIG - SIG on Matrix Math for Python
>
>send messages to: matrix-sig@python.org
>administrivia to: matrix-sig-request@python.org
>_______________
>
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Tue Feb 24 09:50:08 1998
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Tue, 24 Feb 1998 10:50:08 +0100
Subject: [MATRIX-SIG] Suggestion: alias __deepcopy__ as __copy__ for arrays ??
In-Reply-To: <199802212134.NAA24785@solomon.scripps.edu> (message from Bruce
Duncan on Sat, 21 Feb 1998 13:34:48 -0800 (PST))
Message-ID: <199802240950.KAA11047@lmspc1.ibs.fr>
> The standard module copy will not do a deepcopy of class instances
> if they contain Numeric arrays. This seems to be because the
> array type does not define a __deepcopy__ function.
> (Numeric does define a __copy__ function.)
Right.
> However, it seems that for most cases of Numeric arrays
> (float, int, byte, etc.) a __deepcopy__ is equivalent to a __copy__.
> [How could a deepcopy be different than a copy for an array??]
> IF, (repeat IF), this is true, then it seems that a
> __deepcopy__ method can be defined as an alias to the __copy__ method.
> That way, a deepcopy of objects containing arrays should work.
Right again. However, there is one type of array for which deepcopy
is not equivalent to a simple copy: general object arrays. So there
must be a separate routine for deepcopy, and someone must write it...
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Tue Feb 24 09:53:07 1998
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Tue, 24 Feb 1998 10:53:07 +0100
Subject: [MATRIX-SIG] Regression tests
In-Reply-To: <199802212151.NAA24812@solomon.scripps.edu> (message from Bruce
Duncan on Sat, 21 Feb 1998 13:51:54 -0800 (PST))
Message-ID: <199802240953.KAA11060@lmspc1.ibs.fr>
> (The fact that I could not compute the eigenvalues of an
> identity matrix (on a DEC alpha) was particularly troubling.)
Indeed. Could you explain what exactly you did and what went wrong?
Such problems should be investigated. I had access to an Alpha for two
weeks recently, and I used NumPy extensively without problems (I ran
plenty of tests, but tests specific to my applications).
> I hope that in the next release of Numeric there are more extensive
> regression tests and more warnings if (when) things go awry.
Me too! The problem is just that writing regression tests is
time consuming and not very exciting...
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Tue Feb 24 10:12:46 1998
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Tue, 24 Feb 1998 11:12:46 +0100
Subject: [MATRIX-SIG] a faster array io module
In-Reply-To: <t9yaz3po1e.fsf@uinpluxb.npl.uiuc.edu> (message from Mike Miller
on 22 Feb 1998 11:06:21 -0600)
Message-ID: <199802241012.LAA11158@lmspc1.ibs.fr>
> I decided that it was time to learn to extend python so I could
> take a stab at a high speed (well at least fairly quick) module
> to read tables of ASCII data in a file into NumPy arrays. So I
> wrote arrayio [1].
Sounds good...
> But I have some questions that maybe folks here can answer.
>
> 1) The module uses a static C array that is 40 x 10000. Can
> anyone teach me to dynamically extend this as needed?
There are several solution, as always, but here is one suggestion:
1) Scan the first line to find the number of columns, then
rewind the file for real processing.
2) Create an empty Python list (PyList_New(0)).
3) For each line, allocate a 1D array (yes, a NumPy array, using
PyArray_FromDims()), read the data, and append the array to the list.
4) Convert the list of arrays to a 2D array using
PyArray_ContiguousFromObject().
That is more or less a C translation of my Python code, and it saves
you all the trouble of dynamic memory allocation in C. It should still
be a lot faster than the pure Python code because the I/O is handled
in C. In fact, I would be surprised if a "pure" C version were faster.
> 2) Is it possible to use .append somehow on Python arrays in the
> C module? If I could to that, I'd just append each line as I
> read it from the file.
Not on arrays, but on lists - that's essentially what my scheme does.
> 3) Any suggestions on fancier parsing schemes?
I don't know what you do about data types, but here's a wish list:
- Support for integer, real, and complex arrays.
- Support for Fortran-style double precision input (i.e. using a D
as the exponent marker instead of an E).
- A verification that all lines have the same number of values.
Konrad.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From miller5@uiuc.edu Thu Feb 26 18:09:18 1998
From: miller5@uiuc.edu (Mike Miller)
Date: 26 Feb 1998 12:09:18 -0600
Subject: [MATRIX-SIG] Question: [Numeric/Numerical Recipes/Regression tests]
In-Reply-To: "Paul F. Dubois"'s message of Mon, 23 Feb 1998 13:32:02 -0800
References: <01bd40a2$7ac25e40$6c357380@xfiles.llnl.gov>
Message-ID: <t967m2ck6p.fsf@uinpluxb.npl.uiuc.edu>
Something Paul Dubois said a few days ago reminded me that I
hadn't followed up here about the solution to my random.gauss
problems.
>>>>> "Paul" == Paul F Dubois <dubois1@llnl.gov> writes:
> I have the view that the most important reason to strive
> for reusable components is not the time it saves you to
> write them but the reliability of reusing a well-tested
> component.
I second this very strongly. In my experience there has been no
doubt that code that is widely used and believed to have been
well tested gets much more use than something that is less well
known, even if the latter has a lot of potential. Example: there
is a package that I've used regularly for graphics for some
years now, but I'm very hesitant to use it's built-in
computational capabilities because it had some serious problems
with memory management in it's youth. Even though those problems
are long gone, I still remember being bitten by them and a bit
shy because of it.
I bring this up here because of the problem I found with
random.gauss. It was simple to fix this routine so that it does
what it is advertised to do (there is a patch available).
Nonetheless, this has put me off somewhat because it was, IMHO,
an obvious error, visible on first glance at the distribution,
yet the author of the algorithm needed some convincing to see
that it was there.
This all came up the other day when I was trying to explain to a
perl-advocating colleague that Python is a useful object, rather
than a poor second place to perl as he thought. But I had to
admit to him that I found faulty code in my first effort to use
Python for a rigorous calculation.
This is a bit rambly, so let me get to my question: How does one
know what has been carefully tested in Python and what has been
given only a cursory look and declared correct? If someone came
to me and wanted to know about the reliability of some module,
what would I say to them?
Mike
--
Michael A. Miller miller5@uiuc.edu
Department of Physics, University of Illinois, Urbana-Champaign
PGP public key available on request
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Fri Feb 27 17:57:38 1998
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Fri, 27 Feb 1998 18:57:38 +0100
Subject: [MATRIX-SIG] Question: [Numeric/Numerical Recipes/Regression tests]
In-Reply-To: <t967m2ck6p.fsf@uinpluxb.npl.uiuc.edu> (message from Mike Miller
on 26 Feb 1998 12:09:18 -0600)
Message-ID: <199802271757.SAA32541@lmspc1.ibs.fr>
> This is a bit rambly, so let me get to my question: How does one
> know what has been carefully tested in Python and what has been
> given only a cursory look and declared correct? If someone came
> to me and wanted to know about the reliability of some module,
> what would I say to them?
You could do some statistics on how frequently specific modules/functions
are used in published Python code, and there is lots of it.
But no matter what the answer, in my experience there is little
relation between the reliability of some code and its popularity. In
my field of work (simulations of biomolecules) it is an open secret
that you have to verify the programs you are using for your specific
application, even if (like most people) you use very popular programs.
In some fields people are simply used to work with buggy code, and
some even think that this is an inevitable situation.
So the bottom line is: unless some piece of code has a widely
accepted reputation for being good, assume that it's bad and test
it for yourself.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________