From da@skivs.ski.org Sun Feb 1 01:18:16 1998 From: da@skivs.ski.org (David Ascher) Date: Sat, 31 Jan 1998 17:18:16 -0800 (PST) Subject: [MATRIX-SIG] compiling NumPy for Windows In-Reply-To: <199801312054.VAA16626@lmspc1.ibs.fr> Message-ID: On Sat, 31 Jan 1998, Konrad Hinsen wrote: > It would be nice if it were possible to distribute a single source > package for all operating systems that could be compiled and installed > by anyone with the necessary tools (compiler etc.) *automatically*, > i.e. with no manual intervention. For Unix this is already reality > (via Misc/Makefile.pre.in and a Setup file). For Windows we seem to be > far away from such a situation, if it is at all doable. Here's the issue -- with the latest Microsoft Visual compiler suite, the GUI is the predominant way to manipulate the build process. While it can (somewhat) read old makefiles, and it can export makefiles, I think there will be less and less support for that over the years -- for example, I haven't bothered to learn MS's Makefile format, since it's just easier to click here and there. Not impossible, but sort of like maintaining a .el file -- the talent is rare. =) > Except for the "shipping back" part, this looks like what I just > described for some particular compiler. Is that really such a big > problem? Well, there is no equivalent to Makefile.pre.in on PC's. There's the registry, which tells you about things like version number & where the binaries & Lib directory are, but there is no way to find out automatically where the source tree is. I noticed that Tcl/Tk's "full installation" includes the Include subdirectory and the .LIB files (equivalent to libpython15.a). Maybe that's something worth considering. The person compiling extensions shouldn't have to worry about having the source tree, and if the .h files and the .lib file were installed by the binary installer package, we could find out about it from the registry (making a Makefile.pre.in tool much easier to write). Such an extension compilation tool would be easier to write if one could assume that Mark Hammond's Registry and COM tools were installed, as they'd make talking to the registry and compiler much easier. > Another question: is it necessary to consider more than one compiler > for the Windows platform? With several different compilers, it would > already be a problem to provide installation instructions for > everyone. I haven't heard anyone on this list mention anything but Microsoft's compiler. --david _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Mon Feb 2 14:58:28 1998 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Mon, 2 Feb 1998 15:58:28 +0100 Subject: [MATRIX-SIG] Using Fortran-LAPACK on a DEC Alpha Message-ID: <199802021458.PAA26636@lmspc1.ibs.fr> Has anyone succeeded in interfacing a Fortran version of LAPACK to Python on a DEC Alpha? Whatever I do, I always get the error message "unresolved symbols" when I import a module that uses Fortran code, and it won't even tell me which symbols are undefined. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From krodgers@tdyryan.com Mon Feb 2 17:08:00 1998 From: krodgers@tdyryan.com (Kevin Rodgers) Date: Mon, 02 Feb 1998 09:08:00 -0800 Subject: [MATRIX-SIG] Data import into Numeric arrays Message-ID: <3.0.1.32.19980202090800.005321f0@gate.tdyryan.com> I have the need to process large amounts of time history data with NumPy. The data are stored as ordinary ASCII files with each line being one timepoint and its associated data. Currently, the only ways I know to get this data into NumPy all involve reading the data into a Python list (say, via readline or readlines), splitting each line into fields, converting the string representations of numbers to actual numbers, then using the array() function to build an array. This seems really inefficient. What I'd like to do is just read the data directly into NumPy from disk as a Numeric array! Similarly with writing arrays; it would be really nice to easily write this kind of data out to disk. Any ideas on this? Currently, about 80% of my scripts processing time is in the reading and writing of the datafiles. Thanks in advance . . . (BTW, if you're only suggestion is for me to store the data in a different format, thanks but no thanks; this data has to be used by lots of different people, and I am pretty much stuck with this format, as much as I'd like to move it to, say netCDF or HDF or even straight binary.) ---------------------------------------------------------------------- Kevin Rodgers Teledyne Ryan Aeronautical krodgers@tdyryan.com "This one goes up to eleven." -- Nigel Tufnel ---------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From Paul F. Dubois" The input function can be used for breaking up lines if you put commas in the file. Better is to write your own extension in C or C++ that knows how to read your files. You can use the C API to create the Python numerical arrays and then fill them up. Or, you can use the C API to Python "list" and then at the end convert to numerical arrays. Doing the read in C avoids having the loop in Python, for one thing. I do have some help for making the file in the first place. Check out PyHistory at http://xfiles.llnl.gov/pyhistory. Two different text file formats are available as output choices, along with Pact/PDB (which is like netCDF or HDF, only better). -----Original Message----- From: Kevin Rodgers To: matrix-sig@python.org Date: Monday, February 02, 1998 9:13 AM Subject: [MATRIX-SIG] Data import into Numeric arrays >I have the need to process large amounts of time history data with NumPy. >The data are stored as ordinary ASCII files with each line being one >timepoint and its associated data. Currently, the only ways I know to get >this data into NumPy all involve reading the data into a Python list (say, >via readline or readlines), splitting each line into fields, converting the >string representations of numbers to actual numbers, then using the array() >function to build an array. This seems really inefficient. What I'd like >to do is just read the data directly into NumPy from disk as a Numeric >array! Similarly with writing arrays; it would be really nice to easily >write this kind of data out to disk. Any ideas on this? Currently, about >80% of my scripts processing time is in the reading and writing of the >datafiles. Thanks in advance . . . > >(BTW, if you're only suggestion is for me to store the data in a different >format, thanks but no thanks; this data has to be used by lots of different >people, and I am pretty much stuck with this format, as much as I'd like to >move it to, say netCDF or HDF or even straight binary.) >---------------------------------------------------------------------- >Kevin Rodgers Teledyne Ryan Aeronautical krodgers@tdyryan.com >"This one goes up to eleven." -- Nigel Tufnel >---------------------------------------------------------------------- > >_______________ >MATRIX-SIG - SIG on Matrix Math for Python > >send messages to: matrix-sig@python.org >administrivia to: matrix-sig-request@python.org >_______________ > _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Mon Feb 2 19:53:20 1998 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Mon, 2 Feb 1998 20:53:20 +0100 Subject: [MATRIX-SIG] Data import into Numeric arrays In-Reply-To: <3.0.1.32.19980202090800.005321f0@gate.tdyryan.com> (message from Kevin Rodgers on Mon, 02 Feb 1998 09:08:00 -0800) Message-ID: <199802021953.UAA27894@lmspc1.ibs.fr> > The data are stored as ordinary ASCII files with each line being one > timepoint and its associated data. Currently, the only ways I know to get > this data into NumPy all involve reading the data into a Python list (say, > via readline or readlines), splitting each line into fields, converting the > string representations of numbers to actual numbers, then using the array() > function to build an array. This seems really inefficient. What I'd like Right; someone should write a nice C module for ASCII I/O of arrays. In the meantime, I can offer a ready and working "slow" version, so you can at least check how bad it is for you: --------------------------------------------------------------------------- # Array I/O to text files # # Written by Konrad Hinsen # last revision: 1998-1-29 # from TextFile import TextFile import string, Numeric def readArray(filename): data = [] for line in TextFile(filename): data.append(map(eval, string.split(line))) a = Numeric.array(data) if a.shape[0] == 1 or a.shape[1] == 1: a = Numeric.ravel(a) return a def writeArray(a, filename): file = TextFile(filename, 'w') if len(a.shape) == 1: a = a[:, Numeric.NewAxis] for line in a: for element in line: file.write(`element` + ' ') file.write('\n') file.close() --------------------------------------------------------------------------- -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Mon Feb 2 19:57:04 1998 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Mon, 2 Feb 1998 20:57:04 +0100 Subject: [MATRIX-SIG] Reading pickle files from Alpha Message-ID: <199802021957.UAA27911@lmspc1.ibs.fr> Has anyone done something that allows pickle files containing arrays from a DEC Alpha to be read on lowly 32-bit machines? It doesn't look like much work, but currently I'd be happy for any minute I needn't spend on such minor details... Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From busby@icf.llnl.gov Mon Feb 2 18:40:19 1998 From: busby@icf.llnl.gov (L. Busby) Date: Mon, 2 Feb 98 10:40:19 PST Subject: [MATRIX-SIG] ArrayPrinter problem Message-ID: <9802021840.AA24444@icf.llnl.gov.llnl.gov> [ Jeffery Collins says ] >There seems to be a problem in the ArrayPrinter module. Consider >the following: [ deleted problem demonstration ] >I was able to fix the problem by replacing (line 101) > > if max_val >= 1.e12: > exp_format = 1 > >with > > if max_val >= 1.e9: > exp_format = 1 We've got this fixed in our LLNL snapshot, although we used 1.e8 instead of 1.e9. Thanks for the report. Lee Busby _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From jhauser@ifm.uni-kiel.de Tue Feb 3 10:09:53 1998 From: jhauser@ifm.uni-kiel.de (Janko Hauser) Date: Tue, 3 Feb 1998 11:09:53 +0100 (CET) Subject: [MATRIX-SIG] Data import into Numeric arrays In-Reply-To: <3.0.1.32.19980202090800.005321f0@gate.tdyryan.com> References: <3.0.1.32.19980202090800.005321f0@gate.tdyryan.com> Message-ID: Hi, attached is a much faster version for reading, than the one of Konrad. It also seems to me, that the most inefficient part of data reading is the conversion from strings to numbers. I think the time for str=file.read() is reasonable fast. The conversion took most of the time. This is where sscanf is really good at. It is also nice to to use sscanf as an filter like in def scan(self, lineformat, typecode = Float): saved = [] line = 1 while line: try: data = sscanf(line, lineformat) if data: saved.append(data) line = self.file.readline() except: line = self.file.readline() pass return asarray(saved, typecode) with a lineformat of: a = self.file.scan(" Day = %f at iic = %f avgKE = %f avgPE = %f") This skips all lines, which are not of this format, good for report creation. You see, I have build a class for this, but this is rather clumsy at the moment, but I can mail it to you if you want. The bad part at the moment is, that I don't have something equivilant for writing. If something more general should be implemented, then the question arises, what we want. o One 1d or 2d array per file o More of these but all with the same length in the last dimension. o Can text be inserted? Also between the numbers? TeX-tables ... o Should this be a new file object, or some new methods for the arraytype? Besides Yoricks read is only ca. 2 times faster than read_table. And Matlab very similar to read_table. Not so bad at all. __Janko def read_table(fname,header=0): """read_table(filename,header=number_of_header_lines) Reads from filename after number_of_header_lines """ from sscanf import sscanf import string try: fp=open(fname,'r') except: print "Can't open file or wrong filename in function read_table" return scanline='%f' lines = fp.readlines() fp.close() n=len(string.split(lines[header])) new_table=ones((len(lines)-header,n),typecode = Float) while n-1: n=n-1 scanline=scanline+' %f' for item in xrange(len(lines)-header): a=array(sscanf(lines[item],scanline)) new_table[item,:] = a return(new_table) _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Tue Feb 3 10:50:06 1998 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Tue, 3 Feb 1998 11:50:06 +0100 Subject: [MATRIX-SIG] Pickling arrays on a DEC Alpha Message-ID: <199802031050.LAA31073@lmspc1.ibs.fr> Here's a quick hack for those who want to exchange pickle files between DEC Alphas and other machines (with 32-bit longs). If you replace the functions LoadArray() and DumpArray() in Numeric.py, arrays of longs will always be saved and retrieved in 32-bit format, even on Alphas. The modification doesn't make any difference on non-Alpha machines. --------------------------------------------------------------------------- def DumpArray(m, fp): s = m.shape if LittleEndian: endian = "L" else: endian = "B" typecode = m.typecode() # Save 64-int arrays (DEC Alpha) as 32 bit for compatibility if typecode == 'l' and m.itemsize() == 8: m = m.astype('i') fp.write("A%s%s%d " % (typecode, endian, m.itemsize())) for d in s: fp.write("%d "% d) fp.write('\n') fp.write(m.tostring()) SizeOfLong = zeros((1,), 'l').itemsize() def LoadArray(fp): ln = string.split(fp.readline()) if ln[0][0] == 'A': ln[0] = ln[0][1:] # Nasty hack showing my ignorance of pickle typecode = ln[0][0] endian = ln[0][1] shape = map(lambda x: string.atoi(x), ln[1:]) itemsize = string.atoi(ln[0][2:]) sz = reduce(multiply, shape)*itemsize data = fp.read(sz) # Read 32-bit arrays as int (not long) on DEC Alphas if typecode == 'l' and itemsize == 4 and SizeOfLong == 8: m = fromstring(data, 'i').astype('l') else: m = fromstring(data, typecode) m = reshape(m, shape) if (LittleEndian and endian == 'B') or (not LittleEndian and endian == 'L'): return m.byteswapped() else: return m --------------------------------------------------------------------------- -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Tue Feb 3 16:26:06 1998 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Tue, 3 Feb 1998 17:26:06 +0100 Subject: [MATRIX-SIG] compiling NumPy for Windows In-Reply-To: (message from David Ascher on Sat, 31 Jan 1998 17:18:16 -0800 (PST)) Message-ID: <199802031626.RAA32584@lmspc1.ibs.fr> > Here's the issue -- with the latest Microsoft Visual compiler suite, the > GUI is the predominant way to manipulate the build process. While it can > (somewhat) read old makefiles, and it can export makefiles, I think there > will be less and less support for that over the years -- for example, I I'd call that a serious problem for scientific applications, where distribution of source code is nothing unusual. Even if you adopt the Microsoft point of view that non-Windows systems don't matter, you still need executables for different chips (currently Intel and Alpha), and you can't expect every program author to have all systems available for producing binaries. > Well, there is no equivalent to Makefile.pre.in on PC's. There's the > registry, which tells you about things like version number & where the > binaries & Lib directory are, but there is no way to find out > automatically where the source tree is. You don't need the source tree for compiling extensions. If you can locate the libraries and header files, that's sufficient. > Such an extension compilation tool would be easier to write if one could > assume that Mark Hammond's Registry and COM tools were installed, as > they'd make talking to the registry and compiler much easier. That doesn't sound like an unrealistic assumption. > I haven't heard anyone on this list mention anything but Microsoft's > compiler. But there's a port of gcc plus some Unix tools to Windows, right? Would it make sense to recommend this to Windows users? It should make porting Unix stuff much easier. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From Paul F. Dubois" No, we need to supply project files for Windows and your stuff for Unix. I'm in the process of doing that. In the process I'm trying to rationalize the arrangement of the source directory. I have the Wise installer software and am learning how to use it for making the Windows installer. I'm going to proceed on the premise that the installation on each platform includes the library and include files because such a high percentage of users are also going to be extenders. I will have a proposal for you shortly about what I think the "installed" and "source" sides should look like. Suggestions are welcome, of course. Too bad I don't have more experience on Windows...here are some questions I could stand some help with. a. I believe an "extender" would need the .lib and the .pyd both. What about the .exp file? The .def? Anything else? b. How do you tell Wise to adjust Python's path in the registry? I will eventually digest all of this but any pointers would help me get this out faster. -----Original Message----- From: Konrad Hinsen To: da@skivs.ski.org Cc: matrix-sig@python.org Date: Tuesday, February 03, 1998 8:38 AM Subject: Re: [MATRIX-SIG] compiling NumPy for Windows >> Here's the issue -- with the latest Microsoft Visual compiler suite, the >> GUI is the predominant way to manipulate the build process. While it can >> (somewhat) read old makefiles, and it can export makefiles, I think there >> will be less and less support for that over the years -- for example, I > >I'd call that a serious problem for scientific applications, where >distribution of source code is nothing unusual. Even if you adopt the >Microsoft point of view that non-Windows systems don't matter, you >still need executables for different chips (currently Intel and >Alpha), and you can't expect every program author to have all systems >available for producing binaries. > >> Well, there is no equivalent to Makefile.pre.in on PC's. There's the >> registry, which tells you about things like version number & where the >> binaries & Lib directory are, but there is no way to find out >> automatically where the source tree is. > >You don't need the source tree for compiling extensions. If you can >locate the libraries and header files, that's sufficient. > >> Such an extension compilation tool would be easier to write if one could >> assume that Mark Hammond's Registry and COM tools were installed, as >> they'd make talking to the registry and compiler much easier. > >That doesn't sound like an unrealistic assumption. > >> I haven't heard anyone on this list mention anything but Microsoft's >> compiler. > >But there's a port of gcc plus some Unix tools to Windows, right? >Would it make sense to recommend this to Windows users? It should >make porting Unix stuff much easier. > >Konrad. >-- >--------------------------------------------------------------------------- ---- >Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr >Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 >Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 >41, av. des Martyrs | Deutsch/Esperanto/English/ >38027 Grenoble Cedex 1, France | Nederlands/Francais >--------------------------------------------------------------------------- ---- > >_______________ >MATRIX-SIG - SIG on Matrix Math for Python > >send messages to: matrix-sig@python.org >administrivia to: matrix-sig-request@python.org >_______________ > _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From da@skivs.ski.org Tue Feb 3 17:37:22 1998 From: da@skivs.ski.org (David Ascher) Date: Tue, 3 Feb 1998 09:37:22 -0800 (PST) Subject: [MATRIX-SIG] compiling NumPy for Windows In-Reply-To: <01bd30c6$349ffd00$998a7380@pduboispc> Message-ID: > No, we need to supply project files for Windows and your stuff for Unix. Agreed. > I'm going to proceed on the premise that the installation on each platform > includes the library and include files because such a high percentage of > users are also going to be extenders. Again, agreed -- let's get this to Guido ASAP -- the sooner we convince him, the sooner it'll happen. > a. I believe an "extender" would need the .lib and the .pyd both. What > about the .exp file? The .def? Anything else? The .LIB is created on the way to the .PYD file. The .DEF is simply there to tell the compiler which symbols to export in the .LIB. The .EXP is an intermediate file. So .PYD for Python-only runtime use, .PYD + .LIB for those cases which need access to the C API. > b. How do you tell Wise to adjust Python's path in the registry? item: Edit Registry Total Keys=1 Key=Software\Python\PythonCore\%PY_REG_VERSION%\PythonPath\%PY_APP_NAME% New Value=%INSTDIR%;%INSTDIR%\Lib Root=2 end This sort of thing. NB: 1) WISE has an upgrade which you should get -- apparently makes things much more robust. 2) We can take this offline -- I think Mark, Harri, you and myself are the only ones who've played with WISE (except for Guido, but he's busy =) _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From da@skivs.ski.org Tue Feb 3 17:56:18 1998 From: da@skivs.ski.org (David Ascher) Date: Tue, 3 Feb 1998 09:56:18 -0800 (PST) Subject: [MATRIX-SIG] compiling NumPy for Windows In-Reply-To: <199802031626.RAA32584@lmspc1.ibs.fr> Message-ID: > I'd call that a serious problem for scientific applications, where > distribution of source code is nothing unusual. Even if you adopt the > Microsoft point of view that non-Windows systems don't matter, you > still need executables for different chips (currently Intel and > Alpha), and you can't expect every program author to have all systems > available for producing binaries. No, but I think you're being pessimistic. There's really three platforms. Unix (source distribution works fine), Windows, Mac. Even if we can't come up with an automatic way of you (a non-Windows user) generating a Windows makefile, I'm pretty sure I (or others) can take your Unix makefile (provided it's not that complicated) and make a windows binary in a day at most. When there are so many extensions announced that we can't keep up, we'll deal with that bridge. I haven't coded on a mac in a while (since the days of Think C =), so I don't know what the compilation configuration issues are there. Just last weekend I've SWIGged a couple of libraries, and I have been thinking about the right way to package extension *source* distributions so that they work well with both Unix and Windows. Probably not too surprisingly, it looks a fair bit like Tcl's distribution model. > You don't need the source tree for compiling extensions. If you can > locate the libraries and header files, that's sufficient. That's my point exactly. All I meant was that we need to get them in Guido's installer so that they are there. And they need to be locatable via the registry. >> [we need COM and registry support] > That doesn't sound like an unrealistic assumption. Well, the logistics are somewhat tricky, since Mark is the author of those, and Guido releases the base install. > > I haven't heard anyone on this list mention anything but Microsoft's > > compiler. > > But there's a port of gcc plus some Unix tools to Windows, right? > Would it make sense to recommend this to Windows users? It should > make porting Unix stuff much easier. There's a couple of ports, and they are all much much too incomplete or expensive or hard to setup to recommend to the generic Windows user. It took me weeks to get my gnuwin32 setup "just right", and I have a fair bit of experience with Unix. Also, it's not at all trivial to get interoperability between gnuwin/gcc-compiled DLLs and "straight" Windows DLL's. In other words I'd recommend heartily *against* setting up a unix lookalike on Windows, except if, like me, you can't do without the shell, grep, find, etc. opinionatedly yours, --david _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From jhauser@ifm.uni-kiel.de Wed Feb 4 13:13:14 1998 From: jhauser@ifm.uni-kiel.de (Janko Hauser) Date: Wed, 4 Feb 1998 14:13:14 +0100 (CET) Subject: [MATRIX-SIG] HDF reader Message-ID: As we are at this subject, has someone a module to read HDF files, more precisely bitmap information from a HDF file? __Janko _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From npat@ariadne.di.uoa.gr Fri Feb 6 14:23:10 1998 From: npat@ariadne.di.uoa.gr (Nick Patavalis) Date: Fri, 6 Feb 1998 16:23:10 +0200 (EET) Subject: [MATRIX-SIG] The Python interface to PGPLOT graphics library Message-ID: <199802061423.QAA09424@ariadne.di.uoa.gr> A PYTHON / NUMERIC-PYTHON INTERFACE TO THE PGPLOT GRAPHICS LIBRARY. One thing I found missing from Python and Numeric Python was a good library-of-functions (module) specially designed for scientific visualization (as opposed to imaging or GUI implementation). I know there is at least one interface to Gnuplot, and other interfaces for plotting through programs like Mathematica. They are a good solution for simple tasks, and maybe even more. One has to admit, though, that they lack the power (performance) and versatility required for several more complex applications. This is true for a number of reasons: 1. They provide their services through a different task, usually communicating via temporary files. This is not the most efficient architecture, and its also not free of other problems. 2. Some of them are implemented as python scripts. Python is nice, powerful and easy to use, but it lacks the performance of C. 3. The underlying programs that actually provide the graphics capabilities are usually not general-purpose visualization tools but programs primarily designed for interactive use (e.g. Gnuplot) In order to circumvent these limitations I decided to implement an interface module that enables Numeric Python to call the functions of the (really remarkable) PGPLOT graphics library. The interface module is written in C and provides (or at least will do so soon) the full set of PGPLOT facilities, wrapped in a python interface that resembles as much as possible the original PGPLOT semantics and calling conventions. I decided to do so in order to make life easier for those who are already familiar with PGPLOT. I have also coded an alternative interface for *some* PGPLOT functions that had a rather "complicated" interface. These simplified interfaces are more close to python's idiosyncrasies, and were designed to take advantage of some of its special features (like optional arguments or keyword arguments, etc). After the original code is proven stable enough, I plan to implement an even simpler interface. It will be designed for "interactive" use (directly at the python prompt), but not only. I think of making it like Gnuplot's interface, (with a lot of functionality added). Anyway I believe its too soon to say anything more on this subject. I have NOT released the source code yet, but I plan to do so soon (maybe sometime in February). You can find some example screen-shots produced by PPGPLOT in: http://ariadne.di.uoa.gr/ppgplot , along with the python code required to produce them. For those of you who are not familiar with PGPLOT here's a short description: PGPLOT is written by Tim Pearson The PGPLOT Graphics Subroutine Library is a Fortran- or C-callable, device-independent graphics package for making simple scientific graphs. It is intended for making graphical images of publication quality with minimum effort on the part of the user. For most applications, the program can be device-independent, and the output can be directed to the appropriate device at run time. The PGPLOT library consists of two major parts: a device-independent part and a set of device-dependent ``device handler'' subroutines for output on various terminals, image displays, dot-matrix printers, laser printers, and pen plotters. Common file formats supported include PostScript and GIF. The home-page of pgplot is: http://astro.caltech.edu/~tjp/pgplot The main thing missing from ppgplot is a decent build-system, usable under many platforms and configurations. Any help on this would be greatly appreciated. Please send any comments/remarks or questions about the python interface to pgplot (ppgplot) to: ppgplot@ariadne.di.uoa.gr The amount of feedback I receive, will help me find out how many people need this module, and adjust the development effort. Regards Nick Patavalis University of Athens npat@ariadne.di.uoa.gr _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From holz@zib.de Fri Feb 6 17:05:03 1998 From: holz@zib.de (Martin Holz) Date: 06 Feb 1998 18:05:03 +0100 Subject: [MATRIX-SIG] Re: The Python interface to PGPLOT graphics library In-Reply-To: Nick Patavalis's message of Fri, 6 Feb 1998 16:23:10 +0200 (EET) References: <199802061423.QAA09424@ariadne.di.uoa.gr> Message-ID: There is a interface file for SWIG, which creates a PGPLOT module at . It is quite useful, but a simpler, more python-like interface would be interesting. /Martin _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Fri Feb 6 18:32:43 1998 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Fri, 6 Feb 1998 19:32:43 +0100 Subject: [MATRIX-SIG] New version of netCDF module Message-ID: <199802061832.TAA17475@lmspc1.ibs.fr> I have just uploaded a new version of my netCDF module to the Starship: http://starship.skyport.net/crew/hinsen/netcdf.html There are several small bug fixes, and the new version allows access to variables without dimensions, works on DEC Alphas, and should be compilable under Windows. Another innovation is a version number; the current version has the official number 1.0, and future versions will have different numbers. That should eliminate the confusion that past updates have caused. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From jhauser@ifm.uni-kiel.de Mon Feb 9 12:08:50 1998 From: jhauser@ifm.uni-kiel.de (Janko Hauser) Date: Mon, 9 Feb 1998 13:08:50 +0100 (CET) Subject: [MATRIX-SIG] Matrix to Fortran Message-ID: At the weekend I have tried to interface NumPy with some Fortran routines. I use for this the Cfortran package to prototype the interface between C and Fortran, in the hope that this will be portable. I have success with a routine wich accepts 1D arrays. The interface looks like this (I use the LLNL defines for ease): PROTOCCALLSFSUB11(RGRD1, rgrd1, INT, FLOATV, FLOATV, INT, FLOATV, FLOATV, INT, INT, FLOATV, INT, INT) #define RGRID1(nx,x,p,mx,xx,q,intpol,lw,w,lwmin,ier) \ CCALLSFSUB11(RGRD1, rgrd1, INT, FLOATV, FLOATV, INT, FLOATV, \ FLOATV, INT, INT, FLOATV, INT, INT, nx,x,p,mx,xx,q,intpol,lw,w, \ lwmin,ier) Py_Try(PyArg_ParseTuple(args, "iOOiOOiiOii", &nx, &ox, &op, &mx, &oxx, &oq, &intpol, &lw, &ow, &lwmin, &ier)); GET_ARR(ax, ox, PyArray_FLOAT, 1); . . . x = (float *) A_DATA (ax); . . . RGRID1(nx,x,p,mx,xx,q,intpol,lw,w,lwmin,ier); This is ok, but if I want to interface the 2D version of regrid I can't pass the 2D fields correctly. I pass GET_ARR(ax, ox, PyArray_FLOAT, 2); . . . x = (float *) A_DATA (ax); . . . But in the Fortran routine I can't access the field. I have looked into the result of f2c, where also a simple pointer is passed, and every index calculated from the two indices. Then, is this the right way? I hope that this will be portable. Is there another way? This are only four subroutines, so I haven't looked into SWIG yet, but I think I should. __Janko _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From da@skivs.ski.org Mon Feb 9 18:52:45 1998 From: da@skivs.ski.org (David Ascher) Date: Mon, 9 Feb 1998 10:52:45 -0800 (PST) Subject: [MATRIX-SIG] matrix storage using zlib Message-ID: Has anyone tried to use the zlib module for small storage of arrays? --da _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From miller5@uiuc.edu Tue Feb 10 20:33:09 1998 From: miller5@uiuc.edu (Mike Miller) Date: 10 Feb 1998 14:33:09 -0600 Subject: [MATRIX-SIG] Question about Numeric.convolve Message-ID: I'm (trying) to use convolve to convolute a calculation with Gaussian resolution and am having some trouble sorting things out. I have two arrays, x and f = f(x) and I'd like to convolute f(x) with a Gaussian. So I calculate an array g like this: g = Numeric.zeros( x.shape,'Float').real for i in range(len(g)): g[i] = math.exp(-(math.pow((x[i]-mu)/sigma,2))) and convolute the two arrays like this: fg = Numeric.convolve( f, g, 0 ) I've been fiddling with this for a while now, adjusting the location of the Gaussian, zero extending f and g at either end, but I can't seem to sort out where convolve puts the result in fg. I'd like to be able to plot both f and fg versus x to see the result. So, the question is - is there a way to convolute two arrays with x, rather than the array index, as the index. Or am I just totally clueless? Or both? Mike -- Michael A. Miller miller5@uiuc.edu Department of Physics, University of Illinois, Urbana-Champaign PGP public key available on request _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Wed Feb 11 18:42:16 1998 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Wed, 11 Feb 1998 19:42:16 +0100 Subject: [MATRIX-SIG] Question about Numeric.convolve In-Reply-To: (message from Mike Miller on 10 Feb 1998 14:33:09 -0600) Message-ID: <199802111842.TAA12183@lmspc1.ibs.fr> > I'm (trying) to use convolve to convolute a calculation with > Gaussian resolution and am having some trouble sorting things > out. I have two arrays, x and f = f(x) and I'd like to convolute > f(x) with a Gaussian. So I calculate an array g like this: > g = Numeric.zeros( x.shape,'Float').real > for i in range(len(g)): > g[i] = math.exp(-(math.pow((x[i]-mu)/sigma,2))) First of all, that's complicated and inefficient. NumPy can do much better: g = Numeric.exp(((x-mu)/sigma)**2) > So, the question is - is there a way to convolute two arrays with > x, rather than the array index, as the index. Or am I just > totally clueless? Or both? Numeric.convolve is not what you are looking for, as indeed it convolutes according to the array index (which is equivalent for equally-spaced data and much more efficient). A convolution for unevenly spaced data amounts to a explicit numerical integration. If that's what you want, well, Simpson's rule is easy enough to implement in Python. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From pas@xis.xerox.com Wed Feb 11 20:11:41 1998 From: pas@xis.xerox.com (Perry Stoll) Date: Wed, 11 Feb 1998 12:11:41 PST Subject: [MATRIX-SIG] Question about Numeric.convolve Message-ID: <006201bd3729$446e0650$4a4df60d@stoll-pc.xis.xerox.com> >> f(x) with a Gaussian. >> for i in range(len(g)): >> g[i] = math.exp(-(math.pow((x[i]-mu)/sigma,2))) > NumPy can do much better: > g = Numeric.exp(((x-mu)/sigma)**2) but only with the right formula ;) g = Numeric.exp( -.5 * ((x-mu)/sigma)**2) And for a normalized Gaussian, don't forget the scaling term 1.0/(sigma * Numeric.sqrt(2 * Numeric.pi)) -Perry _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From Dirk.Engelmann@iwr.uni-heidelberg.de Wed Feb 11 21:32:50 1998 From: Dirk.Engelmann@iwr.uni-heidelberg.de (Dirk Engelmann) Date: Wed, 11 Feb 1998 22:32:50 +0100 (CET) Subject: [MATRIX-SIG] 2-D convolution ? Message-ID: Hi! Is there a method for a 2 dimnesional convolution ? It could be done by fft inverse_fft2d(fft2d(mask) * fft2d(two_d_array)) but only for square matrix (mask and two_d_array). Thanks ! Cheers, Dirk Engelmann _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From Dirk.Engelmann@iwr.uni-heidelberg.de Sun Feb 15 15:22:21 1998 From: Dirk.Engelmann@iwr.uni-heidelberg.de (Dirk Engelmann) Date: Sun, 15 Feb 1998 16:22:21 +0100 (CET) Subject: [MATRIX-SIG] umathmodule and python-1.5 Message-ID: Hi! Compiling NumPy-1.0b3 for python-1.5 gives the error message: ImportError: ./umathmodule.so: undefined symbol: PyUFunc_OO_O if Setup in Python-Modules is set to shared. How can I get it running for shared modules ? Thanks for any help! Cheers, Dirk _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From Dirk.Engelmann@iwr.uni-heidelberg.de Tue Feb 17 16:58:29 1998 From: Dirk.Engelmann@iwr.uni-heidelberg.de (Dirk Engelmann) Date: Tue, 17 Feb 1998 17:58:29 +0100 (CET) Subject: [MATRIX-SIG] umathmodule and python-1.5 Message-ID: Hi! Compiling NumPy-1.0b3 for python-1.5 gives the error message: ImportError: ./umathmodule.so: undefined symbol: PyUFunc_OO_O if Setup in Python-Modules is set to shared. I'm running gcc-2.8.0 on linux (redhat 5.0, kernel 2.0.33). How can I get it running for shared modules ? Thanks for any help! Cheers, Dirk _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From dars@soton.ac.uk Wed Feb 18 11:03:18 1998 From: dars@soton.ac.uk (Dave Stinchcombe) Date: Wed, 18 Feb 1998 11:03:18 +0000 (GMT) Subject: [MATRIX-SIG] PyGist Message-ID: <199802181103.LAA10247@oak.sucs.soton.ac.uk> Hi , I'm trying to compile Python 1.5 with NumPy-1.0b3, NumPyLib-1.0b2, and pygist-1.2, having first installed yorick-1.4. I have made sure that I don't compile for Readline. I get compilation failure with gistCmodule.c. It now finds all the headers it needs, but I get warning messages like: ./gistCmodule.c: In function `initgistC': ./gistCmodule.c:412: `Pyv_fgets' undeclared (first use this function) and ./gistCmodule.c: In function `pldefault': ./gistCmodule.c:2109: structure has no member named `path' ./gistCmodule.c:2109: `T_DOWN' undeclared (first use this function) ./gistCmodule.c:2109: `T_RIGHT' undeclared (first use this function) I have probably got the line in Python-1.5/Modules/Setup in the wrong place. gistC gistCmodule.c -I$(YINC) -I./../NumPy/Include ... Does anyone know were it belongs?? Also, I've been away for well over six months, and have missed the consequence of the discussion about a drawing package associated with NumPy, I couldn't find any references in the NumPy FAQ, or Home page. (sorry matrix-sig faq/homepage). Thanks for your help Yours Dave Stinchcombe _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From jhauser@ifm.uni-kiel.de Wed Feb 18 12:38:08 1998 From: jhauser@ifm.uni-kiel.de (Janko Hauser) Date: Wed, 18 Feb 1998 13:38:08 +0100 (CET) Subject: [MATRIX-SIG] PyGist In-Reply-To: <199802181103.LAA10247@oak.sucs.soton.ac.uk> References: <199802181103.LAA10247@oak.sucs.soton.ac.uk> Message-ID: This is a very old version of the pygist module. Better you use the code from the CVS tree snapshot which resides in the same directory as pygist. Or ask the developers at LLNL to release a new standalone version ... (hint,hint:-) __Janko _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From Paul F. Dubois" I don't need any hints. I'm working on this. We are going to try to release all our modules in a separate format, not as part of the source tree as they are now. My first goal is NumPy 1.0, which I expect to release the first of next week including a Windows binary. The original reason we made our stuff a part of the source tree was that some of our stuff needed modifications to the source tree. They were small changes, and the need for most of them disappeared with Python 1.5. So it is now possible to make things into stand-alone packages. I'm sorry I have an actual job that is getting in the way of all this (:->. Let me thank in advance Harri Pasanen, David Ascher, Konrad Hinsen, and Guido, who have spent considerable time on my education, especially with the Windows side. -----Original Message----- From: Janko Hauser To: Matrix-Sig Date: Wednesday, February 18, 1998 4:41 AM Subject: [MATRIX-SIG] PyGist >This is a very old version of the pygist module. Better you use the >code from the CVS tree snapshot which resides in the same directory as >pygist. > >Or ask the developers at LLNL to release a new standalone version >... (hint,hint:-) > > >__Janko > > >_______________ >MATRIX-SIG - SIG on Matrix Math for Python > >send messages to: matrix-sig@python.org >administrivia to: matrix-sig-request@python.org >_______________ > _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Wed Feb 18 16:18:39 1998 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Wed, 18 Feb 1998 17:18:39 +0100 Subject: [MATRIX-SIG] umathmodule and python-1.5 In-Reply-To: (message from Dirk Engelmann on Tue, 17 Feb 1998 17:58:29 +0100 (CET)) Message-ID: <199802181618.RAA07725@lmspc1.ibs.fr> > Compiling NumPy-1.0b3 for python-1.5 gives the error message: > ImportError: ./umathmodule.so: undefined symbol: PyUFunc_OO_O > if Setup in Python-Modules is set to shared. > > I'm running gcc-2.8.0 on linux (redhat 5.0, kernel 2.0.33). > > How can I get it running for shared modules ? I don't know what exactly goes wrong there, but my standard recommendation for treating shared library problems is to use my modified NumPy distribution at http://starship.skyport.net/crew/hinsen/numpy.tar.gz It handles the relation between the various dynamic libraries in a much more portable way than the standard distribution. If you are not in a hurry, you might also wait for the next official NumPy release, which Paul Dubois just promised for the near future. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From phil@geog.ubc.ca Wed Feb 18 16:47:59 1998 From: phil@geog.ubc.ca (Phil Austin) Date: Wed, 18 Feb 1998 08:47:59 -0800 Subject: [MATRIX-SIG] PyGist In-Reply-To: <199802181103.LAA10247@oak.sucs.soton.ac.uk> References: <199802181103.LAA10247@oak.sucs.soton.ac.uk> Message-ID: <199802181647.IAA18349@curlew.geog.ubc.ca> >>>>> "DS" == Dave Stinchcombe writes: DS> Hi , I'm trying to compile Python 1.5 with NumPy-1.0b3, DS> NumPyLib-1.0b2, and pygist-1.2, having first installed DS> yorick-1.4. I have made sure that I don't compile for DS> Readline. and (much context deleted): >>>>> "Paul" == Paul F Dubois writes: Paul> I don't need any hints. I'm working on this. We are going to Paul> try to release all our modules in a separate format, not as Paul> part of the source tree as they are now. My first goal is Paul> NumPy 1.0, which I expect to release the first of next week Paul> including a Windows binary. as an interim measure, people are invited to try my standalone install of pygist, which I've tested on Solaris and Linux. It uses Konrad Hinsen's compile.py and install.py to winkle out your --prefix and --exec directories and fetch Makefile.pre.in and build gistCmodule.so and arrayfns.so. It is missing some of the demos, which I need to tweak to remove PDB dependencies. I'm up against a grant proposal deadline, so this won't happen this week, but feel free to try it and report your success/failure. Here's the Readme, to be found with the rest of the package at: http://www.geog.ubc.ca/~phil/python/gist.html _________________________________________________________ This README explains how to install gist as a dynamically loadable library on Linux and Solaris (and presumably other Unices). You should have received it as part of the tar file: http://www.geog.ubc.ca/~phil/python/ubcgist.tar.gz Installation requires that you have installed Python 1.5, Konrad Hinsen's distribution of Numerical Python and gist/yorick. See http://www.geog.ubc.ca/~phil/python/gist.html for the location of these packages. To make the library: 1) uncompress and untar ubcgist.tar.gz. The resulting gist directory contains this README file, C code for gistCmodule and arrayfnsmodule, gistdemo.py (to test the plotting package) and a subdirectory called graphics, which contains gist.py among other things. 2) cd gist 3) create the file Setup, using either Setup.linux or Setup.solaris as a template. You need to supply the location of the Yorick include files (-I/YORPATH/yorick/1.4/h) the library libgist.a (-L/YORPATH/yorick/1.4/lib) and libX11.a 4) type: python compile.py python install.py with luck the files: gistCmodule.so arrayfnsmodule.so have been copied into EXECDIR/lib/python1.5/lib-dynload/, where EXECDIR is the directory given to python by ./configure --prefix=INSTALLDIR --exec=EXECDIR 5) copy the graphics directory to: INSTALLDIR/lib/python1.5/ 6) Edit your .cshrc file so that PYTHONPATH includes INSTALLDIR/lib/python1.5/graphics and INSTALLDIR/lib/python1.5/graphics/gist i.e: setenv PYTHONPATH .:/home/phil/lib/python:/home/phil/lib/linux/python:/home/phil/src.dir/python.dir/python-1.5/install/lib/python1.5/graphics:/home/phil/src.dir/python.dir/python-1.5/install/lib/python1.5/graphics/gist 7) Test with: python gistdemo.py You should be able to step through about 20 plots demonstrating some of gist's features. For more documentation see: http://xfiles.llnl.gov/PyGraphics/gist_frame.htm _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From Paul F. Dubois" I'm sorry, I have no access to this system. I've forwarded your message to someone who just got his Linux set up. I hope someone else on the SIG can help. -----Original Message----- From: Dirk Engelmann To: sig matrix / python Date: Tuesday, February 17, 1998 9:18 AM Subject: [MATRIX-SIG] umathmodule and python-1.5 >Hi! > >Compiling NumPy-1.0b3 for python-1.5 gives the error message: >ImportError: ./umathmodule.so: undefined symbol: PyUFunc_OO_O >if Setup in Python-Modules is set to shared. > >I'm running gcc-2.8.0 on linux (redhat 5.0, kernel 2.0.33). > >How can I get it running for shared modules ? > >Thanks for any help! > > >Cheers, > >Dirk > > > >_______________ >MATRIX-SIG - SIG on Matrix Math for Python > >send messages to: matrix-sig@python.org >administrivia to: matrix-sig-request@python.org >_______________ > _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From Paul F. Dubois" Konrad wrote: >I don't know what exactly goes wrong there, but my standard recommendation >for treating shared library problems is to use my modified NumPy >distribution at > > http://starship.skyport.net/crew/hinsen/numpy.tar.gz > >It handles the relation between the various dynamic libraries in a much >more portable way than the standard distribution. If you are not in a >hurry, you might also wait for the next official NumPy release, which >Paul Dubois just promised for the near future. >-- The diffference between what I will release and Konrad's version is very small, consisting of one minor bug fix and the addition of stuff to make it easy to do Windows, some readmes, etc. I would try his version with every confidence. _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From npat@ariadne.di.uoa.gr Wed Feb 18 18:25:39 1998 From: npat@ariadne.di.uoa.gr (Nick Patavalis) Date: Wed, 18 Feb 1998 20:25:39 +0200 Subject: [MATRIX-SIG] The Python interface to PGPLOT graphics library In-Reply-To: <199802181751.TAA02271@ariadne.di.uoa.gr>; from Holger St. John on Wed, Feb 18, 1998 at 09:55:06AM -0800 References: <199802181751.TAA02271@ariadne.di.uoa.gr> Message-ID: <19980218202539.52678@ariadne.di.uoa.gr> Quoting Holger St. John (stjohn@gav.gat.com) Subject was: Re: [MATRIX-SIG] The Python interface to PGPLOT graphics library Date: Wed, Feb 18, 1998 at 09:55:06AM -0800 > I am trying to get your pgplot stuff working with Python. A minor problem > that I ran into is that my (Numeric)Python distribution doesnt have a > libnumpymodule. > This library is apparently required by your Makefile. Can you point out > to me what you did with NumPy to create this library? I didn't do anything special, as I remeber. I just built the NumPy module following the intructions in "Install.txt". BTW I used NumPy version 1.0b3, what version are you using? Also: is your NumPy module working? I mean can you do: "from Numeric import *", and then use the Numeric facilities? > I suppose I could build the library easily enough if I knew what it > is supposed to contain. The "libnumpymodule.so" that was created when I built my NumPy distribution contains: "arrayobject.o", and "ufuncobject.o" Nick P.S. I' ll try getting the most recent version of NumPy, build it again, and tell you if something goes wrong... -- ------------------------------------------------ Nick Patavalis Snail-mail: (npat@ariadne.di.uoa.gr) Thironos 11, University of Athens, 161 21 Athens, Computer Science Dept. Hellas. ------------------------------------------------ _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From npat@ariadne.di.uoa.gr Wed Feb 18 19:45:41 1998 From: npat@ariadne.di.uoa.gr (Nick Patavalis) Date: Wed, 18 Feb 1998 21:45:41 +0200 Subject: [MATRIX-SIG] The Python interface to PGPLOT graphics library In-Reply-To: <19980218202539.52678@ariadne.di.uoa.gr>; from Nick Patavalis on Wed, Feb 18, 1998 at 08:25:39PM +0200 References: <199802181751.TAA02271@ariadne.di.uoa.gr> <19980218202539.52678@ariadne.di.uoa.gr> Message-ID: <19980218214541.61337@ariadne.di.uoa.gr> Quoting Nick Patavalis (npat@ariadne.di.uoa.gr) Subject was: Re: [MATRIX-SIG] The Python interface to PGPLOT graphics library Date: Wed, Feb 18, 1998 at 08:25:39PM +0200 > P.S. I' ll try getting the most recent version of NumPy, > build it again, and tell you if something goes wrong... Here's EXACTLY what I did in order to build NumPy version 1.0b3: 1. Get NumPy-1.0b3.tar.gz, and NumPyLib-1.0b2.tar.gz from: http://www.sls.lcs.mit.edu/jjh/numpy/NumPy-1.0b3.tar.gz http://www.sls.lcs.mit.edu/jjh/numpy/NumPyLib-1.0b2.tar.gz 2. Unpack them in any directory: > cd /tmp > gzip -dc ~/downloads/NumPy-1.0b3.tar.gz | tar -xvf - > gzip -dc ~/downloads/NumPyLib-1.0b2.tar.gz | tar -xvf - 3. Copy Makefile.pre.in from /Misc to the Numpy build directory: > cd NumPy > cp Makefile.pre.in Makefile.pre.in.orig > cp /opt/python/src-1.4/Misc/Makefile.pre.in . 4. Create the Makefile > make -f Makefile.pre.in boot 5. Build NumPy: > make I got several innocent warnings, but everything built fine. 6. Check for the existence of "libnumpymodule.so": > ls lib*.so libnumpymodule.so* NOTE: In order to see the command that _actually_ created "libnumpymodule.so" I also did: > rm libnumpymodule.so > make gcc -shared arrayobject.o ufuncobject.o -o libnumpymodule.so Hope this helps Nick -- ------------------------------------------------ Nick Patavalis Snail-mail: (npat@ariadne.di.uoa.gr) Thironos 11, University of Athens, 161 21 Athens, Computer Science Dept. Hellas. ------------------------------------------------ _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From mallick@hks.com Thu Feb 19 17:22:45 1998 From: mallick@hks.com (Shashank Mallick) Date: Thu, 19 Feb 1998 12:22:45 -0500 Subject: [MATRIX-SIG] umathmodule and exceptions Message-ID: <9802191222.ZM22558@hks.com> Hello, I'm using the umath module to perform numerical computations on user-defined types. I ran into the following behavior, which I feel is a bug in the umath module. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ $python Python 1.5 (#5, Feb 11 1998, 17:58:28) [C] on irix6 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> class test: ... pass ... >>> import math >>> math.cos(test) Traceback (innermost last): File "", line 1, in ? TypeError: illegal argument type for built-in operation >>> import umath WARNING: Python C API version mismatch for module umath: This Python has API version 1007, module umath has version 1006. >>> umath.cos(test) Traceback (innermost last): File "", line 1, in ? AttributeError: cos >>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This is clearly a TypeError and I see no reason why an AttributeError is raised when I perform this operation. Any comments? Thanks a lot, Shashank -- Shashank Hibbitt, Karlsson & Sorensen, Inc. e-mail : mallick@hks.com 1080 Main Street tel : (401) 727 4200 Pawtucket, RI 02860 fax : (401) 727 4208 _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From da@skivs.ski.org Thu Feb 19 17:40:03 1998 From: da@skivs.ski.org (David Ascher) Date: Thu, 19 Feb 1998 09:40:03 -0800 (PST) Subject: [MATRIX-SIG] umathmodule and exceptions In-Reply-To: <9802191222.ZM22558@hks.com> Message-ID: On Thu, 19 Feb 1998, Shashank Mallick wrote: > I'm using the umath module to perform numerical computations on user-defined > types. I ran into the following behavior, which I feel is a bug in the umath > module. Hi Shashank (remember me?). > >>> import math > >>> math.cos(test) > Traceback (innermost last): > File "", line 1, in ? > TypeError: illegal argument type for built-in operation > >>> import umath > WARNING: Python C API version mismatch for module umath: > This Python has API version 1007, module umath has version 1006. > >>> umath.cos(test) > Traceback (innermost last): > File "", line 1, in ? > AttributeError: cos > >>> > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > This is clearly a TypeError and I see no reason why an AttributeError is > raised when I perform this operation. I'm not so sure it's a TypeError -- what it looks like to me is that there's something wrong with your umath configuration. The AttributeError is saying that there is no 'cos' attribute in the umath module. Do a dir(umath) and see what comes back... But regardless of that, I'm very suspicious of the API version mismatch, which you shouldn't be getting. Sounds to me like a recompile is needed. --david _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From mallick@hks.com Thu Feb 19 17:48:09 1998 From: mallick@hks.com (Shashank Mallick) Date: Thu, 19 Feb 1998 12:48:09 -0500 Subject: [MATRIX-SIG] umathmodule and exceptions In-Reply-To: David Ascher "Re: [MATRIX-SIG] umathmodule and exceptions" (Feb 19, 9:40am) References: Message-ID: <9802191248.ZM23029@hks.com> Hi David, > > Hi Shashank (remember me?). > Sure do ! > I'm not so sure it's a TypeError -- what it looks like to me is that > there's something wrong with your umath configuration. The AttributeError > is saying that there is no 'cos' attribute in the umath module. Do a > dir(umath) and see what comes back... > I'm pretty sure that cos is an attribute of the umath module: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ $python import umaPython 1.5 (#5, Feb 11 1998, 17:58:28) [C] on irix6 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> import umath WARNING: Python C API version mismatch for module umath: This Python has API version 1007, module umath has version 1006. >>> umath.cos(umath.pi) -1.0 >>> dir(umath) ['__doc__', '__file__', '__name__', '__version__', 'absolute', 'add', 'arccos', 'arcsin', 'arctan', 'arctan2', 'boolean_and', 'boolean_or', 'boolean_xor', 'ceil', 'conjugate', 'cos', 'cosh', 'divide', 'divide_safe', 'e', 'equal', 'exp', 'fabs', 'floor', 'fmod', 'greater', 'greater_equal', 'hypot', 'invert', 'left_shift', 'less', 'less_equal', 'log', 'log10', 'logical_and', 'logical_not', 'logical_or', 'logical_xor', 'maximum', 'minimum', 'multiply', 'negative', 'not_equal', 'pi', 'power', 'remainder', 'right_shift', 'sin', 'sinh', 'sqrt', 'subtract', 'tan', 'tanh'] >>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > But regardless of that, I'm very suspicious of the API version mismatch, > which you shouldn't be getting. Sounds to me like a recompile is needed. > Let me try the recompile and see what happens. Thanks a lot, Shashank -- Shashank Hibbitt, Karlsson & Sorensen, Inc. e-mail : mallick@hks.com 1080 Main Street tel : (401) 727 4200 Pawtucket, RI 02860 fax : (401) 727 4208 _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From da@skivs.ski.org Thu Feb 19 17:52:11 1998 From: da@skivs.ski.org (David Ascher) Date: Thu, 19 Feb 1998 09:52:11 -0800 (PST) Subject: [MATRIX-SIG] umathmodule and exceptions In-Reply-To: <9802191248.ZM23029@hks.com> Message-ID: > I'm pretty sure that cos is an attribute of the umath module: Found the problem. umath.cos is smarter than I realized, and will call a method of a class instance if found. Witness: >>> class A: ... def cos(self): ... return 1.0 ... >>> umath.cos(A()) 1.0 So it is an attribute error (your class had no 'cos' method defined), but I agree the error message could be more informative. Hope this helps. --da _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From pas@xis.xerox.com Thu Feb 19 18:14:38 1998 From: pas@xis.xerox.com (Perry Stoll) Date: Thu, 19 Feb 1998 10:14:38 PST Subject: [MATRIX-SIG] umathmodule and exceptions Message-ID: <008401bd3d62$3dc7de40$4a4df60d@stoll-pc.xis.xerox.com> >> This is clearly a TypeError and I see no reason why an AttributeError is >> raised when I perform this operation. > >I'm not so sure it's a TypeError -- what it looks like to me is that >there's something wrong with your umath configuration. The AttributeError >is saying that there is no 'cos' attribute in the umath module. Do a >dir(umath) and see what comes back... Dave, me thinks you answered before trying it yourself... I agree with Shashank that it should not be an AttributeError: >>> import umath >>> c = umath.cos # note: no error, so it's there >>> class test: ... pass ... >>> c(test) Traceback (innermost last): File "", line 1, in ? AttributeError: cos BTW, it generalizes to umath.sin. I'm considering testing other functions, but I think it would be easier to write a large automated test procedure for an exhaustive study of the subject ;) >But regardless of that, I'm very suspicious of the API version mismatch, >which you shouldn't be getting Definitely. Get this resolved before doing anything else. -Perry _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Fri Feb 20 16:48:54 1998 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Fri, 20 Feb 1998 17:48:54 +0100 Subject: [MATRIX-SIG] LinearAlgebra Message-ID: <199802201648.RAA21689@lmspc1.ibs.fr> Some you may have noticed that the routines in LinearAlgebra.py are not the most efficient ones. I wrote most of it with small systems in mind, leaving optimization for later. As usual, "later" didn't arrive for a while... One of the worst offenders are singular_value_decomposition and generalized_inverse. Using these on big matrices can quickly eat up all memory and more. The good news is that I have better versions, and they work. The bad news is that the new singular_value_decomposition is not fully compatible with the old one (generalized_inverse is). I'd like to know if the incompatibility is a problem for anyone or not; all my applications turned out to be unaffected. Here's what happens: The result of a singular value decomposition consists of two orthonormal matrices and one diagonal matrix. The diagonal matrix is returned as a vector of the diagonal elements. For an NxM input matrix, one of the orthonormal matrices has size NxN and the other MxM; both represent complete bases for their respective vector spaces. However, for all applications I can think of, the matrices need to be of size NxK and MxK only, where K = min(N, M), because the singular vectors corresponding to singular values that are exactly zero are not relevant. When M and N are very different, the memory saving is important. So the optimization is to calculate and return only the essential singular vectors instead of a complete basis. Does anyone have an application that would break with such a change? One could always introduce a second singular value function with a different name, of course, but it would probably create a bit of confusion for new users. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From Oliver Gathmann Fri Feb 20 17:58:48 1998 From: Oliver Gathmann (Oliver Gathmann) Date: Fri, 20 Feb 1998 12:58:48 -0500 (EST) Subject: [MATRIX-SIG] LinearAlgebra In-Reply-To: <199802201648.RAA21689@lmspc1.ibs.fr> Message-ID: Konrad, how did you achieve this optimization of singular_value_decomposition? To me, the simplest way seems to be to change the two first arguments passed on to the dgesvd/zgesvd routines of lapack_lite from "A" ('compute all columns') to "S" ('compute only min(n,m) columns'). If that is the approach you have chosen, why not turn this into keyword arguments with a default value like this: def singular_value_decomposition(a,left='S',right='S') and pass these keyword values into lapack_lite? This works fine for me; and, yes, I actually did make use of the "A" option in some of my code (although I don't recall why). Oliver Oliver Gathmann (gathmann@scar.utoronto.ca) Surface and Groundwater Ecology Research Group University of Toronto phone: (416) - 287 7420 ; fax: (416) - 287 7423 _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Fri Feb 20 18:30:22 1998 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Fri, 20 Feb 1998 19:30:22 +0100 Subject: [MATRIX-SIG] LinearAlgebra In-Reply-To: (message from Oliver Gathmann on Fri, 20 Feb 1998 12:58:48 -0500 (EST)) Message-ID: <199802201830.TAA22188@lmspc1.ibs.fr> > how did you achieve this optimization of singular_value_decomposition? To > me, the simplest way seems to be to change the two first arguments passed > on to the dgesvd/zgesvd routines of lapack_lite from "A" ('compute all > columns') to "S" ('compute only min(n,m) columns'). If that is the Right. The array allocations have to be changed as well, and the array dimensions passed to the LAPACK routine, but that's it. > approach you have chosen, why not turn this into keyword arguments with a > default value like this: > > def singular_value_decomposition(a,left='S',right='S') No problem, but that could again break old code. Or the defaults are 'A', and then hardly anyone will profit. In practice I'd use logical variables for the optional arguments to prevent people from passing nonsense, but that's of course a minor detail. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From bsd@scripps.edu Sat Feb 21 21:34:48 1998 From: bsd@scripps.edu (Bruce Duncan) Date: Sat, 21 Feb 1998 13:34:48 -0800 (PST) Subject: [MATRIX-SIG] Suggestion: alias __deepcopy__ as __copy__ for arrays ?? Message-ID: <199802212134.NAA24785@solomon.scripps.edu> Greetings, The standard module copy will not do a deepcopy of class instances if they contain Numeric arrays. This seems to be because the array type does not define a __deepcopy__ function. (Numeric does define a __copy__ function.) However, it seems that for most cases of Numeric arrays (float, int, byte, etc.) a __deepcopy__ is equivalent to a __copy__. [How could a deepcopy be different than a copy for an array??] IF, (repeat IF), this is true, then it seems that a __deepcopy__ method can be defined as an alias to the __copy__ method. That way, a deepcopy of objects containing arrays should work. Comments? -bsd- Bruce Duncan The Scripps Research Institute bsd@scripps.edu _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From bsd@scripps.edu Sat Feb 21 21:51:54 1998 From: bsd@scripps.edu (Bruce Duncan) Date: Sat, 21 Feb 1998 13:51:54 -0800 (PST) Subject: [MATRIX-SIG] Regression tests Message-ID: <199802212151.NAA24812@solomon.scripps.edu> Greetings, The python interpreter has an extensive set of regression tests. Unfortunately, Numeric and related modules (UserArray, LinearAlgebra, etc.) do not. I have found several limitations, (but, as yet, no bugs) in Numeric, however, I have found LinearAlgebra to have problems. (The fact that I could not compute the eigenvalues of an identity matrix (on a DEC alpha) was particularly troubling.) I hope that in the next release of Numeric there are more extensive regression tests and more warnings if (when) things go awry. Liberal use of the python 1.5 assert statement should help. In the interim, I've used SWIG to wrap some routines from Numerical Recipes in C. These routines may not be as fast, but may be more reliable. For scientific computations, the reliability of the core numeric programs is CRITICAL. -bsd- Bruce Duncan The Scripps Research Institute bsd@scripps.edu _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From miller5@uiuc.edu Sun Feb 22 17:06:21 1998 From: miller5@uiuc.edu (Mike Miller) Date: 22 Feb 1998 11:06:21 -0600 Subject: [MATRIX-SIG] a faster array io module Message-ID: I decided that it was time to learn to extend python so I could take a stab at a high speed (well at least fairly quick) module to read tables of ASCII data in a file into NumPy arrays. So I wrote arrayio [1]. At the moment, it contains only one function, arrayio.readASCII. An arrayio.writeASCII is forthcoming. readASCII does the following: - fgets and sscanf's through the file and stores stuff that looks like columns of numbers in a C array. - To allow for comments in the input file, it skips any line that contains a "#" of "!". - makes a NumPy array from the data and returns it. - the shape of the returned array is M x N where M is the number of rows in the file and N is the number of columns in the input line with the /smallest/ number of columns (that sscanf finds). - No attempt is made to deal with non-numeric data, other than the commenting scheme. I've tested this with a 6 x 9818 array that I had lying around and found that it's fairly fast - about 18 times quicker than Konrad Hinsen's all-python ArrayIO, which was the inspiration. But I have some questions that maybe folks here can answer. 1) The module uses a static C array that is 40 x 10000. Can anyone teach me to dynamically extend this as needed? 2) Is it possible to use .append somehow on Python arrays in the C module? If I could to that, I'd just append each line as I read it from the file. 3) Any suggestions on fancier parsing schemes? Any other comments or suggestions are welcome. Regards, Mike [1] -- Michael A. Miller miller5@uiuc.edu Department of Physics, University of Illinois, Urbana-Champaign PGP public key available on request _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From krodgers@tdyryan.com Mon Feb 23 19:27:19 1998 From: krodgers@tdyryan.com (Kevin Rodgers) Date: Mon, 23 Feb 1998 11:27:19 -0800 Subject: [MATRIX-SIG] Regression tests In-Reply-To: <199802212151.NAA24812@solomon.scripps.edu> Message-ID: <3.0.1.32.19980223112719.0062e4a0@gate.tdyryan.com> At 01:51 PM 2/21/98 -0800, Bruce Duncan wrote: > >I hope that in the next release of Numeric there are more extensive >regression tests and more warnings if (when) things go awry. This would indeed be nice. >In the interim, I've used SWIG to wrap some routines from >Numerical Recipes in C. These routines may not be as fast, but >may be more reliable. For scientific computations, the reliability >of the core numeric programs is CRITICAL. It is generally known, but apparently not as well known as it should, that most of the exposition in NR is acceptable, but usually the specific routines are at best naive and at worst egregiously wrong. There's an excellent page out there (see http://math.jpl.nasa.gov/nr/ for details) that lists many of the problems with NR. Bottom line: don't use NR for anything that approaches mission-critical, and always check the NR algorithm with another source to verify correctness. ---------------------------------------------------------------------- Kevin Rodgers Teledyne Ryan Aeronautical krodgers@tdyryan.com "This one goes up to eleven." -- Nigel Tufnel ---------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From bsd@scripps.edu Mon Feb 23 20:54:09 1998 From: bsd@scripps.edu (Bruce Duncan) Date: Mon, 23 Feb 1998 12:54:09 -0800 (PST) Subject: [MATRIX-SIG] Follow up: Numeric/Numerical Recipes/Regression tests Message-ID: <199802232054.MAA09307@sheba.scripps.edu> Greetings, Many thanks to those who reminded me of the problems with Numerical Recipes. We are aware of many of them. However, don't let the fact that Numerical Recipes is not the best code in the world detract from the suggestion that Numeric could use additional regression tests. This is not a flame. We have found python/Numeric to be in general quite robust, elegant, and extensible. In fact, we are using it as the base for just about all of our new work. However, I would like to have better "peace of mind" in knowing that there is a set of tests for Numeric python (and associated packages), that are as extensive as those for the python interpreter itself. Cheers, -bsd- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From Paul F. Dubois" LLNL has committed to supporting NumPy; that includes trying to shape up the source and provide a better package structure, including test routines. We agree with your sentiments completely. It only is a matter of having enough time to get some of this done. I have the view that the most important reason to strive for reusable components is not the time it saves you to write them but the reliability of reusing a well-tested component. Especially in scientific computation, bugs and non-linear effects are easily confounded. -----Original Message----- From: Bruce Duncan To: matrix-sig@python.org Date: Monday, February 23, 1998 12:59 PM Subject: [MATRIX-SIG] Follow up: Numeric/Numerical Recipes/Regression tests >Greetings, > >Many thanks to those who reminded me of the problems >with Numerical Recipes. We are aware of many of them. > >However, don't let the fact that Numerical Recipes is not >the best code in the world detract from the suggestion >that Numeric could use additional regression tests. > >This is not a flame. We have found python/Numeric to >be in general quite robust, elegant, and extensible. >In fact, we are using it as the base for just about all of our new work. > >However, I would like to have better "peace of mind" in knowing that >there is a set of tests for Numeric python (and associated packages), >that are as extensive as those for the python interpreter itself. > >Cheers, > >-bsd- > >_______________ >MATRIX-SIG - SIG on Matrix Math for Python > >send messages to: matrix-sig@python.org >administrivia to: matrix-sig-request@python.org >_______________ > _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Tue Feb 24 09:50:08 1998 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Tue, 24 Feb 1998 10:50:08 +0100 Subject: [MATRIX-SIG] Suggestion: alias __deepcopy__ as __copy__ for arrays ?? In-Reply-To: <199802212134.NAA24785@solomon.scripps.edu> (message from Bruce Duncan on Sat, 21 Feb 1998 13:34:48 -0800 (PST)) Message-ID: <199802240950.KAA11047@lmspc1.ibs.fr> > The standard module copy will not do a deepcopy of class instances > if they contain Numeric arrays. This seems to be because the > array type does not define a __deepcopy__ function. > (Numeric does define a __copy__ function.) Right. > However, it seems that for most cases of Numeric arrays > (float, int, byte, etc.) a __deepcopy__ is equivalent to a __copy__. > [How could a deepcopy be different than a copy for an array??] > IF, (repeat IF), this is true, then it seems that a > __deepcopy__ method can be defined as an alias to the __copy__ method. > That way, a deepcopy of objects containing arrays should work. Right again. However, there is one type of array for which deepcopy is not equivalent to a simple copy: general object arrays. So there must be a separate routine for deepcopy, and someone must write it... -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Tue Feb 24 09:53:07 1998 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Tue, 24 Feb 1998 10:53:07 +0100 Subject: [MATRIX-SIG] Regression tests In-Reply-To: <199802212151.NAA24812@solomon.scripps.edu> (message from Bruce Duncan on Sat, 21 Feb 1998 13:51:54 -0800 (PST)) Message-ID: <199802240953.KAA11060@lmspc1.ibs.fr> > (The fact that I could not compute the eigenvalues of an > identity matrix (on a DEC alpha) was particularly troubling.) Indeed. Could you explain what exactly you did and what went wrong? Such problems should be investigated. I had access to an Alpha for two weeks recently, and I used NumPy extensively without problems (I ran plenty of tests, but tests specific to my applications). > I hope that in the next release of Numeric there are more extensive > regression tests and more warnings if (when) things go awry. Me too! The problem is just that writing regression tests is time consuming and not very exciting... -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Tue Feb 24 10:12:46 1998 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Tue, 24 Feb 1998 11:12:46 +0100 Subject: [MATRIX-SIG] a faster array io module In-Reply-To: (message from Mike Miller on 22 Feb 1998 11:06:21 -0600) Message-ID: <199802241012.LAA11158@lmspc1.ibs.fr> > I decided that it was time to learn to extend python so I could > take a stab at a high speed (well at least fairly quick) module > to read tables of ASCII data in a file into NumPy arrays. So I > wrote arrayio [1]. Sounds good... > But I have some questions that maybe folks here can answer. > > 1) The module uses a static C array that is 40 x 10000. Can > anyone teach me to dynamically extend this as needed? There are several solution, as always, but here is one suggestion: 1) Scan the first line to find the number of columns, then rewind the file for real processing. 2) Create an empty Python list (PyList_New(0)). 3) For each line, allocate a 1D array (yes, a NumPy array, using PyArray_FromDims()), read the data, and append the array to the list. 4) Convert the list of arrays to a 2D array using PyArray_ContiguousFromObject(). That is more or less a C translation of my Python code, and it saves you all the trouble of dynamic memory allocation in C. It should still be a lot faster than the pure Python code because the I/O is handled in C. In fact, I would be surprised if a "pure" C version were faster. > 2) Is it possible to use .append somehow on Python arrays in the > C module? If I could to that, I'd just append each line as I > read it from the file. Not on arrays, but on lists - that's essentially what my scheme does. > 3) Any suggestions on fancier parsing schemes? I don't know what you do about data types, but here's a wish list: - Support for integer, real, and complex arrays. - Support for Fortran-style double precision input (i.e. using a D as the exponent marker instead of an E). - A verification that all lines have the same number of values. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From miller5@uiuc.edu Thu Feb 26 18:09:18 1998 From: miller5@uiuc.edu (Mike Miller) Date: 26 Feb 1998 12:09:18 -0600 Subject: [MATRIX-SIG] Question: [Numeric/Numerical Recipes/Regression tests] In-Reply-To: "Paul F. Dubois"'s message of Mon, 23 Feb 1998 13:32:02 -0800 References: <01bd40a2$7ac25e40$6c357380@xfiles.llnl.gov> Message-ID: Something Paul Dubois said a few days ago reminded me that I hadn't followed up here about the solution to my random.gauss problems. >>>>> "Paul" == Paul F Dubois writes: > I have the view that the most important reason to strive > for reusable components is not the time it saves you to > write them but the reliability of reusing a well-tested > component. I second this very strongly. In my experience there has been no doubt that code that is widely used and believed to have been well tested gets much more use than something that is less well known, even if the latter has a lot of potential. Example: there is a package that I've used regularly for graphics for some years now, but I'm very hesitant to use it's built-in computational capabilities because it had some serious problems with memory management in it's youth. Even though those problems are long gone, I still remember being bitten by them and a bit shy because of it. I bring this up here because of the problem I found with random.gauss. It was simple to fix this routine so that it does what it is advertised to do (there is a patch available). Nonetheless, this has put me off somewhat because it was, IMHO, an obvious error, visible on first glance at the distribution, yet the author of the algorithm needed some convincing to see that it was there. This all came up the other day when I was trying to explain to a perl-advocating colleague that Python is a useful object, rather than a poor second place to perl as he thought. But I had to admit to him that I found faulty code in my first effort to use Python for a rigorous calculation. This is a bit rambly, so let me get to my question: How does one know what has been carefully tested in Python and what has been given only a cursory look and declared correct? If someone came to me and wanted to know about the reliability of some module, what would I say to them? Mike -- Michael A. Miller miller5@uiuc.edu Department of Physics, University of Illinois, Urbana-Champaign PGP public key available on request _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Fri Feb 27 17:57:38 1998 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Fri, 27 Feb 1998 18:57:38 +0100 Subject: [MATRIX-SIG] Question: [Numeric/Numerical Recipes/Regression tests] In-Reply-To: (message from Mike Miller on 26 Feb 1998 12:09:18 -0600) Message-ID: <199802271757.SAA32541@lmspc1.ibs.fr> > This is a bit rambly, so let me get to my question: How does one > know what has been carefully tested in Python and what has been > given only a cursory look and declared correct? If someone came > to me and wanted to know about the reliability of some module, > what would I say to them? You could do some statistics on how frequently specific modules/functions are used in published Python code, and there is lots of it. But no matter what the answer, in my experience there is little relation between the reliability of some code and its popularity. In my field of work (simulations of biomolecules) it is an open secret that you have to verify the programs you are using for your specific application, even if (like most people) you use very popular programs. In some fields people are simply used to work with buggy code, and some even think that this is an inevitable situation. So the bottom line is: unless some piece of code has a widely accepted reputation for being good, assume that it's bad and test it for yourself. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________