From dubois1@llnl.gov Sat Jan 2 18:47:48 1999 From: dubois1@llnl.gov (Paul F. Dubois) Date: Sat, 2 Jan 1999 10:47:48 -0800 Subject: [Matrix-SIG] LLNL release Message-ID: <001b01be3680$64f76040$f4160218@c1004579-c.plstn1.sfba.home.com> On Dec. 28 I put up a new LLNL release. I previously attempted to post an announcement but it seems to have never appeared. The site is ftp://ftp-icf.llnl.gov/pub/Python. Thanks to Travis Oliphant and John Machin who contributed NumPy bug fixes (MLab.py, diagonal(a, n) with n != 0, respectively). The release also contains a fix to CXX to match Python 1.5.2. If you don't use diagonal or MLab.py or CXX you can give this release a pass. From Oliphant.Travis@mayo.edu Sun Jan 3 00:14:50 1999 From: Oliphant.Travis@mayo.edu (Travis Oliphant) Date: Sat, 2 Jan 1999 18:14:50 -0600 (CST) Subject: [Matrix-SIG] Python bindings to FFTW updated to Version 0.5 Message-ID: Python bindings for FFTW Version 0.5 FFTW is a very fast implementation of the Fast Fourier Transform written in C. I have wrapped up the library so that is available to python. It depends on the Numerical Extension module as that is the object used to pass data back and forth to the module. I have updated the python bindings to FFTW to bring them up-to-date with the latest FFTW-2.0.1 release. Also included is a module called FFT2.py which is a drop-in replacement for FFT.py in the Numerical Python distribution. An included benchmark script shows FFT2 to be 25-30% faster than FFT on my pentium-based systems. Tar files and newly available Linux-based-RPMS are available at http://oliphant.netpedia.net Oliphant.Travis@mayo.edu

fftw-numpy 1.0 - Python bindings to FFTW-2.0.1 for NumPy (02-Jan-99)

---------------------------------------------------- Travis Oliphant 200 First St SW Rochester MN 55905 Ultrasound Research Lab (507) 286-5293 Mayo Graduate School Oliphant.Travis@mayo.edu From alan@groucho.med.jhmi.edu Mon Jan 4 14:00:56 1999 From: alan@groucho.med.jhmi.edu (Alan Grossfield) Date: Mon, 04 Jan 1999 09:00:56 -0500 Subject: [Matrix-SIG] parallelized NumPy? Message-ID: <199901041400.JAA14675@groucho.med.jhmi.edu> I'm curious -- is anyone working on a parallelized version of NumPy? I poked around the various archives but didn't turn anything up. Alan Grossfield -------------------------------------------------------------------------- |"In theory, there is no difference between theory and practice. In | |practice, there is." Jan L.A. van de Snepscheut | -------------------------------------------------------------------------- From dubois1@llnl.gov Tue Jan 5 00:28:25 1999 From: dubois1@llnl.gov (Paul F. Dubois) Date: Mon, 4 Jan 1999 16:28:25 -0800 Subject: [Matrix-SIG] parallelized NumPy? Message-ID: <003701be3842$4f903380$f4160218@c1004579-c.plstn1.sfba.home.com> -----Original Message----- From: Alan Grossfield To: matrix-sig@python.org Date: Monday, January 04, 1999 6:03 AM Subject: [Matrix-SIG] parallelized NumPy? >I'm curious -- is anyone working on a parallelized version of NumPy? >I poked around the various archives but didn't turn anything up. > If you mean, a numerical object that is actually a distributed array, no. My best guess is that the quickest means to that end would involve using LANL's data-parallel arrays in their POOMA project. See http://www.acl.lanl.gov/software. If you mean arrays that do their operations on shared data with many threads, no. From jac@lanl.gov Tue Jan 5 18:01:09 1999 From: jac@lanl.gov (James A. Crotinger) Date: Tue, 05 Jan 1999 11:01:09 -0700 Subject: [Matrix-SIG] parallelized NumPy? Message-ID: <199901051800.LAA00571@cic-mail.lanl.gov> ---------- >From: "Paul F. Dubois" >To: , "Alan Grossfield" >Subject: Re: [Matrix-SIG] parallelized NumPy? >Date: Mon, Jan 4, 1999, 5:28 PM > > >-----Original Message----- >From: Alan Grossfield >To: matrix-sig@python.org >Date: Monday, January 04, 1999 6:03 AM >Subject: [Matrix-SIG] parallelized NumPy? > > >>I'm curious -- is anyone working on a parallelized version of NumPy? >>I poked around the various archives but didn't turn anything up. >> >If you mean, a numerical object that is actually a distributed array, no. My >best guess is that the quickest means to that end would involve using LANL's >data-parallel arrays in their POOMA project. >See http://www.acl.lanl.gov/software. If you mean arrays that do their >operations on shared data with many threads, no. Actually, this is exactly what the initial release of the Pooma II Array class does for Arrays with "multi-patch Engines"; i.e. Pooma still expects the user to do the domain decomposition. Given a set of domain-decomposed arrays, Pooma can use threads to run array expressions involving these arrays in parallel. HOWEVER, like Blitz, Pooma II is using expression templates to analyze the expressions and build the "runnable" objects that are passed off to a work queue for threads to work on. It isn't obvious to me that this can be done with Python wrappers around Pooma II arrays, short of doing run-time code generation. One could, of course, set up all of the basic binary operations in such a way that they could be done in a threaded fashion and then have Python operators just call the compiled Pooma operators, which would then run in parallel, but they would still get the binary expression hit that expression templates is trying to avoid. For more information on Pooma II Arrays, take a look at http://www.acl.lanl.gov/pooma/papers/GenericProgrammingPaper/dagstuhl.pdf http://www.acl.lanl.gov/pooma/papers/iscope98.pdf http://www.acl.lanl.gov/pooma/papers/GenericProgrammingInPOOMA Steve Karmesin also gave an excellent presentation at ISCOPE 98 that will be put up on the web site in the near future. You can download the first release of the Pooma II framework from http://www.acl.lanl.gov/pooma/download.html This release has shared-memory parallel arrays that have currently only been tested on the SGIs, where we use a custom light-weight thread package. We're working on an implementation that sits on top of PThreads. We have a version of the light-weight thread package for LINUX, but we've had problems with the thread safety of glibc, owing to the fact that we also use a custom mutex. (A spin-lock). We hope to have shared-memory parallelism working on LINUX in the near future. Our long-term goal is to support cross-box distributed arrays (our primary platform is a collection of 48 128-processor Origin 2000s), but this will probably not be done until the second half of '99 as we're first trying to get the full Pooma I functionality working on-box (particles, fields, and other features used in writing Pooma applications codes). For more information, check out the web site (www.acl.lanl.gov/pooma) or contact me. Jim From HYoon@exchange.ml.com Tue Jan 5 18:32:31 1999 From: HYoon@exchange.ml.com (Yoon, Hoon (CICG - NY Program Trading)) Date: Tue, 5 Jan 1999 13:32:31 -0500 Subject: [Matrix-SIG] Matrix C++ Class site Message-ID: http://webnz.com/robert/ The above address contains Matrix and Random No Gen Class Written in C++. I thought people might be interested in it. ************************************************************** S. Hoon Yoon (Quant) Merrill Lynch Equity Trading yelled@yahoo.com hoon@bigfoot.com(w) "Miracle is always only few standard deviations away, but so is catastrophe." * Expressed opinions are often my own, but NOT my employer's. "I feel like a fugitive from the law of averages." Mauldin ************************************************************** From Oliphant.Travis@mayo.edu Tue Jan 5 19:53:25 1999 From: Oliphant.Travis@mayo.edu (Travis Oliphant) Date: Tue, 5 Jan 1999 13:53:25 -0600 (CST) Subject: [Matrix-SIG] Numerical Integration using Gaussian Quadrature Message-ID: For NumPy users who want to do Numerical integration. I have a module that will do numerical integration on python functions using Gaussian quadrature. The engine behind this module is another module that computes the roots and quadrature weights for a large number of orthogonal polynomials of arbitrary order. It uses a couple of functions from the latest version of MLab.py. The modules are available at http:/oliphant.netpedia.net ---------------------------------------------------- Travis Oliphant 200 First St SW Rochester MN 55905 Ultrasound Research Lab (507) 286-5293 Mayo Graduate School Oliphant.Travis@mayo.edu From Oliphant.Travis@mayo.edu Wed Jan 6 22:55:51 1999 From: Oliphant.Travis@mayo.edu (Travis E. Oliphant) Date: Wed, 06 Jan 1999 16:55:51 -0600 Subject: [Matrix-SIG] New and improved version of cephes module. Message-ID: <3693E9F7.C4CED626@mayo.edu> Hello again, This is the last of the Christmas-vacation updates. Thanks to all of you who have provided feedback on some of the packages I've been releasing. I appreciate knowing who is using them. I have a new version of the cephes module which defines a large number of special functions (including statistical functions) as ufunc objects for use with Numerical Python. This latest release includes an html page describing all of the included functions and their arguments along with the inclusion of many more functions from the cephes library. Now all functions that take one or two arguments and return any number of arguments are included in the module. New functions include Airy, Modified Bessel of arbitrary order, Smirnov statistics, and Fresnel Integrals, plus more. You can see a complete list of all of the included functions here: http://oliphant.netpedia.net/packages/included_functions.html. Although I didn't mention it in the original post this module is necessary for the numerical quadrature routines announced earlier. A gzipped tarfile and RPM's for Linux are available at http://oliphant.netpedia.net/ Please send questions, bug-reports, or feedback. Thanks, Travis Oliphant ---------------------------------------------------- Travis Oliphant 200 First St SW Rochester MN 55905 Ultrasound Research Lab (507) 286-5923 Mayo Graduate School Oliphant.Travis@mayo.edu From Oliphant.Travis@mayo.edu Wed Jan 6 23:01:18 1999 From: Oliphant.Travis@mayo.edu (Travis E. Oliphant) Date: Wed, 06 Jan 1999 17:01:18 -0600 Subject: [Matrix-SIG] Cephes module Message-ID: <3693EB3E.C2CBBEC5@mayo.edu> If you read the previous post in a URL-enabled reader and try to click on the included functions link for the descriptions (like I just did), make sure to delete the "." at the end of the link. The page (without the terminating period) does exist. Sorry about that. Travis From dubois1@llnl.gov Thu Jan 7 00:30:23 1999 From: dubois1@llnl.gov (Paul F. Dubois) Date: Wed, 6 Jan 1999 16:30:23 -0800 Subject: [Matrix-SIG] Re: your mail Message-ID: <002b01be39d4$ea7a82e0$f4160218@c1004579-c.plstn1.sfba.home.com> In re the changes to CXX in release 8, I had noticed that in 1.5.2 there were some new members in the type tables. I didn't track how far back that change went -- 1.5, 1.5.1, ?. But if it doesn't compile you can just kill the new members as I'm not using them for anything yet as I haven't had time to figure them out. As to the comment someone posted about the release 8 Windows installer not wanting to work with 1.5.1, this is because it is supposed to set a registry key based on the Python version. Maybe I'm not doing something right. If you don't have the key that matches your Python then the python path is screwed up. I'm in the process of moving my office so please forgive the slow responses. -----Original Message----- From: David Ascher To: Christopher J. Phoenix Cc: Paul F. Dubois Date: Tuesday, January 05, 1999 1:41 PM Subject: Re: your mail > >On Tue, 5 Jan 1999, Christopher J. Phoenix wrote: > >> Using Visual C++ 5.0 to build the CXX directory of NumPy release 8 (Dec. 28): >> >> r.cxx >> Include\CXX_Objects.h(923) : error C2665: 'seqref::seqref' : none of the 2 >> overloads can convert parameter 1 from type 'class Py::SeqBase> Py::Char> *const ' >> Include\CXX_Objects.h(931) : error C2665: 'seqref::seqref' : none of the 2 >> overloads can convert parameter 1 from type 'class Py::SeqBase> Py::Char> *const ' >> >> I followed these instructions: >> c:\wherever> python makethis.py >> c:\wherever> msdev workspace.dsw >> [Go to "Build", pull down "Batch Build", do "Build All", exit DevStudio] >> >> seqref back() { >> return seqref(this, size()-1); // line 923 >> } >> >> I'm a bit confused; seqref is a template, so >> seqref (SeqBase& seq, int j) >> : s(seq), offset(j), the_item (s.getItem(j)){} >> should be able to create any Py::SeqBase, right? Or is it the >> Py:: that's confusing it? > >I have no idea -- I haven't tried to build r8 yet. > >> Also, the release notes for 8 say that the type table was changed to conform >> to Python 1.5.2. Does that mean it won't work with 1.5.1? > >I hope not. Paul, any ideas? > >--david > > From Oliphant.Travis@mayo.edu Thu Jan 7 08:21:11 1999 From: Oliphant.Travis@mayo.edu (Travis E. Oliphant) Date: Thu, 07 Jan 1999 02:21:11 -0600 Subject: [Matrix-SIG] RPMS of LLNL8, GIST, and RNG Message-ID: <36946E77.4ED17BA0@mayo.edu> To anyone who uses RPM to install Numerical Python: I have the latest LLNL8 packaged into source and binary RPMS. A single SRPM produces three binary RPMS for Linux x86: python-numpy-1.8 python-numpy-gist-1.8 (requires yorick >= 1.4) python-numpy-RNG-1.8 The SRPM is available at http://oliphant.netpedia.net/SRPMS The RPMS are availabe at http://oliphant.netpedia.net/RPMS (RPM is a great way to install packages... It makes it easy to keep python up-to-date. RPM is available for many platforms besides Linux.) From dubois1@llnl.gov Fri Jan 8 22:57:38 1999 From: dubois1@llnl.gov (Paul F. Dubois) Date: Fri, 8 Jan 1999 14:57:38 -0800 Subject: [Matrix-SIG] Warning in re 1.5.2b1 Message-ID: <004a01be3b5a$4a0cc980$f4160218@c1004579-c.plstn1.sfba.home.com> We're tracking this down but I want to warn people that we received a complaint that a real array raised to a real power gets a TypeError. We have discovered, somewhat suprisingly, that this happens only with the 1.5.2b1 release of Python (NumPy, release 8). With 1.5.1 and Release 8, it works. (Thanks to Charlie Crabb and Zane Motteler for the footwork). Color us puzzled in Livermore. I will make release 9 next week. So far I have these fixes. If I'm missing something you have reported and think is still not fixed, please let me know. There is an issue involving assignment to slices of arrays of type object that I haven't done yet. Release 1.9 (January, 1999) 1. resize now works with arrays of length zero. (Keith Junius) 2. Fixed error in integer array to integer power that caused failure if base was zero. (David LaRose, Charles Waldman) 3. Fixed registry key for Windows installer to work with 1.5.1 as well as 1.5.2. (David Ascher) 4. Fixed memory error in array_setattr (Charles Waldman) 5. Fixed error in seed in RandomArray.py (Paul Dubois, Christopher Phoenix) Paul Dubois From Oliphant.Travis@mayo.edu Fri Jan 8 12:34:25 1999 From: Oliphant.Travis@mayo.edu (Travis E. Oliphant) Date: Fri, 08 Jan 1999 06:34:25 -0600 Subject: [Matrix-SIG] Updated cephes module for non-Linux Message-ID: <3695FB51.573BCB52@mayo.edu> I've recently updated cephes module to fix some installation problems on platforms besides PC's. Thanks to Jim Boyle for pointing them out. Changes: -- eliminated log2 function. -- better INSTALL help -- changed so gamma works with Solaris math library. I've compiled and tested the library on IRIX-6.2 and SunOS-5.6 This is a cross-platform improvement upgrade only. Linux users get no benefit from the change. ---------------------------------------------------- Travis Oliphant 200 First St SW Rochester MN 55905 Ultrasound Research Lab (507) 286-5923 Mayo Graduate School Oliphant.Travis@mayo.edu From vanandel@ucar.edu Sun Jan 10 01:22:12 1999 From: vanandel@ucar.edu (Joe Van Andel) Date: Sat, 09 Jan 1999 18:22:12 -0700 Subject: [Matrix-SIG] SWIG, Python, and NumPy Message-ID: <199901100122.SAA09200@stout.atd.ucar.edu> I'm starting a project where I'm writing some analysis code using NumPy. I wanted to manipulate PyArrayObject's, without having to write all the low level Python interface code. I wanted to let you know that your system of typemapping, in SWIG, although complex to use, was powerful enough to let me do what I wanted. I simply wrote the following type maps, and SWIG generates the interface code that I didn't want to think about. Thanks again for furnishing such a powerful tool. I'd appreciate it if you put these typemaps into your next distribution. My C++ class interface definition looks like: %module pulsepair %{ #include "pulsepair.h" %} %include "numpy_typemaps.i" class PulsePair { public: PulsePair(int numGates, int pulsesPerBeam, double powerScale); PyObject *compute(int pulsesPerBeam, PyArrayObject *INPUT, PyArrayObject *INPUT); } ----------------------------------------- // NumPy typemaps %typemap(python,in) PyArrayObject *INPUT { $target = (PyArrayObject *)$source; } %typemap(python,out) PyObject * { $target = $source; } Joe VanAndel Internet: vanandel@ucar.edu National Center for http://www.atd.ucar.edu/~vanandel/home.html Atmospheric Research From emmanuel.viennet@lipn.univ-paris13.fr Sun Jan 10 13:01:30 1999 From: emmanuel.viennet@lipn.univ-paris13.fr (Emmanuel Viennet) Date: Sun, 10 Jan 1999 13:01:30 +0000 Subject: [Matrix-SIG] Gist + Tkinter ? Message-ID: <3698A4AA.AA024ADD@lipn.univ-paris13.fr> Hi, I'm trying to develop an interactive scientific application using Tk. Since Tk seems to lack sophisticated plotting widgets, I'd like to use Gist to open plotting windows. I need a way to tell gist to process all its pending X11 events (window redraw etc). The gist.redraw() function don't do the job. It may be possible to install an event handler in Tk ? Does anyone know a clean way to do this kind of things ? BTW, if anyone knows a Tk widget displaying nice contour plots... Thanks, Emmanuel -- Emmanuel Viennet: LIPN - Institut Galilee - Universite Paris-Nord 93430 Villetaneuse - France http://www-lipn.univ-paris13.fr/membres/viennet/viennet.html From cgw@pgt.com Sun Jan 10 08:05:13 1999 From: cgw@pgt.com (Charles G Waldman) Date: Sun, 10 Jan 1999 03:05:13 -0500 (EST) Subject: [Matrix-SIG] Exponentiation (was: [Matrix-SIG] Warning in re 1.5.2b1) In-Reply-To: <004a01be3b5a$4a0cc980$f4160218@c1004579-c.plstn1.sfba.home.com> References: <004a01be3b5a$4a0cc980$f4160218@c1004579-c.plstn1.sfba.home.com> Message-ID: <13976.24377.686393.47186@sirius> Paul Dubois writes: > I want to warn people that we received a complaint that a real array > raised to a real power gets a TypeError. We have discovered, somewhat > suprisingly, that this happens only with the 1.5.2b1 release of Python > (NumPy, release 8). With 1.5.1 and Release 8, it works. The culprit is some code in Objects/abstract.c which is intended to prevent us from attempting to raise a negative number to a float power. There is a check in the do_pow function which attempts to ensure that when we compute x**y and y is float, x is nonnegative. When x is an array, this check cannot be performed, and an error is raised. There are some interesting differences in behavior in Python between the 1.5, 1.5.1 and 1.5.2 version with respect to exponentiation, as these examples show: Version 1.5 refuses to compute x**y if x<0 and y is real, even if y is actually integral: Python 1.5 >>> (-2)**2.0 ValueError: negative number to float power >>> (-2)**2.1 ValueError: negative number to float power >>> (-2.0)**2.0 ValueError: negative number to float power >>> (-2.0)**2.1 ValueError: negative number to float power Version 1.5.1 lets us compute -2**2.0 but not (-2.0)**2.0: Python 1.5.1 >>> (-2)**2.0 4.0 >>> (-2.0)**2.0 ValueError: negative number to float power >>> (-2)**2.1 ValueError: negative number to a float power Version 1.5.2b2 goes back to the same behavior as 1.5: Python 1.5.2b2 >>> (-2)**2.0 ValueError: negative number to float power >>> (-2)**2.1 ValueError: negative number to float power Anyhow, this may not seem related to the problem of raising arrays to real powers, but it is. The difference between 1.5 and 1.5.1 is due to the following code change: diff -u Python-1.5/Objects/abstract.c Python-1.5.1/Objects/ --- Python-1.5/Objects/abstract.c Tue May 20 18:09:08 1997 +++ Python-1.5.1/Objects/abstract.c Fri Apr 3 18:38:59 1998 @@ -463,7 +463,8 @@ "pow() requires numeric arguments"); return NULL; } - if (PyFloat_Check(w) && PyFloat_AsDouble(v) < 0.0) { + if (PyFloat_Check(v) && PyFloat_Check(w) && + PyFloat_AsDouble(v) < 0.0) { if (!PyErr_Occurred()) PyErr_SetString(PyExc_ValueError, "negative number to float power"); Here "v" is the base and "w" is the exponent. The PyFloat_Check(v) in the 1.5.1 code short-circuits the v<0 test if v is not a float; that's why in 1.5.1 we can compute (-2)**2.0 but not (-2.0)**2.0, and also why we can raise arrays to real powers. The fact that we can do this in 1.5.1 is really a case of "slipping between the cracks" in this test. In 1.5.2 this code has been changed to: if ( #ifndef WITHOUT_COMPLEX !PyComplex_Check(v) && #endif PyFloat_Check(w) && PyFloat_AsDouble(v) < 0.0) { if (!PyErr_Occurred()) PyErr_SetString(PyExc_ValueError, "negative number to float power"); return NULL so we can no longer get away with raising arrays to real powers. An easy fix for this is simply to remove this chunk of code from abstract.c entirely. There is already code in floatobject.c:float_pow() to raise an exception in the bad cases. With the checks removed from abstract.c:do_pow (the "if" block beginning at line 589) I get what I consider to be more reasonable behavior: Python 1.5.2b2 (patched) >>> (-2)**2.0 4.0 >>> (-2.0)**2.0 4.0 >>> (-2.0)**2.1 ValueError: negative number to a float power >>> from Numeric import * >>> a = array((1,2,3)) >>> a**2.0 array([ 1., 4., 9.]) >>> a = array((1,-2,3)) >>> a**2.0 array([ 1., 4., 9.]) >>> a**2.1 ValueError: math domain error Here's a patch for abstract.c =================================================================== RCS file: /projects/cvsroot/python/dist/src/Objects/abstract.c,v retrieving revision 2.26 diff -u -u -r2.26 abstract.c --- abstract.c 1998/12/10 16:54:48 2.26 +++ abstract.c 1999/01/10 07:51:16 @@ -586,16 +586,6 @@ "pow(x, y) requires numeric arguments"); return NULL; } - if ( -#ifndef WITHOUT_COMPLEX - !PyComplex_Check(v) && -#endif - PyFloat_Check(w) && PyFloat_AsDouble(v) < 0.0) { - if (!PyErr_Occurred()) - PyErr_SetString(PyExc_ValueError, - "negative number to float power"); - return NULL; - } if (PyNumber_Coerce(&v, &w) != 0) return NULL; if ((f = v->ob_type->tp_as_number->nb_power) != NULL) From guido@CNRI.Reston.VA.US Sun Jan 10 16:55:24 1999 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Sun, 10 Jan 1999 11:55:24 -0500 Subject: [Matrix-SIG] Re: Exponentiation (was: [Matrix-SIG] Warning in re 1.5.2b1) In-Reply-To: Your message of "Sun, 10 Jan 1999 03:05:13 EST." <13976.24377.686393.47186@sirius> References: <004a01be3b5a$4a0cc980$f4160218@c1004579-c.plstn1.sfba.home.com> <13976.24377.686393.47186@sirius> Message-ID: <199901101655.LAA15578@eric.cnri.reston.va.us> Thanks, Charles. Your patch will be in 1.5.2b2. That's what beta testing is all for! --Guido van Rossum (home page: http://www.python.org/~guido/) From jhauser@ifm.uni-kiel.de Tue Jan 12 10:13:34 1999 From: jhauser@ifm.uni-kiel.de (Janko Hauser) Date: Tue, 12 Jan 1999 11:13:34 +0100 (CET) Subject: [Matrix-SIG] Gist + Tkinter ? In-Reply-To: <3698A4AA.AA024ADD@lipn.univ-paris13.fr> References: <3698A4AA.AA024ADD@lipn.univ-paris13.fr> Message-ID: <13979.7730.303732.497615@lisboa.ifm.uni-kiel.de> In the low level interface (from gist import *) there is an option to the window command window command: window( [n] [, display = "host:server.screen", dpi=100/75, wait=0/1, private=0/1, hcp="hcp_filename", dump=0/1, legends=1/0, style="style_sheet_filename" ] ) ... By default, Python will not wait for the X window to become visible; code which creates a new window, then plots a series of frames to that window should use wait=1 to assure that all frames are actually plotted. ... The plot appears first, when an additional fma() is issued. And I had no success, to have an interactive window. Hope this helps, __Janko From r.hooft@euromail.net Wed Jan 13 09:32:39 1999 From: r.hooft@euromail.net (Rob Hooft) Date: Wed, 13 Jan 1999 10:32:39 +0100 (MET) Subject: [Matrix-SIG] NumPy installation directory Message-ID: <13980.26679.746475.932531@octopus.chem.uu.nl> One Question for the next release of NumPy: can it be installed by default in the site-packages directory? That makes adding the NumPy package to the PYTHONPATH unnecessary. Rob -- ===== R.Hooft@EuroMail.net http://www.xs4all.nl/~hooft/rob/ ===== ===== R&D, Nonius BV, Delft http://www.nonius.nl/ ===== ===== PGPid 0xFA19277D ========================== Use Linux! ========= From da@skivs.ski.org Wed Jan 13 17:29:40 1999 From: da@skivs.ski.org (David Ascher) Date: Wed, 13 Jan 1999 09:29:40 -0800 (Pacific Standard Time) Subject: [Matrix-SIG] NumPy installation directory In-Reply-To: <13980.26679.746475.932531@octopus.chem.uu.nl> Message-ID: On Wed, 13 Jan 1999, Rob Hooft wrote: > One Question for the next release of NumPy: can it be installed > by default in the site-packages directory? That makes adding the > NumPy package to the PYTHONPATH unnecessary. It's on the list. I hadn't understood .pth files until recently. --david From managan@llnl.gov Wed Jan 13 19:22:33 1999 From: managan@llnl.gov (Rob Managan) Date: Wed, 13 Jan 1999 11:22:33 -0800 Subject: [Matrix-SIG] NumPy installation directory In-Reply-To: References: <13980.26679.746475.932531@octopus.chem.uu.nl> Message-ID: AS I tested the latest LLNL release on the Mac I ran up against an old bug that seems to have never been satisfactorily fixed. The problem is in RandomArray.py. When the seed is set using the time you find that on the Mac that the current time is greater than an int can hold. This is because time is measured from Jan 1, 1904. The second issue is that on the Mac the tim eis an integer and has no fraction al part so the second seed was always zero. Here is one attempt at a fix. This is the top part of RandomArray.py The goal was to split the time into two roughly equal parts. import ranlib import Numeric import sys import math from types import * ArgumentError = "ArgumentError" def seed(x=0,y=0): if type (x) != IntType or type (y) != IntType : raise ArgumentError, "seed requires integer arguments." if x == y == 0: import time t = time.time() if sys.platform == 'mac' : ndigits = int(math.log10(t)) base = 10**(ndigits/2) x = int(t/base) y = int(t%base) else: x = int(t) y = int((t-int(t))*1000000) ranlib.set_seeds(x,y) return Comments are appreciated. *-*-*-*-*-*-*-*-*-*-**-*-*-*-*-*-*-*-*-*-*- Rob Managan mailto://managan@llnl.gov LLNL ph: 925-423-0903 P.O. Box 808, L-098 FAX: 925-423-5804 Livermore, CA 94551-0808 From jh@oobleck.tn.cornell.edu Mon Jan 18 06:00:22 1999 From: jh@oobleck.tn.cornell.edu (Joe Harrington) Date: Mon, 18 Jan 1999 01:00:22 -0500 Subject: [Matrix-SIG] a full-blown interactive data analysis environment Message-ID: <199901180600.BAA03770@oobleck.tn.cornell.edu> In late November/early December I posted some messages about building a complete analysis environment around Python. I got 15 responses offering help on the project, and several emails in the mean time wondering when things will get under way. Sorry to those I've kept waiting! I posted a pointer to a paper Paul Barrett and I wrote regarding the direction such a project should take. There hasn't been any discussion of it as I had hoped, so I'll summarize it here, hope interested people will read it (it's not long), and close with a few administrative-type questions. The URL is http://oobleck.tn.cornell.edu/jh/ast/papers/idae96.ps.gz . Data analysis systems have grown nearly as quickly as the volume of data has. 15 years ago, we were mostly using home-brew compiled software. A few institutes had relatively primitive interpreter interfaces around which a lot of code was written to do specific tasks. There wasn't yet a Perl or Python to standardize around; each group wrote an interpreter from scratch, usually knowing little or nothing about how *people* interact with computers. Like the computers of the time, they felt great to use then but you wouldn't touch them now. Just a few years later, it seemed these environments were stuck. Communities made huge investments in these languages in the form of application code that they depended upon for their bread and butter. People had spent years writing code for their projects and they weren't about to do it again. In many cases the original authors were gone and those left behind didn't have the programming skills or the knowledge of the equipment, experiment, method, or whatever to do the job. Improving the languages was impossible without breaking application code and often the resources weren't there because those with funding felt that "things worked", and anyone who would work for them would have to "just deal". More than a few of those 80's-vintage interpreters are still in active use today, being carried along by gigs of really good application code and being cursed by their users each time they fire the old dogs up. Python is not the answer to this problem, nor is any current user interface. The problem is that interface technology (be it a compiler, an interpreted/autocompiled language, WYSIAYG, or whatever) changes too rapidly, and in just 5 years we will look at the current Python and be glad we're not stuck with that "old dog". Python may grow along with the state of the art and may still be a good answer in the future, or it may not. If it doesn't, we'll want something else. For that matter, many will not want Python now. In contrast, our application code is stable. There are usually not a great many "best" ways to solve a problem, and one or a few are generally favored. Once someone has written least squares, why write it again? It's only necessary to do so if a new algorithm or a new language comes along. New algorithms will always require new code. However, in the past 35 years there have been only three "favored languages", those being Fortran, C, and C++. Though there are hundreds of other languages, many technically superior to these three, these three have taken all the prizes in terms of universality. Write in one and your code will run everywhere, forever, and roughly as fast as any portable code can run. Once you can compile it for a given machine, anyone can link to it and few need care what it's written in. There is much application code, of course, that is not written in these languages. It is precisely that code that got us stuck with now-obscure environments in the first place. The solution, then, is to make a strictly-enforced division between application code and user interface, and resist the temptation to do anything that would make changing user interfaces difficult. This allows smooth migration to new interfaces without abandoning the investment in application coding. The tools to do this are very familiar to this group: SWIG, dynamic linking, and the standard compilers. When new UIs come along, people will only need to learn the conventions of that UI, not how to use the new versions of the application code, because the application code will develop on its own (much slower) schedule. In 10 years, when Zappitall becomes the new hot language of choice, all we'll need to do is extend SWIG to make Zappitall wrappers in addition to the languages it currently supports, rebuild the system, and poof, instead of seeing 500 announcements saying "I wrapped my favorite library for Zappitall," we'll see one announcement saying, "all the numerical and graphical packages we've ever wrapped can now be used directly from Zappitall". The picture I'm drawing is nothing terribly new: Application code is in standard, compiled, highly optimizable languages like Fortran, C, and C++. Initially we might use some existing Python numerical code, but we'll always select Fortran, C, and C++ code if it's available, and we'll expend some effort to have a compiled-code option for each major application component. A glue layer describes the application code to the user interface, preferably in a simple and general way that will be easily adapted to future user interfaces. The user interface(s) has all the latest features everyone wants, and when this is no longer true, newer interfaces can be dropped in. It would even be desirable to have several interfaces to begin with. This would ensure that there weren't any hidden dependencies on the UI, and would also appeal to a wider audience since people have their own tastes. Often people want their interface language to resemble their favorite coding language, so they can rapidly prototype a piece of code they will eventually write in a compiled language. That describes the technical end of it, but there's a lot more to it, especially in the department of user appeal. This effort will be large, and it will only be worth it if it gains wide appeal and is in use for a long time. People change software packages if they find something that is much better technically, is easier to use, or is cheaper. It is important that we always keep in mind the question: "What will make the most people happy?" and that we remember we're talking about the population at large, not just people who think like we do. This is a business mindset: make the customer happy and more will come to you. Thus, we'll need to target our packaging from the beginning to the widest range of users. This impacts our supported platforms, the documentation, the release packaging, and the way the package is constructed in general. I feel strongly that it should run on as many platforms as possible. That means PC and Mac, not just Unix. This makes it accessible to vastly more people, particularly those without the boon of government or corporate funding, like high school science and math classes, amateur scientists, hobbyists, etc. I recently visited a class that has some real data acquisition capability. When they measure g in the lab by dropping a ball, they get 9.8 m/s^2, not 9.6 or 9.57!! They have an ultrasonic transducer to measure the motion, and the data it takes goes right into a software package that plots it out. Hundreds of points per second! They paid a lot (for a school) to get this setup, and the software was a good fraction of the price. There's a big "market" of users there if we can produce something as easy to use that is free. Doing so would be a true contribution to society, and it would take much less effort on the application end than making something that would satisfy a professional scientist. This is just one of many possibilities once we move beyond Unix. (For those who care, I'm a die-hard Unix user myself. This laptop I'm writing on doesn't even have a Windows or DOS partition, and I don't know how to compile a program on a Mac or PC. Yet I think it's critical that we support them.) Documentation is a similar item. Packages like IDL win big points with simple, worked examples and pamphlet-sized "getting started" manuals that in a page or two show a user a plot of her own data. The main reference manuals do not showcase the programmers' coding prowess or attitudes, but rather show usually one simple example and one complex version of the same example. They use active and direct language and there are few if any grammar errors. In addition to teaching quickly, such manuals give confidence, somehow, in the people behind the software, as well as in the software itself. Release packaging will necessarily be in multiple forms. We'll have to support the Unix least-common-denominator, tar, as well as vendor-specific formats from Red Hat, Debian, Sun, and so on, and of course formats that work on Macs and PCs. Automating the creation of these will be important. I suspect it isn't hard to do. General construction is where we will have our hardest discussions. The most commonly heard complaint about software is how hard it is to learn to use it. So, simple designs with powerful features will win over complicated designs that depend on the user understanding it all before she can do anything. Certain things have come to work well. Most of us are happiest when we see command history based on GNU Readline, and we get ticked if there isn't command history at all. This is true even if there's a better package out there. Remember Beta video tapes? It was better than VHS, but if you were looking to rent movies, VHS was the way to go. Many decisions will have to be based on what's popular that works, rather than what's pushing the technical cutting edge. This doesn't mean we can't support many options, but my feeling is that the defaults should be things that are familiar to as many people as possible. For example, GNU info is more powerful than HTML, but people know and love their web browsers and the browsers are quite adequate for the job of viewing documentation. So while we can make info docs too, HTML docs should be the showcase item. Finally, "recycling". This is hardly going to be a controversial item, but it bears mentioning. We should spend much more time borrowing than writing. Wrapping will be our friend. It's what makes the difference between a huge commercial effort, where every line is written in-house so that it can be copyrighted and licensed, and a 15-person effort run in our spare time. We should always hunt for free or freeable code before we volunteer to write something new. Then we spend our precious coding resources producing things that don't already exist. That's what free software is all about. The only really daunting thing about the project to me is the likelihood that I will not be able to devote even as much time to it as many who volunteered to help. Ideally, one or more funded data projects would take an interest and contribute some people's time. I will propose that we focus initially on a very small version 1. The main emphases should be catching the docs and distribution infrastructure up to the code, getting a development site set up, and so on. Once we have a small thing of high quality "out there", it will generate interest and further contributions. Ok, administrative stuff. I've sketched out a bit of a vision, but it's far from a plan. I'd like to hear thoughts from the list to refine things, point out pitfalls, and so on. This should be a fairly abstract discussion, and not long. Please don't talk about naming or even very much about specific packages that version 1 should or shouldn't have; please focus on the approach. Then we should decide whether matrix-sig is the right place for the development discussions, and move if that's warranted. We'll take some votes on basic design issues and how to run ourselves, and then we'll divvy up work and be off. There will be many times during the project when we will want input from a larger group than just those actively working on the project. And of course we'll need testers and proofreaders from time to time. In those cases we'll post to this list. There are some embarrassingly grand statements above. I don't mean to sound like I want to save the world, yet after a decade and a half of being frustrated with data analysis, the answers seem so simple and so hard to find in the real world. If we just select the best application packages that are already in use, wrap them (initially) for Python, document them in a clear fashion that connects with a variety of thinking styles, and assemble easy-to-install distributions, we will have laid the foundation of a system that may well survive us. That's quite a grand statement in the computer world! --jh-- From Oliphant.Travis@mayo.edu Sat Jan 16 00:47:03 1999 From: Oliphant.Travis@mayo.edu (Travis E. Oliphant) Date: Fri, 15 Jan 1999 18:47:03 -0600 Subject: [Matrix-SIG] Interactive Data Analysis Message-ID: <369FE186.F7D62DF3@mayo.edu> Thanks to Joe Harrington for his desire and vision to get this project off the ground. I'm writing to voice my opinion and make a couple of comments. I read with quite a bit of interest the paper Joe put on his web page and the post to this list. The paper he references is a good discussion of some important points and I would recommend it for reading. I think the current state of numerical analysis in python is very close to much of the important points in that paper. Overall, what is most lacking is packaging of all (or a selection of) the great wrappers out there into a single available and constistent (and tested) set and improvements on the documentation (including a help system that gets used). While I agree with the sentiment on supporting Windows (I think Joe meant Windows, not just PC's, right :-) ) and Mac's, this is just where the problem lies right now, doesn't it. The single most needed part of the system that isn't just a document-and-package step away is a good, free cross-platform plotting library. However, there are already some highly usable plotting packages available for X windows systems. I find quite usable both GIST (how is the port to windows coming) and DISLIN (free for Linux and FreeBSD). Since UNIX/X or just X is available for all of this hardware a part of me (the anti-social, revenge-seeking part admittedly) wants to say "Come back and do numerical analysis with a real and open OS. You've already got all kinds of look-pretty, draw-me-a-graphic, dont-tell-me-how-it-works software for windows anyway." Maybe we can get something together with just an eye towards the PC/MAC world and even then not be too concerned with making everything. I'm a little concerned that the quality of our offering will be affected by an attempt to outfit the Windows-centric world. O.K. that's enough rant about that. I'm not trying to offend people. I just think that people interested in a free non-proprietary numerical analysis package should take the time to put it on an operating system that doesn't also tie them down.... Now, back to the real world. Perhaps the best solution is to develop some kind standard plotting API that allows construction of 2 and 3-D plots and then wrap up your favorite plotting library behind an implementation of the API. That way people could use (most of) the same commands and syntax but have possibly different output libraries. This API would need to be relatively complete so not every plotting library would do. Actually, this leads to another idea about plotting that I've been discussing with a co-worker later and that is separating the concepts of "interactive plotting" and "publication plotting." In other words with a plotting API it could be feasible to use different packages to interact with the data and to print the data, with the commands being very much the same. The second important issue is the interactive environment. Since I've added a PYTHONSTARTUP file to my environment I've been quite happy with the interactive environment I have. When I startup I have all kinds of functions and variables defined so it is very easy to "interact" the data. (For example I don't call LinearAlgebra.generalized_inverse every time I want to compute a pseudoinverse. I use pinv() ). One question is should we write a new interpreter loop to include some nifty interactive editing features? My personal favorite is using up arrow to bring back previous lines that look like what is typed so far. (just a "very-handy" keystroke variation on the functionality of GNU readline.) At the very least we need to include somebody's fantastic NumPy PYTHONSTARTUP file in the distribution. My final comments have to do with the issue of wrapping libraries. I agree that we should minimize our efforts in writing new code. I also agree that our efforts in organizing this code and any routines we add to it should not be too Python-centric so that they can be used in other interactive environments. It seems to me, however, that what we have left to do at this point is pretty much all Python-centric, i.e. it's the wrappings, packaging, and documentation that needs to be done. I guess the real question is: Can the wrappings be done in a non-python-centric way? Is SWIG really up to the task? I'm no SWIG expert and I think it is a wonderful tool, but right now I don't think it can fully "do the job" in a non Numerical-Python-oriented way. What we need is an implementation/extension of SWIG (or maybe, hopefully, just a good set of examples of how to use SWIG effectively) that is more "array oriented" or "loop oriented." I'll share my experience at wrapping up the FFTW module and the cephes library to explain what I mean. The problem is in handling arbitrary Numerical Python Arrays as inputs and outputs to the wrapped C code. The FFTW library wrapping was easy in SWIG. I made a Numerical-python centric typemap to an otherwise cross-language SWIG input file that was basically a copy of the header file from FFTW. After running swig, I had a great little interface to the FFTW calls that could take as inputs (complex) Numerical python arrays, and return the output as a NumPy array. Code that takes as arguments pointers to data blocks of memory can usually be handled in this way with SWIG. (We definitely need to put out some good documentation on how to do this though...) On the other hand wrapping the cephes library in SWIG did not appear possible (again I don't know everythin about SWIG) for what I wanted to be able to do. What was needed was something like the ufuncs that operate so well an arbitrary sized arrays. This did not seem possible to do in SWIG. So, all of my wrapping was pretty much python-centric. In the process I found myself wanting a way to take a routine in C that has N inputs and M outputs and write a little interface description text and say "wrap" and produce a wrapper that would allow the procedure to be called from python with N input numpy arrays of arbitrary (but consistent) dimensions and return M output numpy arrays. Is this cephes-specific or would the idea extend to other wrappings? Maybe there is a way to add to the SWIGlike tools we have to make them more aware of using libraries in an interactive data analysis library? Maybe that's too hard and not worth the time? But, again for now it seems that most of our work will be pretty much in the python area. I guess if we do have to write up some important numerical routine we should do it in C. It's been a long post but this is an important area for me. I say let's get out there and post suggestions for important libraries to include in the distribtution, resolve the plotting issues, divvy up and get going on the documentation, and smooth out the interactive environment edges. In the process maybe we can come up with a really snazzy way to do the wrappings that will let use the most drop-in code in a way that is still consistent with "array-oriented" environment. I think we are very close to something that we can then tell all our colleagues about and have them jump aboard. For the record the libraries I see as vitally important are: ODE solver Optimization routines Zero finding routines Numerical integration other useful additions general n-d convolution extensive linear filter routines filter design routines image processing routines (we need to get together with the PIL here. I think the PIL should be as NumPy centered as possible. I don't think it is wise to have a bunch of image processing routines that only work on PIL image objects. After all, an image is just a special way of interpreting a matrix (or set of matrices). We do not want to duplicate work and implement image filters for NumPy arrays and PIL objects. I mainly use PIL to get image formats into and out of NumPy arrays at the moment.) Travis From dubois1@llnl.gov Mon Jan 18 18:57:23 1999 From: dubois1@llnl.gov (Paul F. Dubois) Date: Mon, 18 Jan 1999 10:57:23 -0800 Subject: [Matrix-SIG] a full-blown interactive data analysis environment Message-ID: <001901be4314$64606b40$f4160218@c1004579-c.plstn1.sfba.home.com> My system for wrapping Fortran 77, called "Basis" (http://xfiles.llnl.gov/Basis) has been in existance almost 15 years now. Perhaps 200 applications have been written using it by tens of different authors. Basis contains a system that is very much like SWIG, and a user-interface language that is very much like an interpreted Fortran 90. Here are some observations based on that "database" of experience: a. The fact that the interpreted scripting language has to be used by both authors and end users makes it crucial that said language is simple to learn and not "computer-sciency". However, these users are smart and some of them will push the language to its limits, so it should be powerful. b. Contrary to the image presented in Mr. Harrington's discussion, invariably *three* layers of code develop, not two. There is the compiled code (in Fortran, C, and C++ as he said), then there is interpreted code written by the authors, and finally the user's own input. Frequently, of course, the users of a given application chip in with utility routines in interpreted language that are put in a library. The point I am making here is that the authors of the compiled code don't like writing compiled code as much as writing interpreted code and so after a while they only do that for the time-critical parts of the code, and do everything else they can in interpreted code. The more capable the interpreted language (such as having array operations) the less compiled code gets written. c. We have seen great utility in the compiled code interacting with the interpreted code. To give one example, a tokamak modeling code may check to see if the user has defined a certain interpreted array and if so use it as part of a model component, otherwise sticking to some built-in analytic model. d. Whatever the merits or not of any given GUI interface, it is important that the code be runnable as a batch process. Therefore, having a scripting language base is important no matter what the superstructure may look like. It was considerations like this that has led to interest in Python at LLNL, and why we want to strengthen the NumPy facility. From dubois1@llnl.gov Tue Jan 19 02:54:47 1999 From: dubois1@llnl.gov (Paul F. Dubois) Date: Mon, 18 Jan 1999 18:54:47 -0800 Subject: [Matrix-SIG] LLNL Release #9 now available Message-ID: <000f01be4357$15942a40$f4160218@c1004579-c.plstn1.sfba.home.com> Release 1.9 (January 19, 1999) 1. resize now works with arrays of length zero. (Keith Junius) 2. Fixed error in integer array to integer power that caused failure if base was zero. (David LaRose, Charles Waldman) 3. Fixed registry key for Windows installer to work with 1.5.1 as well as 1.5.2. (David Ascher) 4. Fixed memory error in array_setattr (Charles Waldman) 5. RandomArray.py fixed seed(), improved test, added random_integer, doc strings (Paul Dubois, Rob Managan) I hope to address complaints about the packaging / installation / documentation in the next release. From zcm@llnl.gov Tue Jan 19 20:23:23 1999 From: zcm@llnl.gov (Zane Motteler) Date: Tue, 19 Jan 99 12:23:23 PST Subject: [Matrix-SIG] RTLD_GLOBAL gone from 1.5.2; alternative? Message-ID: Hello colleagues, In version 1.5.2, the flag RTLD_GLOBAL has been removed from importdl.c. This flag, if on, allows routines in a shared object to access global variables in another. If it is off, attempting to call a routine in another shared object causes a segmentation fault and a core dump. This is unacceptable behavior to me, as I have extensions which rely on being able to call PyArray_FromDimsAndData. I do not want to have to hand edit importdl.c with each release, besides which this may cause somebody else's code to come to grief. Likewise, I do not want to have to link these modules into python itself. Has anybody out there solved the problem of how to call a routine in a shared module from another shared module without editing core python or changing how it is linked? Thanks, Zane -------------------- Zane C. Motteler, Ph. D. Professor Emeritus of Computer Science and Engineering California Polytechnic State University, San Luis Obispo zmottel@calpoly.edu Currently: Computer Scientist Lawrence Livermore National Laboratory P O Box 808, L-038 (Street address 7000 East Avenue, L-038) Livermore, CA 94551-9900 zcm@llnl.gov 925/423-2143, FAX 925/423-9969 From da@skivs.ski.org Tue Jan 19 20:31:34 1999 From: da@skivs.ski.org (David Ascher) Date: Tue, 19 Jan 1999 12:31:34 -0800 (Pacific Standard Time) Subject: [Matrix-SIG] RTLD_GLOBAL gone from 1.5.2; alternative? In-Reply-To: Message-ID: > In version 1.5.2, the flag RTLD_GLOBAL has been removed from > importdl.c. I'd amend that -- it's been removed in 1.5.2b1. I got the impression that it caused enough problems that it might be back in 1.5.2b2. See GvR's post at: http://www.dejanews.com/getdoc.xp?AN=432423192 --david From dubois1@llnl.gov Tue Jan 19 21:17:04 1999 From: dubois1@llnl.gov (Paul F. Dubois) Date: Tue, 19 Jan 1999 13:17:04 -0800 Subject: [Matrix-SIG] Re: LLNL Release #9 now available Message-ID: <000901be43f1$10884d20$f4160218@c1004579-c.plstn1.sfba.home.com> At about 1:15 p.m. PST I replaced the distribution 9 files to fix a stray right parenthesis that somehow got into the source of arraymodule.c. If you fetched the sources earlier you will find this parenthesis when you compile; the fix is to delete it. I must have twitched a finger during a last-minute browse for news items; my apologies for any inconvenience. Thanks to Zane Motteler for calling this to my attention. Paul F. Dubois wrote in message <6uSo2.347$qW5.25@news.rdc1.sfba.home.com>... >Release 1.9 (January 19, 1999) > 1. resize now works with arrays of length zero. (Keith Junius) > 2. Fixed error in integer array to integer power that caused failure if >base was zero. (David LaRose, Charles Waldman) > 3. Fixed registry key for Windows installer to work with 1.5.1 as well >as 1.5.2. (David Ascher) > 4. Fixed memory error in array_setattr (Charles Waldman) > 5. RandomArray.py fixed seed(), improved test, added random_integer, doc >strings (Paul Dubois, Rob Managan) > >I hope to address complaints about the packaging / installation / >documentation in the next release. > > > From jim@digicool.com Tue Jan 19 22:05:53 1999 From: jim@digicool.com (Jim Fulton) Date: Tue, 19 Jan 1999 22:05:53 +0000 Subject: [Matrix-SIG] Re: [PSA MEMBERS] RTLD_GLOBAL gone from 1.5.2; alternative? References: Message-ID: <36A501C1.8EC21B6A@digicool.com> Zane Motteler wrote: > > Hello colleagues, > > In version 1.5.2, the flag RTLD_GLOBAL has been removed from > importdl.c. This flag, if on, allows routines in a shared object > to access global variables in another. If it is off, attempting > to call a routine in another shared object causes a segmentation > fault and a core dump. This is unacceptable behavior to me, as > I have extensions which rely on being able to call > PyArray_FromDimsAndData. I do not want to have to hand edit > importdl.c with each release, besides which this may cause > somebody else's code to come to grief. Likewise, I do not > want to have to link these modules into python itself. Do you claim that this feature (RTLD+GLOBAL) is portable? I'd be surprized if it was. > Has anybody out there solved the problem of how to call a routine > in a shared module from another shared module without editing > core python or changing how it is linked? Yes and no. IMO, any module that want's to export a C interface should do so explicitly using a Python CObject. This relies only on Python's import machinery and should work on any system on which Python can do dynamic linking. To see how this works, look at how cStringIO exports a C API and how cPickle imports it. Maybe someone will volunteer to provide an exported C interface for the array module. Jim -- Jim Fulton mailto:jim@digicool.com Technical Director (540) 371-6909 Python Powered! Digital Creations http://www.digicool.com http://www.python.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From da@skivs.ski.org Tue Jan 19 22:42:31 1999 From: da@skivs.ski.org (David Ascher) Date: Tue, 19 Jan 1999 14:42:31 -0800 (Pacific Standard Time) Subject: [Matrix-SIG] Re: [PSA MEMBERS] RTLD_GLOBAL gone from 1.5.2; alternative? In-Reply-To: <36A501C1.8EC21B6A@digicool.com> Message-ID: > Yes and no. IMO, any module that want's to export a C > interface should do so explicitly using a Python CObject. > This relies only on Python's import machinery and should > work on any system on which Python can do dynamic linking. > > To see how this works, look at how cStringIO exports a > C API and how cPickle imports it. > > Maybe someone will volunteer to provide an exported > C interface for the array module. I thought that was what the import_array function defined in arrayobject.h did. Zane, did you use import_array? --david From zcm@llnl.gov Tue Jan 19 23:17:58 1999 From: zcm@llnl.gov (Zane Motteler) Date: Tue, 19 Jan 99 15:17:58 PST Subject: [Matrix-SIG] Re: [PSA MEMBERS] RTLD_GLOBAL gone from 1.5.2; alternative? In-Reply-To: References: <36A501C1.8EC21B6A@digicool.com> Message-ID: David Ascher wrote: >I thought that was what the import_array function defined in >arrayobject.h did. > >Zane, did you use import_array? No, I didn't, and --eureka!!-- your succinct answer was exactly what I was looking for. Thanks very much. (Actually, "eureka" means *I* have found it, but I didn't, *you* did, but unfortunately I don't know the Greek second person ending.) Cheers, Zane -------------------- Zane C. Motteler, Ph. D. Professor Emeritus of Computer Science and Engineering California Polytechnic State University, San Luis Obispo zmottel@calpoly.edu Currently: Computer Scientist Lawrence Livermore National Laboratory P O Box 808, L-038 (Street address 7000 East Avenue, L-038) Livermore, CA 94551-9900 zcm@llnl.gov 925/423-2143, FAX 925/423-9969 From hinsen@cnrs-orleans.fr Wed Jan 20 16:55:54 1999 From: hinsen@cnrs-orleans.fr (Konrad Hinsen) Date: Wed, 20 Jan 1999 17:55:54 +0100 Subject: [Matrix-SIG] Re: [PSA MEMBERS] RTLD_GLOBAL gone from 1.5.2; alternative? In-Reply-To: (message from David Ascher on Tue, 19 Jan 1999 14:42:31 -0800 (Pacific Standard Time)) References: Message-ID: <199901201655.RAA19264@dirac.cnrs-orleans.fr> > > Maybe someone will volunteer to provide an exported > > C interface for the array module. > > I thought that was what the import_array function defined in > arrayobject.h did. Exactly, and once you call it, there should be no more references to the array module in your shared library, and hence no problem. Moreover, this is not just for Python 1.5.2b1 compatibility, but for portability in general, as Jim pointed out. On some platforms (e.g. AIX), shared libraries can never access symbols in other shared libraries unless these have been imported explicitly. Fortunately Python provides a portable alternative via CObjects. Tcl doesn't, and that makes it a nightmare to install Tcl extensions under AIX, especially if they also use other shared libraries. Anyone who wants a taste of such problems should try to get Togl to work under AIX (and send me the working binary, of course ;-). -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From hinsen@cnrs-orleans.fr Wed Jan 20 17:12:11 1999 From: hinsen@cnrs-orleans.fr (Konrad Hinsen) Date: Wed, 20 Jan 1999 18:12:11 +0100 Subject: [Matrix-SIG] Re: [PSA MEMBERS] RTLD_GLOBAL gone from 1.5.2; alternative? In-Reply-To: (message from Zane Motteler on Tue, 19 Jan 99 15:17:58 PST) References: <36A501C1.8EC21B6A@digicool.com> Message-ID: <199901201712.SAA23516@dirac.cnrs-orleans.fr> > what I was looking for. Thanks very much. (Actually, "eureka" > means *I* have found it, but I didn't, *you* did, but unfortunately > I don't know the Greek second person ending.) It should be "heurekas", if my memory is still worth something ;-) -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From managan@llnl.gov Wed Jan 20 18:43:25 1999 From: managan@llnl.gov (Rob Managan) Date: Wed, 20 Jan 1999 10:43:25 -0800 Subject: [Matrix-SIG] LLNL Release #9 now available Message-ID: I have compiled LLNL Release #9 of the Numeric module for the Macintosh. For this week it is available at This archive expands into a folder that I put into the Extensions folder. Teh readme.mac file tells you what paths to add with EditPythonPrefs. After the 26th the server will change. I will send another message at that time once I confirm that all is working. *-*-*-*-*-*-*-*-*-*-**-*-*-*-*-*-*-*-*-*-*- Rob Managan mailto://managan@llnl.gov LLNL ph: 925-423-0903 P.O. Box 808, L-098 FAX: 925-423-5804 Livermore, CA 94551-0808 From managan@llnl.gov Wed Jan 20 18:43:25 1999 From: managan@llnl.gov (Rob Managan) Date: Wed, 20 Jan 1999 10:43:25 -0800 Subject: [Matrix-SIG] LLNL Release #9 now available Message-ID: I have compiled LLNL Release #9 of the Numeric module for the Macintosh. For this week it is available at This archive expands into a folder that I put into the Extensions folder. Teh readme.mac file tells you what paths to add with EditPythonPrefs. After the 26th the server will change. I will send another message at that time once I confirm that all is working. *-*-*-*-*-*-*-*-*-*-**-*-*-*-*-*-*-*-*-*-*- Rob Managan mailto://managan@llnl.gov LLNL ph: 925-423-0903 P.O. Box 808, L-098 FAX: 925-423-5804 Livermore, CA 94551-0808 From Oliphant.Travis@mayo.edu Wed Jan 20 19:20:42 1999 From: Oliphant.Travis@mayo.edu (Travis Oliphant) Date: Wed, 20 Jan 1999 13:20:42 -0600 (EST) Subject: [Matrix-SIG] RPMS of NumPy 1.9 Message-ID: For users of Numerical Python on Linux: RPMS of NumPy 1.9 Numerical Graphics and RNG packages are available at http:/oliphant.netpedia.net/RPMS ---Travis--- From managan@llnl.gov Wed Jan 20 19:19:25 1999 From: managan@llnl.gov (Rob Managan) Date: Wed, 20 Jan 1999 11:19:25 -0800 Subject: [Matrix-SIG] LLNL Release #9 now available Message-ID: I have compiled LLNL Release #9 of the Numeric module for the Macintosh. For this week it is available at This archive expands into a folder that I put into the Extensions folder. Teh readme.mac file tells you what paths to add with EditPythonPrefs. After the 26th the server will change. I will send another message at that time once I confirm that all is working. *-*-*-*-*-*-*-*-*-*-**-*-*-*-*-*-*-*-*-*-*- Rob Managan mailto://managan@llnl.gov LLNL ph: 925-423-0903 P.O. Box 808, L-098 FAX: 925-423-5804 Livermore, CA 94551-0808 From perry@stsci.edu Thu Jan 21 16:46:17 1999 From: perry@stsci.edu (Perry Greenfield) Date: Thu, 21 Jan 1999 11:46:17 -0500 (EST) Subject: [Matrix-SIG] Re: Matrixre: a full-blown interactive data analysis environment Message-ID: <199901211646.LAA26979@eclipse> I would like to second Paul Dubois' comments about the original message from Joe Harrington. I think it is probably naive to expect that developers will confine themselves to producing only tasks in C/C++/Fortran when much of what they can do can be done in the scripting language. I think the success of IDL in the astronomy community shows that if scripting language has the features that IDL has, a lot of applications will be written using it and use of C/C++/Fortran will be avoided as much as possible. In other words, I think the idea that splitting development into these two categories make it easy to switch to another scripting language later is unlikely to be right. Indeed in our case, we find Python very attactive for developing software for those cases where the circle of users is fairly small but the need for the functionality is great (for example, analysis of calibration data). In these circumstances, developing such software in C/C++/Fortran is very costly and not worth the trouble. In fact, much of scientific software falls into this category. Very often, the ultimate in performance or in code quality can be sacrificed. Better code that doesn't run quite so fast or may evidence some occasional run time errors that need correction than no program at all. Perry Greenfield Space Telescope Science Institute From ffjhl@uaf.edu Fri Jan 22 00:19:35 1999 From: ffjhl@uaf.edu (Jonah Lee) Date: Thu, 21 Jan 1999 15:19:35 -0900 (AKST) Subject: [Matrix-SIG] Re: Matrixre: a full-blown interactive data analysis environment In-Reply-To: <199901211646.LAA26979@eclipse> Message-ID: On Thu, 21 Jan 1999, Perry Greenfield wrote: I'd like to second Perry Greenfield's comments too. I find that development using Python (with or without Numerical Python) is much faster than using C/C++/Fortran. Efficiency can be improved by using Numerical Python or 'translating' some critical portion of the code to C++ and then wrap the C++ code using SWIG. Translating Python code into C++ is in general not a difficult task. My own benchmark shows that wrapped C++ code using SWIG runs almost as fast as native C++ code. In addition, 'translating' pure Python code into Numerical Python code is very easy too. Python is also wonderful in the iterative process of object-oriented design which may be critical for the maintainence of the code base. I'm in favor of having a reasonably good Python interface, i.e., a Python-centric appoach. Just my two cents:-) > I would like to second Paul Dubois' comments about the original message > from Joe Harrington. I think it is probably naive to expect that developers > will confine themselves to producing only tasks in C/C++/Fortran when much > of what they can do can be done in the scripting language. I think the > success of IDL in the astronomy community shows that if scripting language > has the features that IDL has, a lot of applications will be written using > it and use of C/C++/Fortran will be avoided as much as possible. In other > words, I think the idea that splitting development into these two categories > make it easy to switch to another scripting language later is unlikely > to be right. > > Indeed in our case, we find Python very attactive for developing software > for those cases where the circle of users is fairly small but the need for > the functionality is great (for example, analysis of calibration data). > In these circumstances, developing such software in C/C++/Fortran is > very costly and not worth the trouble. In fact, much of scientific software > falls into this category. Very often, the ultimate in performance or > in code quality can be sacrificed. Better code that doesn't run quite so > fast or may evidence some occasional run time errors that need correction > than no program at all. > > Perry Greenfield > Space Telescope Science Institute > > _______________________________________________ > Matrix-SIG maillist - Matrix-SIG@python.org > http://www.python.org/mailman/listinfo/matrix-sig > Regards, Jonah --------------------------------------------------------------------- | I do not use Microsoft Word. | Linux Rules! www.linux.org | |Please save *.doc file as *.txt | LyX Rules! www.lyx.org | |before sending it as an attachment.| Python Rules! www.python.org | | Many Thanks For Doing This. | | --------------------------------------------------------------------- From vanandel@ucar.edu Mon Jan 25 16:38:02 1999 From: vanandel@ucar.edu (Joe Van Andel) Date: Mon, 25 Jan 1999 09:38:02 -0700 Subject: [Matrix-SIG] coding examples/advice for building Numeric Python extensions? Message-ID: <199901251638.JAA17643@stout.atd.ucar.edu> This is a multipart MIME message. --==_Exmh_9916988280 Content-Type: text/plain; charset=us-ascii I'm starting a new project to build a Doppler radar timeseries processing environment using NumPy. Perhaps I've missed something, but I haven't found any documentation that explains how to write my own extension. I've written the argument handling code by adding some new SWIG templates (attached) Now that I'm writing the body of my functions, I've got some questions: 1) When (if ever) I should be concerned about "non-contiguous" arrays. 2) If I want to avoid the overhead of copying megabytes of data to insure that it is contiguous, how do I access the data in the PyArrayObject? 3) Has someone who is familiar with NumPy written a brief extension writer's guide? 4) At the very least, could someone write somes brief descriptions of the functions that an extension writer should be using? 5) Can someone recommend a short NumPy extension that would serve as an example? (IMHO, I think more people might starting using NumPy if there was documentation of how to write extensions. ) Thanks much for your help! Joe VanAndel Internet: vanandel@ucar.edu National Center for http://www.atd.ucar.edu/~vanandel/home.html Atmospheric Research --==_Exmh_9916988280 Content-Type: text/plain ; name="numpy_typemaps.i"; charset=us-ascii Content-Description: numpy_typemaps.i %typemap(python,in) PyArrayObject *INPUT { $target = (PyArrayObject *)$source; } %typemap(python,in) PyArrayObject *INPUT_1D { $target = (PyArrayObject *)$source; if ($target->nd != 1) { PyErr_SetString(PyExc_TypeError, "must be 1D array"); return NULL; } } %typemap(python,in) PyArrayObject *INPUT_2D { $target = (PyArrayObject *)$source; if ($target->nd != 2) { // printf("%s: num dim = %d\n", "$target", $target->nd); PyErr_SetString(PyExc_TypeError, "must be 2D array"); return NULL; } } %typemap(python,out) PyObject * { $target = $source; } --==_Exmh_9916988280-- From hinsen@cnrs-orleans.fr Mon Jan 25 18:35:02 1999 From: hinsen@cnrs-orleans.fr (Konrad Hinsen) Date: Mon, 25 Jan 1999 19:35:02 +0100 Subject: [Matrix-SIG] coding examples/advice for building Numeric Python extensions? In-Reply-To: <199901251638.JAA17643@stout.atd.ucar.edu> (message from Joe Van Andel on Mon, 25 Jan 1999 09:38:02 -0700) References: <199901251638.JAA17643@stout.atd.ucar.edu> Message-ID: <199901251835.TAA18160@dirac.cnrs-orleans.fr> > 1) When (if ever) I should be concerned about "non-contiguous" arrays. Whenever your input arrays might be non-contiguous, i.e. the result of slicing operations. In many situations you can be sure to have contiguous arrays, e.g. if the array has just been created from scratch (in the C extension or in the Python code that calls it). > 2) If I want to avoid the overhead of copying megabytes of data to insure > that it is contiguous, how do I access the data in the PyArrayObject? All you have to do is take into account the stride information. It is not at all difficult to access a non-contiguous array if you are writing NumPy-specific code; the only real problem is that the array data does not have the layout of standard C-style arrays, so most C library routines that are not NumPy-aware won't work on it. In that case you must either copy or rewrite the code. > 3) Has someone who is familiar with NumPy written a brief extension writer's guide? Not me :-( > 4) At the very least, could someone write somes brief descriptions of the > functions that an extension writer should be using? There are only very few functions that are needed for most applications: PyArray_FromDims(ndim, dim, datatype) creates a new array of "ndim" dimensions and type "datatype", with the length of each dimension (i.e. the shape) defined by the int array "dim". PyArray_FromDimsAndData(ndim, dim, datatype, data) same as above, but uses the memory pointed to by "data" as the data space of the array, no new memory is allocated other than the small amount needed for the array header. Useful to pass arrays created in C/Fortran code to Python, but also dangerous; the user must ensure that the data space is never freed before the end of the program. PyArray_ContiguousFromObject(object, datatype, min_dimensions, max_dimensions) used in C functions that want to accept any Python sequence object as input (like most NumPy functions do). "object" is the input object, "datatype" the array data type. In general, "min_dimensions" and "max_dimensions" are set to zero. The resulting is guaranteed to be a contiguous array. If the input is a contiguous array, the output is just the input, nothing is copied. In all other circumstances, a new array object is allocated and the data is copied (and perhaps converted). PyArray_Return(arrayobject) typically used for returning data at the end of a C function. Returns the array object if its number of dimensions is larger than zero, or an equivalent scalar object (int/float...) instead of a zero-dimensional array. In my experience, these are the only functions that are needed for most C extension modules. > 5) Can someone recommend a short NumPy extension that would serve as an > example? The shortest one that I am aware of is the random number module from the LLNL distribution, but it doesn't use all the functions listed above. Other examples are the LAPACK and FFTPACK interfaces that come with NumPy itself, and my netCDF module. Then there are the FFTW and Cephes modules by Travis Oliphant, which must also use NumPy calls, although I haven't looked at the source. Plus probably many more... -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From vanandel@ucar.edu Thu Jan 28 17:10:28 1999 From: vanandel@ucar.edu (Joe Van Andel) Date: Thu, 28 Jan 1999 10:10:28 -0700 Subject: [Matrix-SIG] contributed archive for NumPy extensions? Message-ID: <199901281710.KAA22673@stout.atd.ucar.edu> Can we establish an easily located central archive or index for NumPy extensions? (Or if it exists, can we publicize it better?) I would like a very simple mechanism for an extension author to 'publish' information about an extension such that a potential user can easily find the extensions. For example, I found some nice NumPy modules by 'Janko Hauser', but only by a careful search of the 'starship' crew pages. http://starship.skyport.net/crew/jhauser/numpyadds.html I'd like to a link from the http://www.python.org/download/Contributed.html#Math http://www.python.org/sigs/matrix-sig/ and http://www.python.org/topics/scicomp/numbercrunching.html pages to an FTP site, or perhaps a page maintained by a FAQ Wizard or the BSCW system on http://starship.skyport.net/ I'd be happy to help, if someone would give me the necessary permissions on python.org or starship . Let me know what you think about such an index. Joe VanAndel Internet: vanandel@ucar.edu National Center for http://www.atd.ucar.edu/~vanandel/home.html Atmospheric Research From tismer@appliedbiometrics.com Thu Jan 28 17:46:32 1999 From: tismer@appliedbiometrics.com (Christian Tismer) Date: Thu, 28 Jan 1999 18:46:32 +0100 Subject: [Matrix-SIG] contributed archive for NumPy extensions? References: <199901281710.KAA22673@stout.atd.ucar.edu> Message-ID: <36B0A278.6A3241A1@appliedbiometrics.com> Joe Van Andel wrote: ... > I'd be happy to help, if someone would give me the necessary permissions on > python.org or starship . > > Let me know what you think about such an index. Great! Thank you for your help. You will have access, soon. I also contacted Gerd Woetzel, in the hope to interest him into setting up the latest BSCW software on the new ship. ciao - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaiserin-Augusta-Allee 101 : *Starship* http://starship.skyport.net 10553 Berlin : PGP key -> http://pgp.ai.mit.edu/ we're tired of banana software - shipped green, ripens at home From dubois1@llnl.gov Fri Jan 29 01:18:16 1999 From: dubois1@llnl.gov (Paul F. Dubois) Date: Thu, 28 Jan 1999 17:18:16 -0800 Subject: [Matrix-SIG] contributed archive for NumPy extensions? Message-ID: <003201be4b25$3fb43380$f4160218@c1004579-c.plstn1.sfba.home.com> I think this is a very good idea and the starship is a good place for it. LLNL is not a good place for it because of our very slow process for changing web page content and the need for disclaimers, etc. -----Original Message----- From: Christian Tismer To: vanandel@ucar.edu Cc: matrix-sig@python.org Date: Thursday, January 28, 1999 9:59 AM Subject: Re: [Matrix-SIG] contributed archive for NumPy extensions? >Joe Van Andel wrote: >... >> I'd be happy to help, if someone would give me the necessary permissions on >> python.org or starship . >> >> Let me know what you think about such an index. > >Great! Thank you for your help. You will have access, soon. >I also contacted Gerd Woetzel, in the hope to interest him >into setting up the latest BSCW software on the new ship. > >ciao - chris > >-- >Christian Tismer :^) >Applied Biometrics GmbH : Have a break! Take a ride on Python's >Kaiserin-Augusta-Allee 101 : *Starship* http://starship.skyport.net >10553 Berlin : PGP key -> http://pgp.ai.mit.edu/ > we're tired of banana software - shipped green, ripens at home > >_______________________________________________ >Matrix-SIG maillist - Matrix-SIG@python.org >http://www.python.org/mailman/listinfo/matrix-sig > > From cgw@alum.mit.edu Fri Jan 29 04:19:34 1999 From: cgw@alum.mit.edu (Charles G Waldman) Date: 28 Jan 1999 23:19:34 -0500 Subject: [Matrix-SIG] Re: NumPy core dump In-Reply-To: Chad Scherrer's message of "Wed, 27 Jan 1999 13:48:45 -0500" References: <36AF5F8D.D6AED69F@indiana.edu> Message-ID: The following message is a courtesy copy of an article that has been posted to comp.lang.python as well. Chad Scherrer writes: > import Numeric > x=Numeric.array([2l,3l,4l]) # those are L's, not ones > print x**2 > Python kicks me out with a segmentation fault and a core dump This is due to the fact that PyNumber_Power defined in the Python core (Objects/abstract.c) is a ternary function (the 3-argument version pow(x,y,z) means x**y%z), whereas NumPy's {fast_,}umathmodule.c casts it to a binary function and calls it with two arguments. The reason different people have seen different behavior with Chad's example is due to the fact that the third argument is basically random junk - in whatever was lying around on the stack when PyNubmer_Power gets called. The test at line 614 in Python/abstract.c (in the current CVS version of 1.5.2) either fails and an exception is raised (if the third argument "z" happens to point to a legal address in memory) or else if "z" is wild enough the attempt to dereference it generates a segmentation fault. Either way, it's a misbehavior. Looking at the Numeric code, it is clear that powers of general-object arrays are intended to be supported. While investigating this problem I noticed that in fact the ternary version of power is not implemented for arrays: Python 1.5.2b2 (#17, Jan 13 1999, 00:17:47) [GCC egcs-2.91.60 19981201 (e on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> import Numeric >>> Numeric.__version__ '1.9' >>> a = Numeric.array((2,3,4)) >>> b=Numeric.array((5,6,7)) >>> c=Numeric.array((4,5,6)) >>> a**b array([ 32, 729, 16384]) >>> pow(a,b,c) array([ 32, 729, 16384]) This should also get fixed. The patch below fixes the core dump problem for powers of general-object arrays, however it does not fix the fact that ternary pow() does not work on arrays. The Right fix would take care of both of these, hence the comment marked with XXX. When I have some more time I'll try to code that up. I found some other less-than-perfect behavior about the support of long integers in NumPy; for instance: Python 1.5.2b2 (#17, Jan 13 1999, 00:17:47) [GCC egcs-2.91.60 19981201 (e on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> from math import * >>> sqrt(4L) 2.0 >>> from Numeric import * >>> sqrt(4L) Traceback (innermost last): File "", line 1, in ? AttributeError: 'long int' object has no attribute 'sqrt' There is certainly some more work to be done to make support of general object arrays a little cleaner in NumPy. =================================================================== RCS file: RCS/umathmodule.c,v retrieving revision 1.1 diff --unified=2 -r1.1 umathmodule.c --- umathmodule.c 1999/01/29 03:07:20 1.1 +++ umathmodule.c 1999/01/29 04:15:48 @@ -1710,4 +1710,13 @@ } } +/* XXX hack to work around the fact that PyNumber_Power in + * Python/abstract.c is ternary; but we call it as a binary function. + * This fixes the bug where array((1L,2L,3L))*2 sometimes dumps core + * However the ternary versions of PyNumber_Power should be properly + * supported, once this gets done we can eliminate this hack and also + * get the ternary version of pow() to work for arrays */ +static PyObject * PyNumber_PowerBinary(PyObject *v,PyObject *w){ + return PyNumber_Power(v,w,Py_None); +} static PyUFuncGenericFunction add_functions[] = { UBYTE_add, SBYTE_add, SHORT_add, INT_add, LONG_add, FLOAT_add, DOUBLE_add, CFLOAT_add, CDOUBLE_add, NULL, }; static PyUFuncGenericFunction subtract_functions[] = { UBYTE_subtract, SBYTE_subtract, SHORT_subtract, INT_subtract, LONG_subtract, FLOAT_subtract, DOUBLE_subtract, CFLOAT_subtract, CDOUBLE_subtract, NULL, }; @@ -1748,5 +1757,5 @@ static void * conjugate_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)"conjugate", }; static void * remainder_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)fmod, (void *)fmod, (void *)PyNumber_Remainder, }; -static void * power_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)pow, (void *)pow, (void *)c_pow, (void *)c_pow, (void *)PyNumber_Power, }; +static void * power_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)pow, (void *)pow, (void *)c_pow, (void *)c_pow, (void *)PyNumber_PowerBinary, }; static void * absolute_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)PyNumber_Absolute, }; static void * negative_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)PyNumber_Negative, }; =================================================================== RCS file: RCS/fast_umathmodule.c,v retrieving revision 1.1 diff --unified=2 -r1.1 fast_umathmodule.c --- fast_umathmodule.c 1999/01/29 03:32:22 1.1 +++ fast_umathmodule.c 1999/01/29 04:15:49 @@ -1711,4 +1711,13 @@ } } +/* XXX hack to work around the fact that PyNumber_Power in + * Python/abstract.c is ternary; but we call it as a binary function. + * This fixes the bug where array((1L,2L,3L))*2 sometimes dumps core + * However the ternary versions of PyNumber_Power should be properly + * supported, once this gets done we can eliminate this hack and also + * get the ternary version of pow() to work for arrays */ +static PyObject * PyNumber_PowerBinary(PyObject *v,PyObject *w){ + return PyNumber_Power(v,w,Py_None); +} static PyUFuncGenericFunction add_functions[] = { UBYTE_add, SBYTE_add, SHORT_add, INT_add, LONG_add, FLOAT_add, DOUBLE_add, CFLOAT_add, CDOUBLE_add, NULL, }; static PyUFuncGenericFunction subtract_functions[] = { UBYTE_subtract, SBYTE_subtract, SHORT_subtract, INT_subtract, LONG_subtract, FLOAT_subtract, DOUBLE_subtract, CFLOAT_subtract, CDOUBLE_subtract, NULL, }; @@ -1749,5 +1758,5 @@ static void * conjugate_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)"conjugate", }; static void * remainder_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)fmod, (void *)fmod, (void *)PyNumber_Remainder, }; -static void * power_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)pow, (void *)pow, (void *)c_pow, (void *)c_pow, (void *)PyNumber_Power, }; +static void * power_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)pow, (void *)pow, (void *)c_pow, (void *)c_pow, (void *)PyNumber_PowerBinary, }; static void * absolute_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)PyNumber_Absolute, }; static void * negative_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)PyNumber_Negative, }; From zcm@llnl.gov Fri Jan 29 15:28:51 1999 From: zcm@llnl.gov (Zane Motteler) Date: Fri, 29 Jan 99 07:28:51 PST Subject: [Matrix-SIG] Re: NumPy core dump In-Reply-To: References: Chad Scherrer's message of "Wed, 27 Jan 1999 13:48:45 -0500" <36AF5F8D.D6AED69F@indiana.edu> Message-ID: Charles, You wrote: >The following message is a courtesy copy of an article >that has been posted to comp.lang.python as well. > > >Chad Scherrer writes: > >> import Numeric >> x=Numeric.array([2l,3l,4l]) # those are L's, not ones >> print x**2 > >> Python kicks me out with a segmentation fault and a core dump ...... David Ascher called my attention to this problem several days ago, and I found and fixed it, so the fix will be present in the next version of NumPy. >While investigating this problem I noticed that in fact the ternary >version of power is not implemented for arrays: ...... I was aware of this problem. It's interesting that array_power is declared as a ternaryfunc but not implemented as one. I don't know why. Most of the work on NumPy (except for PyGist) is going to be done by Paul Dubois, so I'll leave the implementation of ternary pow up to him. (I fixed the core dump problem because he was absent on jury duty this week.) ...... >There is certainly some more work to be done to make support of >general object arrays a little cleaner in NumPy. Well, yeah, I suppose so. Our main concern here at LLNL was to implement fast operations on floating point arrays. Physics simulation programs don't do much work with integers, long or not. I certainly agree that if such operations can be implemented fairly easily, then they should be. If not, at least Python shouldn't core dump--it should give a nice message that the operation is not yet implemented, and gracefully return to the prompt. Zane -------------------- Zane C. Motteler, Ph. D. Professor Emeritus of Computer Science and Engineering California Polytechnic State University, San Luis Obispo zmottel@calpoly.edu Currently: Computer Scientist Lawrence Livermore National Laboratory P O Box 808, L-038 (Street address 7000 East Avenue, L-038) Livermore, CA 94551-9900 zcm@llnl.gov 925/423-2143, FAX 925/423-9969 From hinsen@cnrs-orleans.fr Fri Jan 29 19:24:54 1999 From: hinsen@cnrs-orleans.fr (Konrad Hinsen) Date: Fri, 29 Jan 1999 20:24:54 +0100 Subject: [Matrix-SIG] Re: NumPy core dump In-Reply-To: (message from Charles G Waldman on 28 Jan 1999 23:19:34 -0500) References: <36AF5F8D.D6AED69F@indiana.edu> Message-ID: <199901291924.UAA06642@dirac.cnrs-orleans.fr> > I found some other less-than-perfect behavior about the support of > long integers in NumPy; for instance: > > Python 1.5.2b2 (#17, Jan 13 1999, 00:17:47) [GCC egcs-2.91.60 19981201 (e on linux2 > Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam > >>> from math import * > >>> sqrt(4L) > 2.0 > >>> from Numeric import * > >>> sqrt(4L) > Traceback (innermost last): > File "", line 1, in ? > AttributeError: 'long int' object has no attribute 'sqrt' Changing this would require 1) To make long int arrays a special case (not just general object arrays). 2) Implementing a square root function for long ints (which as far as I know does not exist. I am not sure that the problem is important enough to be worth the effort! Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From godzilla@netmeg.net (Les Schaffer) Sat Jan 30 04:45:36 1999 From: godzilla@netmeg.net (Les Schaffer) (Les Schaffer) Date: Fri, 29 Jan 1999 23:45:36 -0500 (EST) Subject: [Matrix-SIG] howdoi use PyArray_As2D in a sentence? Message-ID: <14002.36464.721401.197624@localhost> I am trying to figure out how to pass NumPy 2D arrays to my own C extension modules as double ** data. i am using typemaps and SWIG to do the wrapping. i am looking in the sources at PyArray_As2D and it appears to let arr->data be set to char **, which i can cast to double **. but arrayobject.c shows that the first argument to PyArray_As2D is a PyObject **, not a PyArrayObject ** which has me confused. what IS the correct way to use PyArray_As2D? here is the SWIG typemap i am using in order to pass double **: %typemap(python, in) double ** { PyArrayObject *arr; int nd, d1, d2, d3; PyObject ** op; char ***ptr; arr = (PyArrayObject *) $source; nd = arr->nd; d1 = arr->dimensions[0]; d1 = arr->dimensions[1]; op = (PyObject **) &arr; PyArray_As2D( op, ptr, &d1, &d2, PyArray_ObjectType( *op, d3 )); $target = (double **) *ptr; } but i am getting segfaults in array_fromobject(op), called from PyArray_ContiguousFromObject(*op), called from PyArray_As2D(). any help would be appreciated. thanks -- ____ Les Schaffer ___| --->> Engineering R&D <<--- Theoretical & Applied Mechanics | Designspring, Inc. Center for Radiophysics & Space Research | http://www.designspring.com/ Cornell Univ. schaffer@tam.cornell.edu | les@designspring.com From Oliphant.Travis@mayo.edu Fri Jan 29 19:36:41 1999 From: Oliphant.Travis@mayo.edu (Travis E. Oliphant) Date: Fri, 29 Jan 1999 13:36:41 -0600 Subject: [Matrix-SIG] Ternary power of arrays Message-ID: <36B20DC9.16C67E30@mayo.edu> Having recently had some experience with the ufunc object while building the cephes module I think I know why the power function was called as a binary function: All ufuncs currently must take only one or two arguments. I don't know of any way in the current code to make a ufunc object out of a function which takes more than two arguments. For the completion of the cephes module I also need functions that will take up to 5 or 6 arbitrary arguments. I don't really need the methods that come with the ufunc objects (reduce, accumulate, etc.) just the ability to take arbitrary arrays and expand dimensions cleanly. So, can we make a more generic ufunc? Or do we hack it to take 3, 4, or maybe 5 arguments? A definite item to think about for any would-be developers. ---------------------------------------------------- Travis Oliphant 200 First St SW Rochester MN 55905 Ultrasound Research Lab (507) 286-5923 Mayo Graduate School Oliphant.Travis@mayo.edu From Oliphant.Travis@mayo.edu Fri Jan 29 20:42:08 1999 From: Oliphant.Travis@mayo.edu (Travis E. Oliphant) Date: Fri, 29 Jan 1999 14:42:08 -0600 Subject: [Matrix-SIG] Ternary functions Message-ID: <36B21D20.C1DCE6FA@mayo.edu> I posted recently regarding my inability to figure out how to make a ufunc with more than two arguments and then rambled a bit about extending ufunc. Well, I've been looking at the source and have found that there does seem to be some hooks for generic functions but I don't understand how to make a generic function just yet from those sources. If anyone understands this source more I'd appreciate any advice or pointers. Thanks, Travis ---------------------------------------------------- Travis Oliphant 200 First St SW Rochester MN 55905 Ultrasound Research Lab (507) 286-5923 Mayo Graduate School Oliphant.Travis@mayo.edu From Oliphant.Travis@mayo.edu Sat Jan 30 20:56:06 1999 From: Oliphant.Travis@mayo.edu (Travis E. Oliphant) Date: Sat, 30 Jan 1999 14:56:06 -0600 Subject: [Matrix-SIG] Cephes-1.0 available incorporating a better understanding of ufuncs. Message-ID: <36B371E6.DD1BAEFA@mayo.edu> NumPy users wanting access to all kinds of great special functions: I am happy to announce that I have released version 1.0 of the cephes modules which makes available all of the functions from the cephes library to python so that they can be called with arbitrary sized arrays with familiar broadcasting rules being applied. Functions added to the module include the incomplete beta integral, F distribution functions, binomial distribution functions, and hypergeometeric functions. The discussion about power taking three arguments prompted me to look at the ufunc object source code more closely and I realized there was already a general function hook already in place. I just needed to add a few auxilliary routines. Again I was very impressed with the ufuncobject. It is a nice design---if not well documented :) The added auxilliary routines allowed me to eliminate the wrapper functions I was using as a hack to call some of the cephes functions that take integer arguments or return more than one value. So, the interface is cleaner and all of the (non-redundant) cephes calls are available in the package of your choice. http://oliphant.netpedia.net/packages/cephes-1.0.tgz http://oliphant.netpedia.net/RPMS/python-numpy-cephes-1.0-1.i386.rpm http://oliphant.netpedia.net/SRPMS/python-numpy-cephes-1.0-1.src.rpm ---------------------------------------------------- Travis Oliphant 200 First St SW Rochester MN 55905 Ultrasound Research Lab (507) 286-5923 Mayo Graduate School Oliphant.Travis@mayo.edu From Oliphant.Travis@mayo.edu Sat Jan 30 21:18:27 1999 From: Oliphant.Travis@mayo.edu (Travis E. Oliphant) Date: Sat, 30 Jan 1999 15:18:27 -0600 Subject: [Matrix-SIG] Why is there fast_umathmodule.c and umathmodule.c ? Message-ID: <36B37723.8D7A31A4@mayo.edu> I have been looking at and using the source from the umathmodules and noticed that fast_umathmodule and umathmodule are nearly identical. Perhaps somebody could tell me why there two diffeerent versions of basically the same code?? Travis From Oliphant.Travis@mayo.edu Sat Jan 30 21:43:43 1999 From: Oliphant.Travis@mayo.edu (Travis E. Oliphant) Date: Sat, 30 Jan 1999 15:43:43 -0600 Subject: [Matrix-SIG] Re: Why is there fast_umathmodule.c and umathmodule.c ? References: <199901302123.QAA04140@groucho.med.jhmi.edu> Message-ID: <36B37D0F.986C247F@mayo.edu> > > :I have been looking at and using the source from the umathmodules and > :noticed that fast_umathmodule and umathmodule are nearly identical. > :Perhaps somebody could tell me why there two diffeerent versions of > :basically the same code?? > IIRC, umath raises exception on overflows etc, fast_umath just returns > NaN. Some speed vs safety. Thanks for the quick response. I see the intention, now. I'm a bit confused at how this is done, though since the latest release of Numeric shows only the following differences. [olipt@amylou Src]$ diff fast_umathmodule.c umathmodule.c 6d5 < 1982c1981 < void initfast_umath() { --- > void initumath() { 1986c1985 < m = Py_InitModule("fast_umath", methods); --- > m = Py_InitModule("umath", methods); These seem like only superficial differences. I don't see how the two modules can be functionally different, here. From godzilla@netmeg.net (Les Schaffer) Sat Jan 30 23:02:57 1999 From: godzilla@netmeg.net (Les Schaffer) (Les Schaffer) Date: Sat, 30 Jan 1999 18:02:57 -0500 (EST) Subject: [Matrix-SIG] thisishowi use PyArray_As2D in a sentence. In-Reply-To: <14002.36464.721401.197624@localhost> References: <14002.36464.721401.197624@localhost> Message-ID: <14003.36769.567928.330643@localhost> Okay, passing 2D arrays to C extensions using a double ** pointer is easy using PyArray_As2D. Here is the typemap in case anyone else is wondering how to do this: %typemap(python, in) double ** { PyArrayObject *arr; /* will point to our array object. */ PyObject ** op; /* pointer to pointer to array object */ /* cast as PyObject; */ /* (needed by PyArray_As2d). */ int d1, d2, type; /* ints neeed by PyArray_As2D. */ char ***ptr; /* need by PyArray_As2D. */ double ** arrayPtr_2D; /* this is our double ** array pointer. */ /* C extensions can then access */ /* arbitrary sized 2D arrays as */ /* arrayPtr[i][j]. */ /* point ptr to the address of ArrayPtr_2D. */ /* PyArray_As2D will then place in it a */ /* char ** pointer aimed at a set of */ /* pointers to the rows of our array. */ ptr = (char ***) & arrayPtr_2D; /* here is our array object from caller */ arr = (PyArrayObject *) $source; /* point op at our array object pointer */ op = (PyObject **)&arr; /* i use double almost always */ type = (int) PyArray_DOUBLE; /* now let PyArray_As2D re-arrange, if necessary (?), */ /* our array and set the pointers to the rows of array */ PyArray_As2D( op, ptr, &d1, &d2, type); /* done. hand arrayPtr2D to our C ext */ $target = arrayPtr_2D; } -- ____ Dr. Les Schaffer ___| --->> Engineering R&D <<--- Theoretical & Applied Mechanics | Designspring, Inc. Center for Radiophysics & Space Research | http://www.designspring.com/ Cornell Univ. schaffer@tam.cornell.edu | les@designspring.com From mhagger@blizzard.harvard.edu Sun Jan 31 05:20:22 1999 From: mhagger@blizzard.harvard.edu (Michael Haggerty) Date: Sun, 31 Jan 1999 00:20:22 -0500 (EST) Subject: [Matrix-SIG] ANNOUNCE: Gnuplot.py 1.1 Message-ID: <199901310520.AAA06537@cyclone.harvard.edu> Hello, I would like to announce the release of version 1.1 of my Gnuplot.py module. Gnuplot.py is an interface between python and the gnuplot plotting program allowing you to make plots from Python programs. Gnuplot.py can be obtained from It requires gnuplot, Python, and the Python Numerical extension to be installed. Major new features: + 3-D plots are now supported, through the `splot' method. + The `Data' class allows a more convenient argument style (in addition to the old style). + Additional customization options. In addition, I have created more extensive notes about the package and how to install it; they can be found on the page Please, pass along any comments or feedback that you might have, or if you find Gnuplot.py useful drop me a quick email saying so. I'm interested to know how Gnuplot.py is being used and what limitations you have found. Yours, Michael -- Michael Haggerty mhagger@blizzard.harvard.edu