Mailman 3 November 2001 - NumPy-Discussion

Numeric Python EM Project has moved
by Rob Nov. 30, 2001

Nov. 30, 2001

Its now at www.pythonemproject.com. I can be reached at rob(a)pythonemproject.com. All this has come about since @home is possibly suspending operation at midnite tonight :( Rob. Looks like I need to change my sig too :) -- The Numeric Python EM Project www.members.home.net/europax

1 0

equal() and complex
by Mike Romberg Nov. 30, 2001

Nov. 30, 2001

I'm wondering if there is some good reason why equal(), not_equal(), nonzero() and the like do not work with numeric arrays of tyep complex. I can see why operators like less() and less_equal() do not work. But the pure equality ones seem like they should work. Or am I missing something :). Thanks, Mike Romberg (romberg(a)fsl.noaa.gov)

2 1

Using Reduce with Multi-dimensional Masked array
by sag＠hydrosphere.com Nov. 29, 2001

Nov. 29, 2001

I posted the following inquiry to python-list(a)python.org earlier this week, but got no responses, so I thought I'd try a more focused group. I assume MA module falls under NumPy area. I am using 2 (and more) dimensional masked arrays with some numeric data, and using the reduce functionality on the arrays. I use the masking because some of the values in the arrays are 'missing' and should not be included in the results of the reduction. For example, assume a 5 x 2 array, with … [View More]

4 6

Re: [Numpy-discussion] Meta: too many numerical libraries doing thesame
by Konrad Hinsen Nov. 28, 2001

Nov. 28, 2001

"eric" <eric(a)enthought.com> writes: > The standard version that Robin Dunn distributes is compiled with MSVC. If > you build a small > extensions with gcc that make wxPython call, it'll link just fine, but > seg-faults during execution. > Does anyone know if the same sorta thing is true on the Unices? If it is, > and Numeric was written in C++ then you'd have to compile extension modules > that use Numeric arrays with the same compiler that was used to compile &… [View More]

1 0

RE: [Numpy-discussion] Meta: too many numerical libraries doing thesame
by eric Nov. 27, 2001

Nov. 27, 2001

Hey group, Blitz++ is very cool, but I'm not sure it would make a very good underpinning for reimplementing Numeric. There are 2 (well maybe 3) main points. 1. Blitz++ declares arrays in the following way: The first issue deals with how you declare arrays in Blitz++. Array<float,3> A(N,N,N); The big deal here is that the dimensionality of Array is a template parameter, not a constructor parameter. In other words, 2D arrays are effectively a different type than 3D arrays. … [View More]Numeric, on the other hand represents arrays of all dimensions with a single class/type. For Python, this makes the most sense. I think you could fanagle some way of getting blitz to work, but I'm not sure it would be the desired elegant solution. I've also tinkered with building a simple C++ templated (non-blitz) implementation of Numeric for kicks, but kept coming back to using the dreaded void* to store the data arrays. I still haven't completely given up on a templated solution, but it wasn't as obvious as I thought it would be. 2. Compiling Blitz++ is slooooow. scipy.compiler spits out 200-300 line extension modules at the most. Depending on hox complicated expressions are, it can take .5-1.5 minutes to compile a single extension funtion on an 850 MHz PIII. I can't imagine how long it would take to compile Numeric arrays for 1 through 11 dimensions (the most blitz supports as I remember) for all the different data types with 100s of extension functions. The cost wouldn't be linear because you do pay a one time hit for some of the template instantiation. Also, I've heard gcc 3.0 might be better. Still, it'd be a painful development process. 3. Portability. This comes at two levels. The first is that blitz++ has heavy duty requirements of the compiler. gcc works fine which is a huge plus, but a lot of other compilers don't. MSVC is the most notable of these because it is so heavily used on windows. The second level is the portability of C++ extension modules in general. I've run into this on windows, but I think it is an issue pretty much everywhere. For example, MSVC and GCC compiled C extension libraries can call each other on Windows because they the are binary compatible. C++ classes are _not_ binary compatible. This has come up for me with wxPython. The standard version that Robin Dunn distributes is compiled with MSVC. If you build a small extensions with gcc that make wxPython call, it'll link just fine, but seg-faults during execution. Does anyone know if the same sorta thing is true on the Unices? If it is, and Numeric was written in C++ then you'd have to compile extension modules that use Numeric arrays with the same compiler that was used to compile Numeric. This can lead to all sorts of hassles, and it has made me lean back towards C as the preferred language for something as fundemental as Numeric. (Note that I do like C++ for modules that don't really define an API called by other modules). Ok, so maybe there's a 4th point. Paul D. pointed out that blitz isn't much of a win unless you have lazy evaluation (which scipy.compiler already provides). I also think improved speed _isn't_ the biggest goal of a reimplementation (although it can't be sacrificed either). I'm more excited about a code base that more people can comprehend. Perry G. et al's mixed Python/C implementation with the code generators is a very good idea and a step in this direction. I hope the speed issues for small arrays can be solved. I also hope the memory mapped aspect doesn't complicate the code base much. see ya, eric [View Less]

1 0

BLAS and innerproduct
by R.M.Everson＠exeter.ac.uk Nov. 27, 2001

Nov. 27, 2001

Hello, So far as I can tell Numeric.dot(), which uses innerproduct() from multiarraymodule.c doesn't call the BLAS, even if Numeric was compiled against native BLAS. This means (at least on my machine) that X = ones((150, 16384), 'd') C = dot(X, tranpose(X)) is about 15 times as slow as the comparable operations in Matlab (v6), which does, I think, use the native BLAS. I guess that multiarray.c is not particularly optimised to use the BLAS because of the difficulties of coping with all … [View More]

5 4

RE: [Numpy-discussion] Re: Re-implementation of Python Numerical arrays (Numeric) available for download
by Krishnaswami, Neel Nov. 27, 2001

Nov. 27, 2001

Perry Greenfield [mailto:perry@stsci.edu] wrote: > > > > I know large datasets were one of your driving factors, but I really > > don't want to make performance on smaller datasets secondary. > > That's why we are asking, and it seems so far that there are enough > of those that do care about small arrays to spend the effort to > significantly improve the performance. Well, here's my application. I do data mining work, and one of the techniques I want to use … [View More]

1 0

Numpy2 and GSL
by Martin Wiechert Nov. 27, 2001

Nov. 27, 2001

Hi! Just an uneducated question. Are there any plans to wrap GSL for Numpy2? I did not actually try it (It's not Python ;-)), but it looks clean and powerful. Regards, Martin.

5 4

Re: [Numpy-discussion] Meta: too many numerical libraries doing the same thing?
by Konrad Hinsen Nov. 26, 2001

Nov. 26, 2001

"Christos Siopis <siopis(a)umich.edu>" <siopis(a)umich.edu> writes: > Don't take me wrong, i am not saying that Numpy, PDL, GSL & co. should be > somehow "merged" --obviously, one needs different wrappers to call > numerical routines from Python, Perl, C, C++ or Java. But there should be > a way so that the actual *implementation* of the numerical algorithms is > only done once and for all. I agree that sounds nice in theory. But even if it were technically … [View More]

3 4

Re: [Numpy-discussion] Meta: too many numerical libraries doing the same thing?
by Chris Barker Nov. 26, 2001

Nov. 26, 2001

"Paul F. Dubois" wrote: > We had some meetings to discuss using blitz and the truth is that as > wrapped by Python there is not much to gain. The efficiency of blitz > comes up when you do an array expression in C++. Then x = y + z + w + a > + b gets compiled into one loop with no temporary objects created. But > this trick is possible because you can bind the assignment. In python > you cannot bind the assignment so you cannot do a lazy evaluation of the > operations, … [View More]

2 2