Mailman 3 June 2007 - NumPy-Discussion

problem compiling v.1.0.3 on a Mac
by John Cartwright 07 Jul '07

07 Jul '07

Hello All, I'm having trouble compile on a Mac 10.4.10. It seems as if it's not finding /usr/include: ... from /Library/Frameworks/Python.framework/Versions/ 2.4/include/python2.4/Python.h:81, from _configtest.c:2: /usr/include/stdarg.h:4:25: error: stdarg.h: No such file or directory ... I tried setting the "CFLAG=-I/usr/include", but w/o success. Can anyone help me? Thanks! --john

3 9

Problems with RandomArray
by Alexander Dietz 07 Jul '07

07 Jul '07

Hi, I am not sure how to subscribe to this mailing list (there was no link for that, just this email adress), but hopefully someone will get this email and can me subscribe to this list, answer my question or ask someone else. Anyway, here is my question: I am using python with matplotlib version 0.90.1 and with numpy (as recommended), on a Linux box. So far matplotlib and numpy is working, but I need to use RandomArray! So, RandomArray can be found in the "Numerical Python" documentation and RandomArray.py can also be found within the "numeric" directory. If including this to my PYTHONPATH variable I can import RandomArray and also use some of the functions! But the function I am interested in is: multivariate_normal. When I try to use this function python stops responding, I have to kill python from outside! Is there a way to make this function work? Or maybe there is a quick workaround using functions in random and else? That would be really great! Cheers Alex

2 1

Aligning an array on Windows
by Francesc Altet 07 Jul '07

07 Jul '07

Hi, Some time ago I made an improvement in speed on the numexpr version of PyTables so as to accelerate the operations with unaligned arrays (objects that can appear quite commonly when dealing with columns of recarrays, as PyTables does). This improvement has demostrated to work correctly and flawlessly in Linux machines (using GCC 4.x and in both 32-bit and 64-bit Linux boxes) for several weeks of intensive testing. Moreover, its speed-up is ranging from a 40% on modern processors and up to a 70% in older ones, so I'd like to keep it. The surprise came today when I tried to compile the same code on a Windows box (Win XP Service Pack 2) using MSVC 7.1, through the free (as in beer) Toolkit 2003. The compilation process went fine, but the problem is that I'm getting crashes from time to time when running the numexpr test suite. After some in-depth investigation, I'm pretty sure that the problem is in a concrete part of the code that I'd modified for this improvement. IMO, the affected code is in numexpr/interp_body.c and reads like: case OP_COPY_II: VEC_ARG1(memcpy(dest, x1, sizeof(int)); dest += sizeof(int); x1 += stride1); case OP_COPY_LL: VEC_ARG1(memcpy(dest, x1, sizeof(long long)); dest += sizeof(long long); x1 += stride1); case OP_COPY_FF: VEC_ARG1(memcpy(dest, x1, sizeof(double)); dest += sizeof(double); x1 += stride1); case OP_COPY_CC: VEC_ARG1(memcpy(dest, x1, sizeof(double)*2); dest += sizeof(double)*2; x1 += stride1); This might seem complicated, but it is not. Each of the OP_COPY_XX is a function that has to copy source (x1) to destination (dest) for int, long long, double and complex data types (this function will be called in a loop for copying all the data in array). The code for the original numexpr reads as: case OP_COPY_BB: VEC_ARG1(b_dest = b1); case OP_COPY_II: VEC_ARG1(i_dest = i1); case OP_COPY_FF: VEC_ARG1(f_dest = f1); case OP_COPY_CC: VEC_ARG1(cr_dest = c1r; ci_dest = c1i); i.e. the copy is done through direct assignment. This can be done because, in the original numexpr, an array is always guaranteed to reach this part of the code (the computing kernel) in the aligned form. But in my code, this is not guaranteed (the copy is made precisely for alignment purposes), so this is why I need to make use of memcpy/memmove calls. The thing I don't see is why my version of the code can create problems on Windows platforms and work perfectly on Linux ones. I've tried to use memmove instead of memcpy, but the problem persists. I've had a look at how numpy makes an 'aligned' copy of an unaligned array, and it seems to me that it uses memcpy/memmove (not sure when you should use one or another) just as I use it above. It might be possible that the problem is in another place, but my tests reaffirm me in the possibility that something is wrong with my copy code above (but again, I can't see where). Of course, we can get rid of this optimization but it is a bit depressing to have renounce to it just because it doesn't work on Windows :( Thanks in advance for any hint you may provide! -- Francesc Altet | Be careful about using the following code -- Carabos Coop. V. | I've only proven that it works, www.carabos.com | I haven't tested it. -- Donald Knuth

2 4

how do I configure with gfortran
by Mathew Yeates 06 Jul '07

06 Jul '07

Does anyone know how to run python setup.py build and have gfortran used? It is in my path. Mathew

6 17

arr.dtype.byteorder == '=' --- is this "good code"
by Sebastian Haase 03 Jul '07

03 Jul '07

Hi, Suppose I'm on a little-edian system. Could I have a little-endian numpy array arr, where arr.dtype.byteorder would actually be "<" instead of "=" !? There are two kinds of systems: little edian and big endian. But there are three possible byteorder values: "<", ">" and "=" I assume that if arr.dtype.byteorder is "=" then, even on a little endian system the comparison arr.dtype.byteorder == "<" still fails !? Or are the == and != operators overloaded !? Thanks, Sebastian Haase

2 4

bug in lexsort with two different dtypes?
by Tom Denniston 30 Jun '07

30 Jun '07

In [1]: intArr1 = numpy.array([ 0, 1, 2,-2,-1, 5,-5,-5]) In [2]: intArr2 = numpy.array([1,1,1,2,2,2,3,4]) In [3]: charArr = numpy.array(['a','a','a','b','b','b','c','d']) Here I sort two int arrays. As expected intArr2 dominates intArr1 but the items with the same intArr2 values are sorted forwards according to intArr1 In [6]: numpy.lexsort((intArr1, intArr2)) Out[6]: array([0, 1, 2, 3, 4, 5, 6, 7]) This, however, looks like a bug to me. Here I sort an int array and a str array. As expected charArray dominates intArr1 but the items with the same charArray values are sorted *backwards* according to intArr1 In [5]: numpy.lexsort((intArr1, charArr)) Out[5]: array([2, 1, 0, 5, 4, 3, 6, 7]) Is this a bug or am I missing something? --Tom

2 4

Accelerate your Python code with parallel processing
by Ronnie Hoogerwerf 30 Jun '07

30 Jun '07

I am an Application Engineer at Interactive Supercomputing and we are rolling out a beta version of our Star-P product for Python. We are actively looking for computationally intensive Python application to port to Star-P. Star-P is a parallel application development platform that allows users to tap into the power and memory of supercomputers from the comfort of the favorite desktop applications, in this case Python. Star-P is capable of both fine-grained parallel computation and embarrassingly parallel computation. The fine-grained mode of our Star-P Python implementation has been modeled on the Python NumPy package - for example: x = starp.random.rand(20000,20000) y = starp.linalg.inv(x) instead of x = numpy.random.rand(20000,20000) y = numpy.linalg.inv(x) Where the first couple of lines are executed on the Star-P parallel server in full C/MPI mode and the last couple of lines are executed on the desktop using Python. The embarrassingly parallel mode is capable of executing any Python module, although input and output parameters are currently limited to NumPy arrays, scalars, and strings - for example: y = starp.ppeval(mymodule.dosomething,x) instead of for i in range(0,n): y[:,:,i] = mymodule.dosomething(x[:,i]) Where again in the former example the iterations are spread out over the available CPUs (note the abstraction - user need not worry regarding the number of CPUs) on the Star-P server using Python and in the latter the looping is doing in serial on the client using Python. We are looking for real Python application that you would be willing to share with us that we can port to Star-P. We want to use this experience as a basis for further improvements and development of our Python client. Thanks, Ronnie

3 4

[ANN]New numpy, scipy and atlas rpms for FC 5, 6 and 7 and openSUSE (with 64 bits arch support)
by David Cournapeau 29 Jun '07

29 Jun '07

Hi there, After quite some pain, I finally managed to build a LAPACK + ATLAS rpm useful for numpy and scipy. Read the following if you use Fedora Core or OpenSuse and are tired to install unsuccessfully numpy, scipy, BLAS, LAPACK or ATLAS. Instructions are given there: http://www.scipy.org/Installing_SciPy/Linux (ashigabou repository) Basically: - Fedora Core 5, 6 and 7 and openSUSE 10.2 are supported (x86, and x86_64 for FC 7 and openSuse). - binary rpms for numpy, scipy and blas/lapack dependencies. - source rpm for atlas, for a really easy, 3 commands build of ATLAS (should work for both x86 and x86_64). numpy and scipy are the last releases, including some backported changes to make it work on 64 bits. Atlas is the last developement version, with a trivial patch to build shared blas and lapack which can be used as drop in replacements for netlib blas and lapack. I would like to hear people complains. If people want other distributions supported by the opensuse build system (such as mandriva), I would like to hear it too. cheers, David

2 3

Re: [Numpy-discussion] Python equivalent of bwboundaries, bwlabel
by Nadav Horesh 29 Jun '07

29 Jun '07

There are several image processing function (including labeling) in numpy.numarray.nd_image package. You can find nd_image documentation in numarray-1.5.pdf (http://downloads.sourceforge.net/numpy/numarray-1.5.pdf?modtime=1133880381&…) Nadav. -----Original Message----- From: numpy-discussion-bounces(a)scipy.org on behalf of Cameron Walsh Sent: Fri 29-Jun-07 09:47 To: numpy-discussion(a)scipy.org; image-sig(a)python.org Cc: Subject: [Numpy-discussion] Python equivalent of bwboundaries, bwlabel Hi all, I'm trying to do object segmentation from image slices, and have found the matlab functions bwlabel and bwboundaries. I haven't been able to find a python equivalent in the pylab, scipy, numpy, or Image modules, nor has google been fruitful. Could somebody point me in the right direction to find equivalent functions if they exist? Cameron.

3 2

Python equivalent of bwboundaries, bwlabel
by Cameron Walsh 28 Jun '07

28 Jun '07

Hi all, I'm trying to do object segmentation from image slices, and have found the matlab functions bwlabel and bwboundaries. I haven't been able to find a python equivalent in the pylab, scipy, numpy, or Image modules, nor has google been fruitful. Could somebody point me in the right direction to find equivalent functions if they exist? Cameron.

1 0