Mailman 3 August 2012 - NumPy-Discussion

Limit in NpyIter_MultiNew: NPY_MAXARGS
by Sergio Pascual Aug. 11, 2012

Aug. 11, 2012

I have written a program that makes use of NpyIter_Multi to iterate simultaneously over a large number of 2d arrays. But I have noticed that there is a limit of 32 in the number of simultaneous iterators allowed. This number is NPY_MAXARGS. I typically need to iterate over hundreds of images. So, what is the rationale to have the limit in 32 instead of, say, 1024? If this number cannot easily be increased in the source code, what is the alternative to iterate over a large number of images? Could it be an array of NpyIter objects? Regards, Sergio

1 0

Second try: possible bug in assignment to complex array
by Mark Bakker Aug. 10, 2012

Aug. 10, 2012

I am giving this a second try. Can anybody help me out? > > I think there is a problem with assigning a 1D complex array of length one > to a position in another complex array. > > Example: > > a = ones(1,'D') > b = ones(1,'D') > a[0] = b > --------------------------------------------------------------------------- > TypeError Traceback (most recent call last) > <ipython-input-37-0c4fc6d780e3> in <module>() > ----> 1 a[0] = b > > TypeError: can't convert complex to float > > This works correctly when a and b are real arrays: > > a = ones(1) > b = ones(1) > a[0] = b > > Bug or feature? > > Thanks, > > Mark >

6 6

Bug in as_strided/reshape
by Dave Hirschfeld Aug. 10, 2012

Aug. 10, 2012

It seems that reshape doesn't work correctly on an array which has been resized using the 0-stride trick e.g. In [73]: x = array([5]) In [74]: y = as_strided(x, shape=(10,), strides=(0,)) In [75]: y Out[75]: array([5, 5, 5, 5, 5, 5, 5, 5, 5, 5]) In [76]: y.reshape([10,1]) Out[76]: array([[ 5], [ 8], [ 762933412], [-2013265919], [ 26], [ 64], [ 762933414], [-2013244356], [ 26], [ 64]]) <================ Should all be 5???????? In [77]: y.copy().reshape([10,1]) Out[77]: array([[5], [5], [5], [5], [5], [5], [5], [5], [5], [5]]) In [78]: np.__version__ Out[78]: '1.6.2' Perhaps a clause such as below is required in reshape? if any(stride == 0 for stride in y.strides): return y.copy().reshape(shape) else: return y.reshape(shape) Regards, Dave

2 3

how to uninstall numpy
by Petro Aug. 9, 2012

Aug. 9, 2012

Hi list This is a general python question but I will ask it here. To install a new numpy on Debian testing I remove installed version with "aptitude purge python-numpy" download numpy source code and install numpy with "sudo python setup.py install". If I want to remove the installed numpy how do I proceed? Thanks. Petro

6 6

nested loops too slow
by nicolas aunai Aug. 9, 2012

Aug. 9, 2012

Hi, I'm trying to write a code for doing a 2D integral. It works well when I'm doing it with normal "for" loops, but requires two nested loops, and is much too slow for my application. I would like to know if it is possible to do it faster, for example with fancy indexing and the use of numpy.cumsum(). I couln't find a solution, do you have an idea ? The code is the following : http://bpaste.net/show/cAkMBd3sUmhDXq0sIpZ5/ 'flux2' is the result of the calculation with 'for' loops implementation, and 'flux' is supposed to be the same result without the loop. If I managed to do it for the single loops (line 23 is identical to lines 20-21, and line 30 is identical to line 27,28) and don't know how to do for the nested loops lines 33-35 (line 40 does not give the same result). Any idea ? Thanks much Nico

2 1

Re: [Numpy-discussion] building numpy 1.6.2 on OSX 10.6 / Python2.7.3
by Andrew Nelson Aug. 9, 2012

Aug. 9, 2012

> > > Message: 1 > Date: Wed, 8 Aug 2012 08:43:18 +0100 > From: David Cournapeau <cournape(a)gmail.com> > Subject: Re: [Numpy-discussion] building numpy 1.6.2 on OSX 10.6 / > Python2.7.3 > To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> > Message-ID: > < > CAGY4rcW4B9krQXBibyEjeW8DZRjwBxCgV4yDZiBpWcG2zVH5VA(a)mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > On Wed, Aug 8, 2012 at 6:15 AM, Andrew Nelson <andyfaff(a)gmail.com> wrote: > > Dear Pierre, > > as indicated yesterday OSX system python is in: > > > > /System/Library/Frameworks/Python.framework/ > > > > I am installing into: > > > > > /Library/Frameworks/Python.framework/Versions/Current/lib/python2.7/site-packages > > > > This should not present a problem and does not explain why numpy does not > > build/import correctly on my setup. > > Please give us the build log (when rebuilding from scratch to have the > complete log) so that we can have a better idea of the issue, > > David > > > The build log for the build that fails on my machine can be found at: http://dl.dropbox.com/u/15288921/log Examining the symbols again: p0100m:core anz$ pwd /Users/anz/Downloads/numpy-1.6.2/build/lib.macosx-10.6-intel-2.7/numpy/core p0100m:core anz$ nm multiarray.so | grep ceil U _npy_ceil U _npy_ceil

1 0

Is there a more efficient way to do this?
by Laszlo Nagy Aug. 8, 2012

Aug. 8, 2012

Is there a more efficient way to calculate the "slices" array below? import numpy import numpy.random # In reality, this is between 1 and 50. DIMENSIONS = 20 # In my real app, I have 100...1M data rows. ROWS = 1000 DATA = numpy.random.random_integers(0,100,(ROWS,DIMENSIONS)) # This is between 0..DIMENSIONS-1 DRILLBY = 3 # Array of row incides that orders the data by the given dimension. o = numpy.argsort(DATA[:,DRILLBY]) # Input of my task: the data ordered by the given dimension. print DATA[o,DRILLBY] #~ [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 #~ 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 #~ 2 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 #~ 4 4 4 4 5 5 5 5 5 5 5 5 5 5 6 6 6 6 #~ .... many more things here #~ 96 96 96 97 97 97 97 97 97 97 97 97 98 98 98 98 98 98 #~ 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 100 100 #~ 100 100 100 100 100 100 100 100 100 100] # Output of my task: determine slices for the same values on the DRILLBY dimension. slices = [] prev_val = None sidx = -1 # Dimension values for the given dimension. fdv = DATA[:,DRILLBY] # Go over the rows, sorted by values of didx for oidx,rowidx in enumerate(o): val = fdv[rowidx] if val!=prev_val: if prev_val is None: prev_val = val sidx = oidx else: slices.append((prev_val,sidx,oidx)) sidx = oidx prev_val = val if (sidx>=0) and (sidx<ROWS): slices.append((val,sidx,ROWS)) slices = numpy.array(slices,dtype=numpy.int64) # This is what I want to have! print slices #~ #~ [[ 0 0 14] #~ [ 1 14 26] #~ [ 2 26 37] #~ [ 3 37 44] #~ .... many more values here #~ [ 4 44 58] #~ [ 96 952 957] #~ [ 97 957 966] #~ [ 98 966 972] #~ [ 99 972 988] #~ [ 100 988 1000]] So for example, to get all row incides where dimension value is zero: zeros at rows o[0:14] Or, to get all row incides where dimension value is 99: o[988:1000] etc. I do not want to make copies of DATA, because it can be huge. The argsort is fast enough. I just need to create slices for different dimensions. The above code works, but it does a linear time search, implemented in pure Python code. For every iteration, Python code is executed. For 1 million rows, this is very slow. Is there a way to produce "slices" with numpy code? I could write C code for this, but I would prefer to do it with mass numpy operations. Thanks, Laszlo

2 2

possible bug in assignment to complex array
by Mark Bakker Aug. 8, 2012

Aug. 8, 2012

Dear List, I think there is a problem with assigning a 1D complex array of length one to a position in another complex array. Example: a = ones(1,'D') b = ones(1,'D') a[0] = b --------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-37-0c4fc6d780e3> in <module>() ----> 1 a[0] = b TypeError: can't convert complex to float This works correctly when a and b are real arrays: a = ones(1) b = ones(1) a[0] = b Bug or feature? Thanks, Mark

1 0

Licensing question
by Damon McDougall Aug. 8, 2012

Aug. 8, 2012

Hi, I have a question about the licence for NumPy's codebase. I am currently writing a library and I'd like to release under some BSD-type licence. Unfortunately, my choice to link against MIT's FFTW library (released under the GPL) means that, in its current state, this is not possible. I'm an avid NumPy user and thought to myself that, since NumPy's licence is BSD, I'd be able to use some of the source code (with due credit, of course) instead of FFTW. Is this possible? I mean, can I redistribute *PART* of NumPy's codebase? Namely, the fftpack.c file? I was under the impression that I could only redistribute BSD source code as a whole and then I read the licence more carefully and it states that I can modify the source to suit my needs. I consider 'redistributing a single file and ignoring the other files' as a 'modification' under the BSD definition, but maybe I'm thinking too wishfully here. Any information on this matter would be greatly appreciated since I am a total code licence noob. Thank you. P.S. Yes, I know I could just release under the GPL, but I don't want to turn people off of packaging my work into a useful product licensed under BSD, or even make money from it. -- Damon McDougall http://damon-is-a-geek.com B2.39 Mathematics Institute University of Warwick Coventry West Midlands CV4 7AL United Kingdom

9 11

building numpy 1.6.2 on OSX 10.6 / Python2.7.3
by Andrew Nelson Aug. 8, 2012

Aug. 8, 2012

Dear list, I am trying to build numpy 1.6.2 from source but am running up against a few problems. Platform: OSX10.6.8 Python: 2.7.3 (compiled using gcc 4.2.1) gcc: 4.2.1 gfortran: 4.2.1 I try the normal build sequence: python setup.py build sudo python setup.py install However, when I try to import numpy I get: >>> import numpy Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/__init__.py", line 137, in <module> import add_newdocs File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/add_newdocs.py", line 9, in <module> from numpy.lib import add_newdoc File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/lib/__init__.py", line 4, in <module> from type_check import * File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/lib/type_check.py", line 8, in <module> import numpy.core.numeric as _nx File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/__init__.py", line 5, in <module> import multiarray ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/multiarray.so, 2): Symbol not found: _npy_ceil Referenced from: /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/multiarray.so Expected in: flat namespace in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/multiarray.so The numpy source was from the Sourceforge official page. When I run nm on the multiarray module I get: %nm /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/multiarray.so .......<snip a lot of stuff> U _npy_ceil U _npy_double_to_half U _npy_doublebits_to_halfbits U _npy_float_to_half U _npy_floatbits_to_halfbits U _npy_half_isnan U _npy_half_iszero U _npy_half_le U _npy_half_lt_nonan U _npy_half_to_double U _npy_half_to_float U _npy_halfbits_to_doublebits U _npy_halfbits_to_floatbits So it seems that the _npy_ceil symbol is undefined. I looked at /build/src.macosx-10.6-intel-2.7/numpy/core/include/numpy/config.h and it contains: #define HAVE_CEIL Am I doing something wrong? regards, Andrew -- _____________________________________ Dr. Andrew Nelson _____________________________________

4 4