Mailman 3 December 2010 - NumPy-Discussion

Refactor fork uses the ./configure, make, make install process.
by Charles R Harris 08 Dec '10

08 Dec '10

Hi Jason, Just wondering if this is temporary or the intention is to change the build process? I also note that the *.h files in libndarray are not complete and a *lot* of trailing whitespace has crept into the files. Chuck

8 16

ctypes and numpy
by Sebastian Walter 07 Dec '10

07 Dec '10

Hello all, I'd like to call a Python function from a C++ code. The Python function has numpy.ndarrays as input. I figured that the easiest way would be to use ctypes. However, I can't get numpy and ctypes to work together. ----------- run.c ------------ #include <Python.h> #include <numpy/noprefix.h> void run(PyArrayObject *y, PyObject *f) { npy_intp Ny = PyArray_SIZE(y); } ------- end run.c ---------- -------- run.py ---------- import os, ctypes, numpy _r = numpy.ctypeslib.load_library('librun.so', os.path.dirname(__file__)) _r.run.argtypes = [ctypes.py_object] x = numpy.array([1,2,3.]) _r.run(x) -------- end run.py ---------- Compiling gives me the warning: gcc -o librun.so -I/usr/include/python2.6 -I/usr/lib/python2.6/dist-packages/numpy/core/include -O0 -fpic -shared -Wall run.c run.c: In function ‘run’: run.c:5: warning: unused variable ‘Ny’ run.c: At top level: /usr/include/python2.6/numpy/__multiarray_api.h:968: warning: ‘_import_array’ defined but not used and when I run it I get a segmentation fault. I guess I'm not the first one who has this problem, but I couldn't find something useful on the web. Any pointers are suggestions are welcome. cheers, Sebastian

1 0

The power of strides - Combinations
by Mario Moura 07 Dec '10

07 Dec '10

Hi Folks Is it possible some example how deal with strides with combinations, let see: >>> from numpy import * >>> import itertools >>> dt = dtype('i,i,i') >>> a = fromiter(itertools.combinations(range(10),3), dtype=dt, count=-1) >>> a array([(0, 1, 2), (0, 1, 3), (0, 1, 4), (0, 1, 5), (0, 1, 6), (0, 1, 7), (0, 1, 8), (0, 1, 9), (0, 2, 3), (0, 2, 4), (0, 2, 5), (0, 2, 6), (0, 2, 7), (0, 2, 8), (0, 2, 9), (0, 3, 4), (0, 3, 5), (0, 3, 6), (0, 3, 7), (0, 3, 8), (0, 3, 9), (0, 4, 5), (0, 4, 6), (0, 4, 7), (0, 4, 8), (0, 4, 9), (0, 5, 6), (0, 5, 7), (0, 5, 8), (0, 5, 9), (0, 6, 7), (0, 6, 8), (0, 6, 9), (0, 7, 8), (0, 7, 9), (0, 8, 9), (1, 2, 3), (1, 2, 4), (1, 2, 5), (1, 2, 6), (1, 2, 7), (1, 2, 8), (1, 2, 9), (1, 3, 4), (1, 3, 5), (1, 3, 6), (1, 3, 7), (1, 3, 8), (1, 3, 9), (1, 4, 5), (1, 4, 6), (1, 4, 7), (1, 4, 8), (1, 4, 9), (1, 5, 6), (1, 5, 7), (1, 5, 8), (1, 5, 9), (1, 6, 7), (1, 6, 8), (1, 6, 9), (1, 7, 8), (1, 7, 9), (1, 8, 9), (2, 3, 4), (2, 3, 5), (2, 3, 6), (2, 3, 7), (2, 3, 8), (2, 3, 9), (2, 4, 5), (2, 4, 6), (2, 4, 7), (2, 4, 8), (2, 4, 9), (2, 5, 6), (2, 5, 7), (2, 5, 8), (2, 5, 9), (2, 6, 7), (2, 6, 8), (2, 6, 9), (2, 7, 8), (2, 7, 9), (2, 8, 9), (3, 4, 5), (3, 4, 6), (3, 4, 7), (3, 4, 8), (3, 4, 9), (3, 5, 6), (3, 5, 7), (3, 5, 8), (3, 5, 9), (3, 6, 7), (3, 6, 8), (3, 6, 9), (3, 7, 8), (3, 7, 9), (3, 8, 9), (4, 5, 6), (4, 5, 7), (4, 5, 8), (4, 5, 9), (4, 6, 7), (4, 6, 8), (4, 6, 9), (4, 7, 8), (4, 7, 9), (4, 8, 9), (5, 6, 7), (5, 6, 8), (5, 6, 9), (5, 7, 8), (5, 7, 9), (5, 8, 9), (6, 7, 8), (6, 7, 9), (6, 8, 9), (7, 8, 9)], dtype=[('f0', '<i4'), ('f1', '<i4'), ('f2', '<i4')]) >>> Many thanks Mr. Warren about this ((itertools.combinations(range(10),3), dtype=dt, count=-1)) But as you can see itertools.combinations are emitted in lexicographic sort order but NOT with "power of" strides. So what I see is every element in this array into one memory spot but I would like to know if is possible, use "the power of strides"! >>> x = a.reshape(120,1) x = stride_tricks.as_strided(a,shape=(120,),strides=(4,4)) Should I use some sub-class like record array, scalar array? So what I want is repetitive elements on same memory spot. I want save memory in big arrays (main reason) and want go fast. How can I deal with this in random arrays but with repetitive elements? Is it possible have custom strides in subclass(that change in dimension) ? How do this? Best Regards Mario

3 3

Can I add rows and columns to recarray?
by Wai Yip Tung 07 Dec '10

07 Dec '10

I'm fairly new to numpy and I'm trying to figure out the right way to do things. Continuing on my question about using recarray as a relation. I have a recarray like this In [339]: arr = np.array([ .....: (1, 2.2, 0.0), .....: (3, 4.5, 0.0) .....: ], .....: dtype=[ .....: ('unit',int), .....: ('price',float), .....: ('amount',float), .....: ] .....: ) In [340]: data = arr.view(recarray) One of the most common thing I want to do is to append rows to data. I think concatenate() might be the method. But I get a problem: In [342]: np.concatenate((data0,[1,9.0,9.0])) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) c:\Python26\Lib\site-packages\numpy\<ipython console> in <module>() TypeError: expected a readable buffer object The other thing I want to do is to calculate the column value. Right now it can do great thing like In [343]: data.amount = data.unit * data.price But sometimes it may require me to add a new column not already exist, e.g.: In [344]: data.discount_price = data.price * 0.9 How can I add a new column? I tried column_stack. But it give a similar TypeError. I figure I need to first specify the type of the column. But I don't know how. Thanks, Wai Yip

5 7

numpy rec array and sorting
by John 06 Dec '10

06 Dec '10

Hello, I have been trying two methods for creating a rec array from my data (or a structured array -- I'm still not completely clear on the distinction). In terms of data, you can see what types they are, basically simple (n,1) np.ndarrays. I had to reshape them to (n,1) to get them to work with hstack. The 'NON WORKING' method returns no errors, but when I go to 'sort' the data array that is returned, no sorting takes place, whereas with the 'WORKING' method, I can do: data.sort(order='sza') and my data.indices match the sorted 'sza' data. You can see I also tried to include a 2-d array, but I haven't managed to get this to work... Could someone please explain what is going on here? Thanks, john ## Create recarray so we can easily sort dtype=np.dtype([('indices','int32'),('time','f8'),('zen','f4'),\ ('az','f4'),('sza','f4'),('saz','f4'),('muslope','f4'),\ ('roll','f4'),('pitch','f4'),('yaw','f4')\ #,('spectra',np.ma.core.MaskedArray,c.shape) ]) ### WORKING METHOD values = np.hstack((indices,time,zen,az,sza,saz,musl,roll,pitch,yaw)) data = [[] for dummy in xrange(len(dtype))] for i in xrange(len(dtype)): data[i] = cast[dtype[i]](values[:,i]) data = np.rec.array(data,dtype=dtype) ### NON WORKING METHOD ### values = (indices,time,zen,az,sza,saz,musl,roll,pitch,yaw) data = np.rec.fromarrays(values,dtype=dtype)

1 0

Structured array? recarray? issue access by attribute name
by Wai Yip Tung 06 Dec '10

06 Dec '10

I'm trying to use numpy to manipulate CSV file. I'm looking for feature similar to relational database. So I come across a class recarray that seems to meet my need. And then I see other references of structured array. Are these just different name of the same feature? Also I encounter a problem trying to access an field by attribute name. I have In [303]: arr = np.array([ .....: (1, 2.2, 0.0), .....: (3, 4.5, 0.0) .....: ], .....: dtype=[ .....: ('unit',int), .....: ('price',float), .....: ('amount',float), .....: ] .....: ) In [304]: data0 = arr.view(recarray) In [305]: data0.price[0] Out[305]: 2.2000000000000002 It works fine when I get a price vector and pick the first element of it. But if instead I select the first row and try to access its price attribute, it wouldn't work In [306]: data0[0].price --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) c:\Python26\Lib\site-packages\numpy\<ipython console> in <module>() AttributeError: 'numpy.void' object has no attribute 'price' Then I come across an alternative way to build a recarray. In that case both usage work fine. In [307]: data1 = np.rec.fromarrays( .....: [[1,3],[2.2,4.5],[0.0,0.0]], .....: names='unit,price,amount') In [309]: data1.price[0] Out[309]: 2.2000000000000002 In [310]: data1[0].price Out[310]: 2.2000000000000002 What's going on here? Wai Yip

2 1

Re: [Numpy-discussion] [PATCH] gfortran under macports
by Paul Anton Letnes 05 Dec '10

05 Dec '10

Mabe I am wrong somehow, but in my experience the easiest install of scipy is 'port install py26-scipy'. For new users, I do not see why one would recommend to build manually from source? Macports can do it for you, automagically... Paul 4. des.. 2010 15.04 "Ralf Gommers" <ralf.gommers(a)googlemail.com>: On Sat, Dec 4, 2010 at 9:47 PM, Fabian Pedregosa <fabian.pedregosa(a)inria.fr> wrote: > > On Sat, Dec ... I would prefer to just document the gcc_select solution, since it solves the problem at hand. > > > On the other hand, as installing on macports is not that trivial, I > strongly feel that a sub... The most up-to-date instructions are at http://www.scipy.org/Installing_SciPy/Mac_OS_X, so those should be updated as well. That said, if a "Macports" section is added there should be a strong disclaimer that it is *not* the recommended way to install numpy/scipy. A good portion of the build problems reported on these lists are related to Fortran on OS X, and a specific gfortran build at http://r.research.att.com/tools/ is recommended for a reason. If a user really wants to use Macports some notes in the docs may help, but let's not give the impression that it's a good/default option for a new user. Cheers, Ralf _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion(a)scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

3 2

ANN: NumPy 1.5.1
by Ralf Gommers 05 Dec '10

05 Dec '10

Hi, I am pleased to announce the availability of NumPy 1.5.1. This bug-fix release comes almost 3 months after the 1.5.0 release, it contains no new features compared to 1.5.0. Binaries, sources and release notes can be found at https://sourceforge.net/projects/numpy/files/. Thank you to everyone who contributed to this release. Enjoy, The numpy developers. ========================= NumPy 1.5.1 Release Notes ========================= Numpy 1.5.1 is a bug-fix release with no new features compared to 1.5.0. Numpy source code location changed ================================== Numpy has stopped using SVN as the version control system, and moved to Git. The development source code for Numpy can from now on be found at http://github.com/numpy/numpy Note on GCC versions ==================== On non-x86 platforms, Numpy can trigger a bug in the recent GCC compiler versions 4.5.0 and 4.5.1: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=45967 We recommend not using these versions of GCC for compiling Numpy on these platforms. Bugs fixed ========== Of the following, #1605 is important for Cython modules. - #937: linalg: lstsq should always return real residual - #1196: lib: fix negative indices in s_ and index_exp - #1287: core: fix uint64 -> Python int cast - #1491: core: richcompare should return Py_NotImplemented when undefined - #1517: lib: close file handles after use in numpy.lib.npyio.* - #1605: core: ensure PEP 3118 buffers can be released in exception handler - #1617: core: fix clongdouble cast to Python complex() - #1625: core: fix detection for ``isfinite`` routine - #1626: core: fix compilation with Solaris 10 / Sun Studio 12.1 Scipy could not be built against Numpy 1.5.0 on OS X due to a numpy.distutils bug, #1399. This issue is fixed now. - #1399: distutils: use C arch flags for Fortran compilation on OS X. Python 3 specific; #1610 is important for any I/O: - #----: f2py: make f2py script runnable on Python 3 - #1604: distutils: potential infinite loop in numpy.distutils - #1609: core: use accelerated BLAS, when available - #1610: core: ensure tofile and fromfile maintain file handle positions Checksums ========= b3db7d1ccfc3640b4c33b7911dbceabc release/installers/numpy-1.5.1-py2.5-python.org-macosx10.3.dmg 55f5863856485bbb005b77014edcd34a release/installers/numpy-1.5.1-py2.6-python.org-macosx10.3.dmg 420113e2a30712668445050a0f38e7a6 release/installers/numpy-1.5.1-py2.7-python.org-macosx10.3.dmg 757885ab8d64cf060ef629800da2e65c release/installers/numpy-1.5.1-py2.7-python.org-macosx10.5.dmg 11e60c3f7f3c86fcb5facf88c3981fd3 release/installers/numpy-1.5.1-win32-superpack-python2.5.exe 3fc14943dc2fcf740d8c204455e68aa7 release/installers/numpy-1.5.1-win32-superpack-python2.6.exe a352acce86c8b2cfb247e38339e27fd0 release/installers/numpy-1.5.1-win32-superpack-python2.7.exe 160de9794e4a239c9da1196a5eb30f7e release/installers/numpy-1.5.1-win32-superpack-python3.1.exe 376ef150df41b5353944ab742145352d release/installers/numpy-1.5.1.tar.gz ab6045070c0de5016fdf94dd2a79638b release/installers/numpy-1.5.1.zip

4 5

Faster than ndindex?
by Eleftherios Garyfallidis 05 Dec '10

05 Dec '10

Hi guys, I would like to know if there is any way to make the following operation faster. def test(): shape=(200,200,200,3) refinds = np.ndindex(shape[:3]) reftmp=np.zeros(shape) for ijk_t in refinds: i,j,k = ijk_t reftmp[i,j,k,0]=i reftmp[i,j,k,1]=j reftmp[i,j,k,2]=k %timeit test() 1 loops, best of 3: 19.5 s per loop I am using ndindex and then a for loop. Is there a better/faster way? Thank you, Eleftherios

2 2

Re: [Numpy-discussion] [PATCH] gfortran under macports
by Fabian Pedregosa 04 Dec '10

04 Dec '10

On Sat, Dec 4, 2010 at 10:29 AM, Gael Varoquaux <gael.varoquaux(a)normalesup.org> wrote: > On Sat, Dec 04, 2010 at 10:25:52AM +0100, Fabian Pedregosa wrote: >> The correct command is "sudo gcc_select mp-gcc45" which effectively >> does all the symbolic links for you and works like a charm, so please >> ignore my previous patch. > > I am not a mac user, so I guess that my opinion is not very educated, but > isn't your patch still useful: test if 'gcc' exists, and if not fallback > to your patch, so that it still works for the clueless user? Indeed, having scipy build out of the box would be nice, but it's not for me to decide if numpy.distutils should overcome these limitations in macports ... On the other hand, as installing on macports is not that trivial, I strongly feel that a subsection 'Macports' should be added to scipy's INSTALL.txt file, where it details needed packages, the gcc_select trick and options needed in site.cfg for umfpack. I'll gladly provide a patch for that if people are OK. Fabian. > > My 2 cents, > > Gaël > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion >

2 1