Mailman 3 February 2013 - NumPy-Discussion

Leaking memory problem
by Jaakko Luttinen 26 Feb '13

26 Feb '13

Hi! I was wondering if anyone could help me in finding a memory leak problem with NumPy. My project is quite massive and I haven't been able to construct a simple example which would reproduce the problem.. I have an iterative algorithm which should not increase the memory usage as the iteration progresses. However, after the first iteration, 1GB of memory is used and it steadily increases until at about 100-200 iterations 8GB is used and the program exits with MemoryError. I have a collection of objects which contain large arrays. In each iteration, the objects are updated in turns by re-computing the arrays they contain. The number of arrays and their sizes are constant (do not change during the iteration). So the memory usage should not increase, and I'm a bit confused, how can the program run out of memory if it can easily compute at least a few iterations.. I've tried to use Pympler, but I've understood that it doesn't show the memory usage of NumPy arrays.. ? I also tried gc.set_debug(gc.DEBUG_UNCOLLECTABLE) and then printing gc.garbage at each iteration, but that doesn't show anything. Does anyone have any ideas how to debug this kind of memory leak bug? And how to find out whether the bug is in my code, NumPy or elsewhere? Thanks for any help! Jaakko

5 5

Calling scipy blas from cython is extremely slow
by Sergio Callegari 24 Feb '13

24 Feb '13

Hi, following the excellent advice of V. Armando Sole, I have finally succeeded in calling the blas routines shipped with scipy from cython. I am doing this to avoid shipping an extra blas library for some project of mine that uses scipy but has some things coded in cython for extra speed. So far I managed getting things working on Linux. Here is what I do: The following code snippet gives me the dgemv pointer (which is a pointer to a fortran function, even if it comes from scipy.linalg.blas.cblas, weird). from cpython cimport PyCObject_AsVoidPtr import scipy as sp __import__('scipy.linalg.blas') ctypedef void (*dgemv_ptr) (char *trans, int *m, int *n,\ double *alpha, double *a, int *lda, double *x,\ int *incx,\ double *beta, double *y, int *incy) cdef dgemv_ptr dgemv=<dgemv_ptr>PyCObject_AsVoidPtr(\ sp.linalg.blas.cblas.dgemv._cpointer) Then, in a tight loop, I can call dgemv by first defining the constants and then calling dgemv inside the loop cdef int one=1 cdef double onedot = 1.0 cdef double zerodot = 0.0 cdef char trans = 'N' for i in xrange(N): dgemv(&trans, &nq, &order,\ &onedot, <double *>np.PyArray_DATA(C), &order, \ <double*>np.PyArray_DATA(c_x0), &one, \ &zerodot, <double*>np.PyArray_DATA(y0), &one) It works, but it is many many times slower than linking to the cblas that is available on the same system. Specifically, I have about 8 calls to blas in my tight loop, 4 of them are to dgemv and the others are to dcopy. Changing a single dgemv call from the system cblas to the blas function returned by scipy.linalg.blas.cblas.dgemv._cpointer makes the execution time of a test case jump from about 0.7 s to 1.25 on my system. Any clue about why is this happening? In the end, on linux, scipy dynamically link to atlas exactly as I link to atlas when I use the cblas functions.

3 4

Tabular package updated to v0.1
by Elaine Angelino 23 Feb '13

23 Feb '13

Hi there, We are writing to announce that we've updated "Tabular" to v0.1. Tabular is a package of Python modules for working with tabular data. Its main object is the tabarray class, a data structure for holding and manipulating tabular data. By putting data into a tabarray object, you’ll get a representation of the data that is more flexible and powerful than a native Python representation. More specifically, tabarray provides: -- ultra-fast filtering, selection, and numerical analysis methods, using convenient Matlab-style matrix operation syntax -- spreadsheet-style operations, including row & column operations, 'sort', 'replace', 'aggregate', 'pivot', and 'join' -- flexible load and save methods for a variety of file formats, including delimited text (CSV), binary, and HTML -- helpful inference algorithms for determining formatting parameters and data types of input files You can download Tabular from PyPI (http://pypi.python.org/pypi/tabular/) or alternatively clone our git repository from github (http://github.com/yamins81/tabular). We also have posted tutorial-style Sphinx documentation (http://web.mit.edu/yamins/www/tabular/). The tabarray object is based on the record array object from the Numerical Python package (NumPy), and Tabular is built to interface well with NumPy in general. Our intended audience is two-fold: (1) Python users who, though they may not be familiar with NumPy, are in need of a way to work with tabular data, and (2) NumPy users who would like to do spreadsheet-style operations on top of their more "numerical" work. We hope that some of you find Tabular useful! Best, Elaine and Dan

1 0

Smart way to do this?
by santhu kumar 23 Feb '13

23 Feb '13

Hi all, I dont want to run a loop for this but it should be possible using numpy "smart" ways. a = np.ones(30) idx = np.array([2,3,2]) # there is a duplicate index of 2 a += 2 >>>a array([ 1., 1., 3., 3., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.]) But if we do this : for i in range(idx.shape[0]): a[idx[i]] += 2 >>> a array([ 1., 1., 5., 3., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.]) How to achieve the second result without looping?? Thanks Santhosh

4 4

Windows, blas, atlas and dlls
by Sergio Callegari 22 Feb '13

22 Feb '13

Hi, I have a project that includes a cython script which in turn does some direct access to a couple of cblas functions. This is necessary, since some matrix multiplications need to be done inside a tight loop that gets called thousands of times. Speedup wrt calling scipy.linalg.blas.cblas routines is 10x to 20x. Now, all this is very nice on linux where the setup script can assure that the cython code gets linked with the atlas dynamic library, which is the same library that numpy and scipy link to on this platform. However, I now have trouble in providing easy ways to use my project in windows. All the free windows distros for scientific python that I have looked at (python(x,y) and winpython) seem to repackage the windows version of numpy/scipy as it is built in the numpy/scipy development sites. These appear to statically link atlas inside some pyd files. So I get no atlas to link against, and I have to ship an additional pre-built atlas with my project. All this seems somehow inconvenient. In the end, when my code runs, due to static linking I get 3 replicas of 2 slightly different atlas libs in memory. One coming with _dotblas.pyd in numpy, another one with cblas.pyd or fblas.pyd in scipy. And the last one as the one shipped in my code. Would it be possible to have a win distro of scipy which provides some pre built atlas dlls, and to have numpy and scipy dynamically link to them? This would save memory and also provide a decent blas to link to for things done in cython. But I believe there must be some problem since the scipy site says "IMPORTANT: NumPy and SciPy in Windows can currently only make use of CBLAS and LAPACK as static libraries - DLLs are not supported." Can someone please explain why or link to an explanation? Unfortunately, not having a good, prebuilt and cheap blas implementation in windows is really striking me as a severe limitation, since you loose the ability to prototype in python/scipy and then move to C or Cython the major bottlenecks to achieve speed. Many thanks in advance!

12 25

numpy.einsum bug?
by Jaakko Luttinen 22 Feb '13

22 Feb '13

Hi, Is this a bug in numpy.einsum? >>> np.einsum(3, [], 2, [], []) ValueError: If 'op_axes' or 'itershape' is not NULL in theiterator constructor, 'oa_ndim' must be greater than zero I think it should return 6 (i.e., 3*2). Regards, Jaakko

1 0

Seeking help and support for next-gen math modeling tools using Python
by Rob Clewley 20 Feb '13

20 Feb '13

Hi all, and apologies for a little cross-posting: First, thanks to those of you who have used and contributed to the PyDSTool math modeling environment [1]. This project has greatly benefitted from the underlying platform of numpy / scipy / matplotlib / ipython. Going forward I have three goals, for which I would like the urgent input and support of existing or potential users. (i) I have ideas for expanding PyDSTool with innovative tools in my research area, which is essentially for the reverse engineering of complex mechanisms in multi-scale dynamic systems [2]. These tools have already been prototyped and show promise, but they need a lot of work. (ii) I want to grow and develop the community of users who will help drive new ideas, provide feedback, and collaborate on writing and testing code for both the core and application aspects of PyDSTool. (iii) The first two goals will help me to expand the scientific / engineering applications and use cases of PyDSTool as well as further sustain the project in the long-term. I am applying for NSF funding to support these software and application goals over the next few years [3], but the proposal deadline is in just four weeks! If you are interested in helping in any way I would greatly appreciate your replies (off list) to either of the following queries: I need to better understand my existing and potential users, many of whom may not be registered on the sourceforge users list. Please tell me who you are and what you use PyDSTool for. If you are not using it yet but you’re interested in this area then please provide feedback regarding what you would like to see change. If you are interested in these future goals, even if you are not an existing user but may be in the future, please write a brief letter of support on a letterhead document that I will send in with the proposal as PDFs. I have sample text that I can send you, as well as my draft proposal’s introduction and specific aims. These letters can make a great deal of difference during review. Without funding, collaborators, user demand and community support, these more ambitious goals for PyDSTool will not happen, although I am committed to a basic level of maintenance. For instance, based on user feedback I am about to release an Ubuntu-based Live CD [4] that will allow users to try PyDSTool on any OS without having to install it. PyDSTool will also acquire an improved setup procedure and will be added to the NeuroDebian repository [5], among others. I am also finalizing an integrated interface to CUDA GPUs to perform fast parallel ODE solving [6]. Thanks for your time, Rob Clewley [1] http://pydstool.sourceforge.net [2] http://www.ni.gsu.edu/~rclewley/Research/index.html, and in particular http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.100… [3] NSF Software Infrastructure for Sustained Innovation (SI2-SSE) program solicitation: http://www.nsf.gov/pubs/2013/nsf13525/nsf13525.htm [4] http://help.ubuntu.com/community/LiveCD [5] http://neuro.debian.net/ [6] http://www.nvidia.com/object/cuda_home_new.html -- Robert Clewley, Ph.D. Assistant Professor Neuroscience Institute and Department of Mathematics and Statistics Georgia State University PO Box 5030 Atlanta, GA 30302, USA tel: 404-413-6420 fax: 404-413-5446 http://neuroscience.gsu.edu/rclewley.html

4 4

pull request: generalized ufunc signature fix and lineal algebra generalized ufuncs
by Oscar Villellas 20 Feb '13

20 Feb '13

Hello, At Continuum Analytics we've been working on a submodule implementing a set of lineal algebra operations as generalized ufuncs. This allows specifying arrays of lineal algebra problems to be computed with a single Python call, allowing broadcasting as well. As the vectorization is handled in the kernel, this gives a speed edge on the operations. We think this could be useful to the community and we want to share the work done. I've created a couple of pull-requests: The first one contains a fix for a bug in the handling of certain signatures in the gufuncs. This was found while building the submodule. The fix was done by Mark Wiebe, so credit should go to him :). https://github.com/numpy/numpy/pull/2953 The second pull request contains the submodule itself and builds on top of the previous fix. It contains a rst file that explains the submodule, enumerates the functions implemented and details some implementation bits. The entry point to the module is in written in Python and contains detailed docstrings. https://github.com/numpy/numpy/pull/2954 We are open to discussion and to make improvements to the code if needed, in order to adapt to NumPy standards. Thanks, Oscar.

3 4

savetxt trouble
by Neal Becker 20 Feb '13

20 Feb '13

I tried to save a vector as a csv, but it didn't work. The vector is: a[0,0] array([-0.70710678-0.70710678j, 0.70710678+0.70710678j, 0.70710678-0.70710678j, 0.70710678+0.70710678j, -0.70710678-0.70710678j, 0.70710678-0.70710678j, -0.70710678+0.70710678j, -0.70710678+0.70710678j, -0.70710678-0.70710678j, -0.70710678-0.70710678j, 0.70710678-0.70710678j, -0.70710678-0.70710678j, -0.70710678+0.70710678j, 0.70710678-0.70710678j, 0.70710678-0.70710678j, 0.70710678+0.70710678j, 0.70710678-0.70710678j, -0.70710678-0.70710678j, -0.70710678-0.70710678j, 0.70710678-0.70710678j, 0.70710678+0.70710678j, 0.70710678+0.70710678j, -0.70710678+0.70710678j, 0.70710678+0.70710678j, -0.70710678-0.70710678j, -0.70710678+0.70710678j, 0.70710678-0.70710678j, -0.70710678+0.70710678j, 0.70710678+0.70710678j, 0.70710678+0.70710678j, 0.70710678-0.70710678j, -0.70710678-0.70710678j, 0.70710678-0.70710678j, -0.70710678-0.70710678j, -0.70710678+0.70710678j, 0.70710678+0.70710678j, 0.70710678-0.70710678j, 0.70710678-0.70710678j, 0.70710678+0.70710678j, -0.70710678+0.70710678j]) np.savetxt ('test.out', a[0,0], delimiter=',') The saved txt file says: (-7.071067811865540120e-01+-7.071067811865411334e-01j) (7.071067811865535679e-01+7.071067811865415775e-01j) (7.071067811865422437e-01+-7.071067811865529018e-01j) (7.071067811865520136e-01+7.071067811865431318e-01j) ...

3 3

Array accumulation in numpy
by Tony Ladd 19 Feb '13

19 Feb '13

I want to accumulate elements of a vector (x) to an array (f) based on an index list (ind). For example: x=[1,2,3,4,5,6] ind=[1,3,9,3,4,1] f=np.zeros(10) What I want would be produced by the loop for i=range(6): f[ind[i]]=f[ind[i]]+x[i] The answer is f=array([ 0., 7., 0., 6., 5., 0., 0., 0., 0., 3.]) When I try to use implicit arguments f[ind]=f[ind]+x I get f=array([ 0., 6., 0., 4., 5., 0., 0., 0., 0., 3.]) So it takes the last value of x that is pointed to by ind and adds it to f, but its the wrong answer when there are repeats of the same entry in ind (e.g. 3 or 1) I realize my code is incorrect, but is there a way to make numpy accumulate without using loops? I would have thought so but I cannot find anything in the documentation. Would much appreciate any help - probably a really simple question. Thanks Tony -- Tony Ladd Chemical Engineering Department University of Florida Gainesville, Florida 32611-6005 USA Email: tladd-"(AT)"-che.ufl.edu Web http://ladd.che.ufl.edu Tel: (352)-392-6509 FAX: (352)-392-9514

4 4

Re: [Numpy-discussion] Array accumulation in numpy
by Alan G Isaac 19 Feb '13

19 Feb '13

x=[1,2,3,4,5,6] ind=[1,3,9,3,4,1] f=np.zeros(10) np.bincount(ind,x) hth, Alan Isaac

2 1

Suggestion for a speed improved accumarray receipe
by Michael Löffler 18 Feb '13

18 Feb '13

Hello numpy list! I had tried to contribute to the AccumarrayLike recipe cookbook, but access seems to be restricted and no registration allowed. The current recipe mimics the matlab version in most aspects, but concerning performance it's horribly slow. The following snippet handles only the most simple usecase, but gives about 16x speed improvement compared to the current receipe, so it would probably worth to also mention it in the cookbook: def accum_custom(accmap, a, func=np.sum): indices = np.where(np.ediff1d(accmap, to_begin=[1], to_end=[np.nan]))[0] vals = np.zeros(len(indices) - 1) for i in xrange(len(indices) - 1): vals[i] = func(a[indices[i]:indices[i+1]]) return vals accmap = np.repeat(np.arange(100000), 20) a = np.random.randn(accmap.size) %timeit accum(accmap, a, func=np.sum) >>> 1 loops, best of 3: 16.7 s per loop %timeit accum_custom(accmap, a, func=np.sum) >>> 1 loops, best of 3: 945 ms per loop Best regards, Michael

1 0

calling NumPy from Julia - a plea for fewer macros
by Steven G. Johnson 17 Feb '13

17 Feb '13

Dear NumPy developers, I've been working on a glue package that allows the Julia language (http://julialang.org/) to call Python routines easily https://github.com/stevengj/PyCall.jl and I'm using NumPy to pass multidimensional arrays between languages. Julia has the ability to call C functions directly (without writing C glue), and I've been exploiting this to write PyCall purely in Julia. (This is nice for a number of reasons; besides programming and linking convenience, it means that I can dynamically load different Python versions on the same machine, and don't need to recompile if e.g. NumPy is updated.) However, calling NumPy has been a challenge, because of NumPy's heavy reliance on macros in its C API. I wanted to make a couple of suggestions to keep in mind as you plan for NumPy 2.0: 1) Dynamically linking to NumPy's C API was challenging, to say the least. Assuming you stick with the PyArray_API lookup table of pointers, it would be much easier to call from other languages if you include e.g. a numpy.core.multiarray._ARRAY_API_NAMES variable in the Python module that is a list of strings giving the symbol names corresponding to the numpy.core.multiarray._ARRAY_API pointer. (Plus documentation, of course.) Currently I need to parse __multiarray_api.h to extract this information, which is somewhat hackish. 2) Please provide non-macro equivalents (exported in the _ARRAY_API symbol table or otherwise) of PyArray_NDIM etcetera to access PyArrayObject members. (e.g. call them PyArray_ndim etc. Note that inline functions are not enough, since they are not loadable dynamically.) Right now, the only ways[*] I can see to access this information are either to use C glue (which I want to avoid for the reasons above) or to call Python to access the __array_interface__ attribute (which is suboptimal from a performance standpoint). Thanks for all your efforts! Any feedback on PyCall would be welcome, too. --SGJ [*] A third way would be to parse ndarraytypes.h to extract the format of the PyArrayObject_fields structure, and use upcoming Julia support for accessing C struct types to read the fields. This is likely to require tracking NumPy releases carefully to avoid breakage, however, as well as involving some care with the PyObject_HEAD macro. PS. If you want to try out PyCall with NumPy, note that a patch to Julia is currently required for this to work: https://github.com/JuliaLang/julia/pull/2317

3 4

FFTW bindings now implement numpy.fft interface
by Henry Gomersall 17 Feb '13

17 Feb '13

Some of you may be interested in the latest release of my FFTW bindings. It can now serve as a drop in replacement* for numpy.fft and scipy.fftpack. This means you can get most of the speed-up of FFTW with a one line code change or monkey patch existing libraries. Lots of other goodness too of course. Source here: https://github.com/hgomersall/pyFFTW pypi here: http://pypi.python.org/pypi/pyFFTW docs here: http://hgomersall.github.com/pyFFTW/ It's GPL3 due to license restrictions on FFTW. Get in touch if you want a different license and I'm sure we can reach an agreement ;) Cheers, Henry *In the case where the input array is not a numpy array, it doesn't work. This was an oversight and will be fixed in the next release. That said, if you're converting from a list on every transform, you have better optimisations than using FFTW. One other small caveat in a corner case to do with repeated axes - described in the docs.

2 2

np.percentile docstring
by Andreas Hilboll 17 Feb '13

17 Feb '13

In my numpy 1.6.1 (from Ubuntu 12.04LTS repository), the docstring of np.percentile is wrong. I'm not just submitting a PR because I don't understand something. in the "Notes" and "Examples" sections, there seems to be some confusion if ``q`` should be in the range [0,1] or in [0,100]. The parameters section is very clear about this, it should be [0,100]. However, in the Examples section, it says >>> np.percentile(a, 0.5, axis=0) array([ 6.5, 4.5, 2.5]) The given array is clearly the result of a call to np.percentile(a, 50., axis=0). I thought the docs are auto-generated, and that the "array..." result of the docstring would be calculated by numpy while building the docs? Or am I misunderstanding something here? Cheers, Andreas.

2 1

A new webpage promoting Compiler technology for CPython
by Travis Oliphant 16 Feb '13

16 Feb '13

Hey all, With Numba and Blaze we have been doing a lot of work on what essentially is compiler technology and realizing more and more that we are treading on ground that has been plowed before with many other projects. So, we wanted to create a web-site and perhaps even a mailing list or forum where people could coordinate and communicate about compiler projects, compiler tools, and ways to share efforts and ideas. The website is: http://compilers.pydata.org/ This page is specifically for Compiler projects that either integrate with or work directly with the CPython run-time which is why PyPy is not presently listed. The PyPy project is a great project but we just felt that we wanted to explicitly create a collection of links to compilation projects that are accessible from CPython which are likely less well known. But that is just where we started from. The website is intended to be a community website constructed from a github repository. So, we welcome pull requests from anyone who would like to see the website updated to reflect their related project. Jon Riehl (Mython, PyFront, ROFL, and many other interesting projects) and Stephen Diehl (Blaze) and I will be moderating the pull requests to begin with. But, we welcome others with similar interests to participate in that effort of moderation. The github repository is here: https://github.com/pydata/compilers-webpage This is intended to be a community website for information spreading, and so we welcome any and all contributions. Thank you, Travis Oliphant

5 8

Re: [Numpy-discussion] A proposed change to rollaxis() behavior for negative 'start' values
by Thomas Coffee 16 Feb '13

16 Feb '13

Adding to an old discussion thread (see below) ... an implementation of the proposed functionality: from numpy import rollaxis def moveaxis(a, i, j = 0): """ move axis i of array a to position j """ n = a.ndim i = i if i >= 0 else i + n if j > i: return rollaxis(a, i, j + 1) elif j >= 0: return rollaxis(a, i, j) elif j == -1: return rollaxis(a, i, n) elif j >= -i: return rollaxis(a, i, j + 1) else: return rollaxis(a, i, j) Examples: In [464]: a = numpy.ones((3,4,5,6)) In [465]: moveaxis(a,2,0).shape Out[465]: (5, 3, 4, 6) In [466]: moveaxis(a,2,1).shape Out[466]: (3, 5, 4, 6) In [467]: moveaxis(a,2,2).shape Out[467]: (3, 4, 5, 6) In [468]: moveaxis(a,2,3).shape Out[468]: (3, 4, 6, 5) In [469]: moveaxis(a,2,4).shape --------------------------------------------------------------------------- ValueError: rollaxis: start (5) must be >=0 and < 5 In [470]: moveaxis(a,2,-1).shape Out[470]: (3, 4, 6, 5) In [471]: moveaxis(a,2,-2).shape Out[471]: (3, 4, 5, 6) In [472]: moveaxis(a,2,-3).shape Out[472]: (3, 5, 4, 6) In [473]: moveaxis(a,2,-4).shape Out[473]: (5, 3, 4, 6) In [474]: moveaxis(a,2,-5).shape --------------------------------------------------------------------------- ValueError: rollaxis: start (-1) must be >=0 and < 5 On Thu, Sep 23, 2010 at 11:33 AM, Nathaniel Smith wrote: > > On Tue, Sep 21, 2010 at 12:48 PM, Ken Basye <kbasye1(a)jhu.edu> wrote: > > If that's going to break too much code, here's a pathway that might be > > acceptable: Add a new function moveaxis() which works the way > > rollaxis() does for positive arguments but in the new way for negative > > arguments. Eventually, rollaxis could be deprecated to keep things > > tidy. This has the added advantage of using a name that seems to fit > > what the function does better - 'rollaxis' suggests a behavior like the > > roll() function which affects other axes, which isn't what happens. > > My 2 cents: +1 on a new function, but I'd change the behavior for > positive arguments too. > > Currently, the API is (AFAICT): You give the index of the axis you > want to move, and you give the index of the axis that you want the > first axis to be moved in front of. This is super confusing! > > I propose that a much better API would be: You give the index of the > axis you want to move, and you give the index you *want* that axis to > have. So we'd have the invariant: > b = np.moveaxis(a, i, j) > assert a.shape[i] == b.shape[j] > This is way easier to think about, at least for me. And it solves the > problem with negative indices too. > > BTW, note that that the documentation for rollaxis is actually > self-contradictory at the moment: > http://docs.scipy.org/doc/numpy/reference/generated/numpy.rollaxis.html > At the top it seems to document the behavior that I propose ("Roll the > specified axis backwards, until it lies *in a given* position."), and > then in the details it describes the actual behavior("The axis is > rolled until it lies *before* this position"). I take this as further > evidence that the current behavior is unnatural and confusing :-). > > -- Nathaniel

2 2

NumPy 1.7.0 for MeeGo Harmattan OS
by Roberto Colistete Jr. 16 Feb '13

16 Feb '13

Hi, It is my first participation here. About NumPy on Mobile OS : - NumPy 1.7.0 was released today (11/02/2013) for MeeGo Harmattan OS (for Nokia N9/N950), just 1 day after the mainstream release. See the Talk Maemo.org topic : http://talk.maemo.org/showthread.php?p=1322503 MeeGo Harmattan OS also has NumPy 1.4.1 from Nokia repositories : http://wiki.maemo.org/Python/Harmattan - Maemo 5 OS (Nokia N900) has NumPy 1.4.0 : http://maemo.org/packages/view/python-numpy/ Also for MeeGo Harmattan OS : - MatPlotLib 1.2.0 was released (in 09/02/2013) for MeeGo Harmattan OS. Including Qt4/PySide backend : http://talk.maemo.org/showthread.php?p=1128672 - IPython 0.13.1, including Notebook and Qt console interfaces, was released in 22/01/2013. See the Talk Maemo.org topic : http://talk.maemo.org/showthread.php?p=1123672 It can work as an IPython Notebook server , with web clients running on Android, desktop OS, etc via via WiFi hotspot of Nokia N9/N950. See my blog article comparing scientific Python tools for computers, tablets and smartphones : http://translate.google.com/translate?hl=pt-BR&sl=pt&tl=en&u=http%3A%2F%2Fr… The conclusion is very simple : real mobile Linux OS (with glibc, X11, dependencies, etc) are better for scientific Python. Like Maemo 5 OS and MeeGo Harmattan OS. And future Sailfish OS and Ubuntu Phone OS can follow the same path. Best regards, Roberto Colistete Jr.

2 1

confused about tensordot
by Neal Becker 15 Feb '13

15 Feb '13

In the following code c = np.multiply (a, b.conj()) d = np.abs (np.sum (c, axis=0)/rows) d2 = np.abs (np.tensordot (a, b.conj(), ((0,),(0,)))/rows) print a.shape, b.shape, d.shape, d2.shape The 1st compute steps, where I do multiply and then sum give the result I want. I wanted to try to combine these 2 steps using tensordot, but it's not doing what I want. The print statement outputs this: (1004, 13) (1004, 13) (13,) (13, 13) The correct output should be (13,), but the tensordot output is (13,13). It's supposed to take 2 matrixes, each (1004, 13) and do element-wise multiply, then sum over axis 0.

2 3

Purpose of this list
by Juan Luis Cano 15 Feb '13

15 Feb '13

Hello all, I have a brief question about the general purpose of Numpy-discussion. I suscribed a month ago more or less to keep an eye on NumPy development (and perhaps contributing eventually), for I saw that some decisions about the roadmap were discussed here, but 90 % of the emails that arrive through the list are related to support. Whats is actually the scope/purpose of the list? I am not critizicing anyone, just asking for a clarification on this point, to manage my mail client filters etc. Best regards, Juan Luis Cano

2 2

SciPy 2013 Announcement -- June 24 - 29, 2013, Austin, TX!
by Anthony Scopatz 15 Feb '13

15 Feb '13

Hello All, As this years conference communications co-chair, it is my extreme pleasure to announce the *SciPy 2013 conference from June 24th - 29th in sunny Austin, Texas, USA.* Please see our website<http://conference.scipy.org/scipy2013/about.php>(or below) for more details. A call for presentations, tutorials, and papers will be coming out very soon. Please make sure to sign up on our mailing list<http://eepurl.com/dyoeL>, follow us on twitter <https://twitter.com/SciPy2013>, or Google Plus<https://plus.google.com/u/0/100948873231627513165/posts> so that you don't miss out on any important updates. I sincerely hope that you can attend this year and make 2013 the best year for SciPy yet! Happy Hacking! Anthony Scopatz SciPy 2013 Conference Announcement ---------------------------------- SciPy 2013, the twelfth annual Scientific Computing with Python conference, will be held this June 24th-29th in Austin, Texas. SciPy is a community dedicated to the advancement of scientific computing through open source Python software for mathematics, science, and engineering. The annual SciPy Conference allows participants from academic, commercial, and governmental organizations to showcase their latest projects, learn from skilled users and developers, and collaborate on code development. The conference consists of two days of tutorials by followed by two days of presentations, and concludes with two days of developer sprints on projects of interest to the attendees. Specialized Tracks ------------------ This year we are happy to announce two specialized tracks run in parallel to the general conference: *Machine Learning* In recent years, Python's machine learning libraries rapidly matured with a flurry of new libraries and cutting-edge algorithm implement and development occurring within Python. As Python makes these algorithms more accessible, machine learning algorithm application has spread across disciplines. Showcase your favorite machine learning library or how it has been used as an effective tool in your work! *Reproducible Science* Over recent years, the Open Science movement has stoked a renewed acknowledgement of the importance of reproducible research. The goals of this movement include improving the dissemination of progress, prevent fraud through transparency, and enable deeper/wider development and collaboration. This track is to discuss the tools and methods used to achieve reproducible scientific computing. Domain-specific Mini-symposia ----------------------------- Introduced in 2012, mini-symposia are held to discuss scientific computing applied to a specific scientific domain/industry during a half afternoon after the general conference. Their goal is to promote industry specific libraries and tools, and gather people with similar interests for discussions. Mini-symposia on the following topics will take place this year: - Meteorology, climatology, and atmospheric and oceanic science - Astronomy and astrophysics - Medical imaging - Bio-informatics Tutorials --------- Multiple interactive half-day tutorials will be taught by community experts. The tutorials provide conceptual and practical coverage of tools that have broad interest at both an introductory or advanced level. This year, a third track will be added, targeting specifically programmers with no prior knowledge of scientific python. Developer Sprints ----------------- A hackathon environment is setup for attendees to work on the core SciPy packages or their own personal projects. The conference is an opportunity for developers that are usually physically separated to come together and engage in highly productive sessions. It is also an occasion for new community members to introduce themselves and recieve tips from community experts. This year, some of the sprints will be scheduled and announced ahead of the conference. Birds-of-a-Feather (BOF) Sessions --------------------------------- Birds-of-a-Feather sessions are self-organized discussions that run parallel to the main conference. The BOFs sessions cover primary, tangential, or unrelated topics in an interactive, discussion setting. This year, some of the BOF sessions will be scheduled and announced ahead of the conference. Important Dates --------------- - March 18th: Presentation abstracts, poster, tutorial submission deadline. Application for sponsorship deadline. - April 15th: Speakers selected - April 22nd: Sponsorship acceptance deadline - May 1st: Speaker schedule announced - May 5th: Paper submission deadline - May 6th: Early-bird registration ends - June 24th-29th: 2 days of tutorials, 2 days of conference, 2 days of sprints We look forward to a very exciting conference and hope to see you all at the conference. The SciPy2013 organization team: * Andy Terrel, Co-chair * Jonathan Rocher, Co-chair * Katy Huff, Program Committee co-chair * Matt McCormick, Program Committee co-chair * Dharhas Pothina, Tutorial co-chair * Francesc Alted, Tutorial co-chair * Corran Webster, Sprint co-chair * Peter Wang, Sprint co-chair * Matthew Turk, BoF co-chair * Jarrod Millman, Proceeding co-chair * Stéfan van der Walt, Proceeding co-chair * Anthony Scopatz, Communications co-chair * Majken Tranby, Communications co-chair * Jeff Daily, Financial Aid co-chair * John Wiggins, Financial Aid co-chair * Leah Jones, Operations chair * Brett Murphy, Sponsor chair * Bill Cowan, Financial chair

1 0

Trouble building numpy on different version of OSX.
by Steve Spicklemire 14 Feb '13

14 Feb '13

Hi Numpy Folks! When I try to build numpy on MacOSX 10.6 with Xcode 3.2.5 installed (python3.2 setup.py build) things go great! At some point I get this: -------------------- Generating build/src.macosx-10.6-intel-3.2/numpy/core/include/numpy/config.h C compiler: gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -O3 -isysroot /Developer/SDKs/MacOSX10.6.sdk -arch i386 -arch x86_64 -isysroot /Developer/SDKs/MacOSX10.6.sdk compile options: '-Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/Library/Frameworks/Python.framework/Versions/3.2/include/python3.2m -c' gcc-4.2: _configtest.c success! removing: _configtest.c _configtest.o C compiler: gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -O3 -isysroot /Developer/SDKs/MacOSX10.6.sdk -arch i386 -arch x86_64 -isysroot /Developer/SDKs/MacOSX10.6.sdk -------------------- Keeping everything as nearly as possible the same on MacOSX 10.7 with Xcode 4.6 installed I get this: -------------------- Generating build/src.macosx-10.6-intel-3.2/numpy/core/include/numpy/config.h C compiler: gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -O3 -isysroot /Developer/SDKs/MacOSX10.6.sdk -arch i386 -arch x86_64 -isysroot /Developer/SDKs/MacOSX10.6.sdk compile options: '-Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -c' gcc-4.2: _configtest.c _configtest.c:1:20: error: Python.h: No such file or directory _configtest.c:1:20: error: Python.h: No such file or directory lipo: can't figure out the architecture type of: /var/folders/4h/7kcqgdb55yjdtfs6dpwjytjh0000gn/T//ccIEwAT5.out _configtest.c:1:20: error: Python.h: No such file or directory _configtest.c:1:20: error: Python.h: No such file or directory lipo: can't figure out the architecture type of: /var/folders/4h/7kcqgdb55yjdtfs6dpwjytjh0000gn/T//ccIEwAT5.out failure. removing: _configtest.c _configtest.o -------------------- Obviously the -I/Library/Frameworks/etc... is missing. I get the same thing with Xcode 4.6 on 10.8. ;-(. I can *run* my numpy build from 10.6 on 10.7 and 10.8, but I'd really like to be able to build it without having to reboot from an old backup disk. ;-) For what it's worth, on 10.7 and 10.8 (and 10.6 for that matter) python3.2-config works and returns reasonable results. Where does setup.py decide about which paths to include in the compile options string? thanks! -steve

4 8

Python trademark in legal trouble in Europe, please help
by Fernando Perez 14 Feb '13

14 Feb '13

Hi all, please do NOT respond to this thread or to me directly. This is strictly to spread this message as widely as possible, so that anyone who receives it and can act on it does so. Needless to say, do forward this to anyone you think might be in a position to take useful action. The Python trademark is in serious risk across Europe due to the actions of a UK-based IP troll. If you operate in Europe, please help the Python Software Foundation gather evidence in the legal battle to protect the Python trademark across the EU. You can find the official blog post from the PSF with instructions on how to help here: http://pyfound.blogspot.com/2013/02/python-trademark-at-risk-in-europe-we.h… and this thread on Hacker News is being monitored by the PSF Chairman Van Lindberg in case you want to ask the team directly any questions: http://news.ycombinator.com/item?id=5221093 Cheers, f

1 0

How to upload to pypi
by Ondřej Čertík 14 Feb '13

14 Feb '13

Hi, I have uploaded the NumPy 1.7.0 source distribution to pypi: http://pypi.python.org/pypi/numpy/1.7.0 I did it by uploading the file PKG-INFO from numpy-1.7.0.tar.gz. It said "Error processing form. Form Failure; reset form submission" about 3x times, but on the 4th try it went through. I reported the issue here: https://sourceforge.net/tracker/?func=detail&aid=3604194&group_id=66150&ati… I then attached the numpy-1.7.0.tar.gz and numpy-1.7.0.zip source files. Now I am having trouble attaching the windows installers, just like they are here: http://pypi.python.org/pypi/numpy/1.6.2 but whenever I upload the file numpy-1.7.0-win32-superpack-python2.5.exe (and set it as "MS Windows installer"), it uploads and then I get a blank page with the text: """ Error processing form invalid distribution file """ That's it.... Not very useful. Do you know if the sources of pypi are somewhere online? (I didn't find them, only a similar package https://github.com/schmir/pypiserver, but that doesn't seem to be it.) Anyway, so I at least reported the bug here: http://sourceforge.net/tracker/?func=detail&aid=3604193&group_id=66150&atid… Finally, should we try to upload the Windows installers to pypi at all? I am probably doing something wrong if I am hitting these pypi bugs. Many thanks, Ondrej

3 5

Fwd: Package: scipy-0.11.0-0.1.rc2.fc18 Tag: f18-updates-candidate Status: failed Built by: orion
by Orion Poplawski 13 Feb '13

13 Feb '13

This is a plea for some help. We've been having trouble getting scipy to pass all of the tests in the Fedora 18 build with python 3.3 (although it seems to build okay in Fedora 19). Below are the logs of the build. There appears to be some kind of memory corruption that manifests itself a little differently on 32-bit vs. 64-bit. I really have no idea myself how to pursue debugging this, though I'm happy to provide any more needed information. Thanks, Orion -------- Original Message -------- Subject: Package: scipy-0.11.0-0.1.rc2.fc18 Tag: f18-updates-candidate Status: failed Built by: orion Date: Thu, 20 Sep 2012 21:01:54 +0000 (UTC) From: Fedora Koji Build System <buildsys(a)fedoraproject.org> To: ausil(a)fedoraproject.org, jspaleta(a)fedoraproject.org, voronov(a)fedoraproject.org, torwangjl(a)fedoraproject.org, alagunambi(a)fedoraproject.org, urkle(a)fedoraproject.org, orion(a)fedoraproject.org Package: scipy-0.11.0-0.1.rc2.fc18 Tag: f18-updates-candidate Status: failed Built by: orion ID: 350761 Started: Thu, 20 Sep 2012 20:39:45 UTC Finished: Thu, 20 Sep 2012 21:01:32 UTC scipy-0.11.0-0.1.rc2.fc18 (350761) failed on buildvm-33.phx2.fedoraproject.org (x86_64), buildvm-35.phx2.fedoraproject.org (i386), buildvm-34.phx2.fedoraproject.org (noarch): BuildError: error building package (arch x86_64), mock exited with status 1; see build.log for more information SRPMS: scipy-0.11.0-0.1.rc2.fc18.src.rpm Failed tasks: ------------- Task 4509076 on buildvm-33.phx2.fedoraproject.org Task Type: buildArch (scipy-0.11.0-0.1.rc2.fc18.src.rpm, x86_64) logs: http://koji.fedoraproject.org/koji/getfile?taskID=4509076&name=build.log http://koji.fedoraproject.org/koji/getfile?taskID=4509076&name=mock_output.… http://koji.fedoraproject.org/koji/getfile?taskID=4509076&name=root.log http://koji.fedoraproject.org/koji/getfile?taskID=4509076&name=state.log Task 4509077 on buildvm-35.phx2.fedoraproject.org Task Type: buildArch (scipy-0.11.0-0.1.rc2.fc18.src.rpm, i686) logs: http://koji.fedoraproject.org/koji/getfile?taskID=4509077&name=build.log http://koji.fedoraproject.org/koji/getfile?taskID=4509077&name=mock_output.… http://koji.fedoraproject.org/koji/getfile?taskID=4509077&name=root.log http://koji.fedoraproject.org/koji/getfile?taskID=4509077&name=state.log Task 4509063 on buildvm-34.phx2.fedoraproject.org Task Type: build (f18-candidate, /scipy:cb69bd06f0d930fbe8840d89b918b617e28af63f) Closed tasks: ------------- Task 4509064 on buildvm-30.phx2.fedoraproject.org Task Type: buildSRPMFromSCM (/scipy:cb69bd06f0d930fbe8840d89b918b617e28af63f) logs: http://koji.fedoraproject.org/koji/getfile?taskID=4509064&name=build.log http://koji.fedoraproject.org/koji/getfile?taskID=4509064&name=checkout.log http://koji.fedoraproject.org/koji/getfile?taskID=4509064&name=mock_output.… http://koji.fedoraproject.org/koji/getfile?taskID=4509064&name=root.log http://koji.fedoraproject.org/koji/getfile?taskID=4509064&name=state.log Task Info: http://koji.fedoraproject.org/koji/taskinfo?taskID=4509063 Build Info: http://koji.fedoraproject.org/koji/buildinfo?buildID=350761

4 8