Mailman 3 January 2012 - NumPy-Discussion

Re: [Numpy-discussion] Counting the Colors of RGB-Image
by apo＠pdauf.de 23 Jan '12

23 Jan '12

Sorry, that i use this way to send an answer to Tony Yu , Nadav Horesh , Chris Barker. When iam direct answering on Your e-mail i get an error 5. I think i did a mistake. Your ideas are very helpfull and the code is very fast. Thank You elodw

4 3

Easy module installation with less human intervention.
by Peng Yu 21 Jan '12

21 Jan '12

Hi, Perl has something like ppm so that I can just use one command to download and install perl modules. But I don't find such thing in python. As shown on http://docs.python.org/install/index.html, it seems that I have to download the packages first unzip it then install it. I'm wondering if there is a better way to install python packages that require less human intervention. Thanks! NAME ppm - Perl Package Manager, version 4.14 SYNOPSIS Invoke the graphical user interface: ppm ppm gui Install, upgrade and remove packages: ppm install [--area <area>] [--force] <pkg> ppm install [--area <area>] [--force] <module> ppm install [--area <area>] <url> ppm install [--area <area>] <file>.ppmx ppm install [--area <area>] <file>.ppd ppm install [--area <area>] <num> ppm upgrade [--install] ppm upgrade <pkg> ppm upgrade <module> ppm remove [--area <area>] [--force] <pkg> -- Regards, Peng

2 1

views and mask NA
by Charles R Harris 21 Jan '12

21 Jan '12

Hi All, I'd like some feedback on how mask NA should interact with views. The immediate problem is how to deal with the real and imaginary parts of complex numbers. If the original has a masked value, it should show up as masked in the real and imaginary parts. But what should happen on assignment to one of the masked views? This should probably clear the NA in the real/imag part, but not in the complex original. However, that does allow touching things under the mask, so to speak. Things get more complicated if the complex original is viewed as reals. In this case the mask needs to be "doubled" up, and there is again the possibility of touching things beneath the mask in the original. Viewing the original as bytes leads to even greater duplication. My thought is that touching the underlying data needs to be allowed in these cases, but the original mask can only be cleared by assignment to the original. Thoughts? Chuck

1 0

Upgrade to 1.6.x: frompyfunc() ufunc casting issue
by Aditya Sethi 20 Jan '12

20 Jan '12

Hi all, I am facing an issue upgrading numpy from 1.5.1 to 1.6.1. In numPy 1.6, the casting behaviour for ufunc has changed and has become stricter. Can someone advise how to implement the below simple example which worked in 1.5.1 but fails in 1.6.1? >>> import numpy as np >>> def add(a,b): ... return (a+b) >>> uadd = np.frompyfunc(add,2,1) >>> uadd <ufunc 'add (vectorized)'> >>> uadd.accumulate([1,2,3]) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: could not find a matching type for add (vectorized).accumulate, requested type has type code 'l' Thanks, Aditya

4 7

Cross-covariance function
by Elliot Saba 20 Jan '12

20 Jan '12

Greetings, I recently needed to calculate the cross-covariance of two random vectors, (e.g. I have two matricies, X and Y, the columns of which are observations of one variable, and I wish to generate a matrix pairing each value of X and Y) and so I wrote a small utility function to do so, and I'd like to try and get it integrated into numpy core, if it is deemed useful. I have never submitted a patch to numpy before, so I'm not sure as to the protocol; do I ask someone on this list to review the code? Are there conventions I should be aware of? Etc... Thank you all, -E

3 3

getting position index from array
by Ruby Stevenson 20 Jan '12

20 Jan '12

hi, all I am a newbie on numpy ... I am trying to figure out, given an array, how to get back position value based on some conditions. Say, array([1, 0, 0, 0 1], and I want to get a list of indices where it is none-zero, [ 0 , 4 ] The closest thing I can find from the doc is select(), but I can't figure out how to use it properly. Thanks for your help. Ruby

2 2

(no subject)
by Hänel Nikolaus Valentin 20 Jan '12

20 Jan '12

Hi, I would like to make a sanity test to check that calling the same function with different parameters actually gives different results. I am currently using:: try: npt.assert_almost_equal(numpy_result, result) except AssertionError: assert True else: assert False But maybe you have a better way? I couldn't find a 'assert_not_equal' and the above just feels stupid. thanks for your advice. V- -- Valentin Hänel Scientific Software Developer Blue Brain Project http://bluebrain.epfl.ch/

2 2

swaxes(0, 1) 10% faster than transpose on 2D matrix?
by Mark Bakker 19 Jan '12

19 Jan '12

Hello List, I noticed that swapaxes(0,1) is consistently (on my system) 10% faster than transpose on a 2D matrix. Any reason why? Any reason why the swapaxes algorithm is not used in transpose? Just wondering. Thanks, Mark

2 1

NumPy / SciPy related tutorials at PyCon 2012
by Olivier Grisel 19 Jan '12

19 Jan '12

Hi all, Just a quick email to advertise this year's PyCon tutorials as they are very focused on HPC & data analytics. In particular the numpy / scipy ecosystem is well covered, see: https://us.pycon.org/2012/schedule/tutorials/ Here is a selection of tutorials with an abstracts that mention numpy or a related project (scipy, ipython, matplotlib...): - Bayesian statistics made (as) simple (as possible) - Allen Downey https://us.pycon.org/2012/schedule/presentation/10/ - IPython in-depth: high-productivity interactive and parallel python - Fernando Pérez , Brian E. Granger , Min Ragan-Kelley https://us.pycon.org/2012/schedule/presentation/121/ - Faster Python Programs through Optimization - Mike Müller https://us.pycon.org/2012/schedule/presentation/245/ - Graph Analysis from the Ground Up - Van Lindberg https://us.pycon.org/2012/schedule/presentation/228/ - Data analysis in Python with pandas - Wes McKinney https://us.pycon.org/2012/schedule/presentation/427/ - Social Network Analysis with Python - Maksim Tsvetovat https://us.pycon.org/2012/schedule/presentation/15/ - High Performance Python I - Ian Ozsvald https://us.pycon.org/2012/schedule/presentation/174/ - Plotting with matplotlib - Mike Müller https://us.pycon.org/2012/schedule/presentation/238/ - Introduction to Interactive Predictive Analytics in Python with scikit-learn - Olivier Grisel https://us.pycon.org/2012/schedule/presentation/195/ - High Performance Python II - Travis Oliphant https://us.pycon.org/2012/schedule/presentation/343/ Also the main conference has also very interesting talks: https://us.pycon.org/2012/schedule/ The early birds rate for the PyCOn ends on Jan 25. See you in PyCon in March, -- Olivier http://twitter.com/ogrisel - http://github.com/ogrisel

3 3

"Reference count error detected" bug appears with multithreading (OpenMP & TBB)
by Malcolm Reynolds 18 Jan '12

18 Jan '12

Hi, I've built a system which allocates numpy arrays and processes them in C++ code (this is because I'm building a native code module using boost.python and it makes sense to use numpy data storage to then deal with outputs in python, without having to do any copying). Everything seems fine except when I parallelise the main loop, (openmp and TBB give the same results) in which case I see a whole bunch of messages saying "reference count error detected: an attempt was made to deallocate 12 (d)" sometimes during the running of the program, sometimes all at the end (presumably when all the destructors in my program run). To clarify, the loop I am now running parallel takes read-only parameters (enforced by the C++ compiler using 'const') and as far as I can tell there are no race conditions with multiple threads writing to the same numpy arrays at once or anything obvious like that. I recompiled numpy (I'm using 1.6.1 from the official git repository) to print out some extra information with the reference count message, namely a pointer to the thing which is being erroneously deallocated. Surprisingly, it is always the same address for any run of the program, considering this is a message printed out hundreds of times. I've looked into this a little with GDB and as far as I can see the object which the message pertains to is an "array descriptor", or at least that's what I conclude from backtraces similar to the following: Breakpoint 1, arraydescr_dealloc (self=0x1029083a0) at descriptor.c:1501 1501 fprintf(stderr, "*** Reference count error detected: \n" \ (gdb) bt #0 arraydescr_dealloc (self=0x1029083a0) at descriptor.c:1501 #1 0x0000000102897fc4 in array_dealloc (self=0x117e0e850) at arrayobject.c:271 #2 0x0000000103e592d7 in boost::detail::sp_counted_impl_p<garf::multivariate_normal<double> const>::dispose (this=<value temporarily unavailable, due to optimizations>) at refcount.hpp:36 #3 .... my code Obviously I can turn off the parallelism to make this problem go away, but since my underlying algorithm is trivially parallelisable I was counting on being able to achieve linear speedup across cores.. Currently I can, and as far as I know there are no actual incorrect results being produced by the program. However, in my field (Machine Learning) it's difficult enough to know whether the numbers calculated are sensible even without the presence of these kind of warnings, so I'd like to get a handle on at least why this is happening so I'd know know whether I can safely ignore it. My guess at what might be happening is that the multiple threads are dealing with some object concurrently and the updates to the reference count are not processed atomically, meaning that there are too many DECREFs which happen later on. I had presumed that allocated different numpy matrices in different threads, and then all reading from central numpy matrices would work fine, but apparently there is something I missed, pertaining to descriptors.. Can anyone offer any guidance, or at least tell me this is safe to ignore? I can reproduce the problem reliably, so if you need me to do some digging with GDB at the point the error takes place I can do that. Many thanks, Malcolm

2 3