Mailman 3 June 2011 - NumPy-Discussion

Pleasantly surprised by recarray
by Neal Becker 24 Jun '11

24 Jun '11

I was pleasantly surprised to find that recarrays can have a recursive structure, and can be defined using a nice syntax: dtype=[('deltaf', float),('eq', bool),('filt', [('pulse', 'a10'), ('alpha', float)])] However, this I discovered just by guessing. It would be good to mention this in the reference doc. BTW, one unfortunate side effect of using recarrays with elements that are not simple types is that identity comparison seems to fail. That is, when addressing my previous problem for e in arr: experiments[tuple((e['a'],e['b']))].append(e) That would apparantly need to be: for e in array: experiments[e['deltaf'], tuple(e['filt'])].append(e) Unless e['filt'] is converted to tuple, it's type is: type(cases[0][4]) Out[55]: <type 'numpy.void'> which does not seem to identity compare as I would want.

1 0

A bit of advice?
by Neal Becker 24 Jun '11

24 Jun '11

I have a set of experiments that I want to plot. There will be many plots. Each will show different test conditions. Suppose I put each of the test conditions and results into a recarray. The recarray could be: arr = np.empty ((#experiments,), dtype=[('x0',int), ('x1',int), ('y0',int)] where x0,x1 are 2 test conditions, and y0 is a result. First I want to group the plots such according to the test conditions. So, I want to first find what all the combinations of test conditions are. Dunno if there is anything simpler than: cases = tuple (set ((e['x0'], e['x1'])) for e in arr) Next, need to select all those experiments which match each of these cases. Now I know of no easy way. Any suggestions? Perhaps I should look again at pytables?

3 4

Parallelize repeated operations
by Jonathan Tu 23 Jun '11

23 Jun '11

Hi, I have found a very strange bug that I cannot understand. I would like to do something like this: Given 4 pairs of numpy arrays (x1, y1, x2, y2, x3, y3, x4, y4), I would like to compute each corresponding inner product ip = np.dot(yi.T, xj), for i = 1,...4 and j = 1,...,4. (Something like a correlation matrix.) What I did was use shared memory and the multiprocessing module (with pool) to load the data in parallel. Each processor loads one pair of the snapshots, so I can do 4 simultaneous loads on my four-core machine, and it takes two cycles of loads to get all the data in memory. Then I tried to do the inner products in parallel as well, asking each processors to do 4 of the 16 total inner products. As it turned out, this was slower in parallel than in serial!!! This only occurs when I use numpy functions. If instead I replace the inner product task by printing to stdout or something of that sort, I get the 4x speedup that I expect. Any ideas? Jonathan Tu

1 0

review/test pull request for tightening default ufunc casting rule
by Mark Wiebe 23 Jun '11

23 Jun '11

Pull request is here: https://github.com/numpy/numpy/pull/95 -Mark

1 1

Python plugins for gcc static analysis
by Charles R Harris 22 Jun '11

22 Jun '11

Thought gcc-python-plugin<http://article.gmane.org/gmane.comp.gcc.devel/121307>might be of interest to some. One of the motivations seems to have been checking reference handling in cpython. Chuck

1 0

Re: [Numpy-discussion] argmax for top N elements
by RadimRehurek 22 Jun '11

22 Jun '11

> Date: Wed, 22 Jun 2011 11:30:47 -0400 > From: Alex Flint <alex.flint(a)gmail.com> > Subject: [Numpy-discussion] argmax for top N elements > > Is it possible to use argmax or something similar to find the locations of > the largest N elements in a matrix? I would also be interested in an O(N) argmax/argmin for indices of top k values in an array. I'm currently using argsort[:k] in a performance sensitive part and it's not ideal. Best, Radim

4 3

write access through iterator
by Neal Becker 22 Jun '11

22 Jun '11

Maybe I'm being dense today, but I don't see how to iterate over arrays with write access. You could read through iterators like: fl = u.flat >>> for item in fl: ... print item but you can't do for item in fl: item = 10 (or, it won't do what you want). Is there any way to do this?

2 1

struct dtype cleanup for testing and review
by Mark Wiebe 22 Jun '11

22 Jun '11

This set of patches cleans up struct dtypes quite a bit and generalizes them to allow overlapping and out-of-order fields with non-object dtypes. The pull request is here: https://github.com/numpy/numpy/pull/94 The commit with documentation changes is here: https://github.com/m-paradox/numpy/commit/b23337999abcb3ecfa648d86f0bf049ef… This fixes ticket (http://projects.scipy.org/numpy/ticket/1790), and some of the problems discussed in ticket ( http://projects.scipy.org/numpy/ticket/1619). -Mark

2 2

argmax for top N elements
by Alex Flint 22 Jun '11

22 Jun '11

Is it possible to use argmax or something similar to find the locations of the largest N elements in a matrix?

2 1

datetime pull request for review and build/test
by Mark Wiebe 22 Jun '11

22 Jun '11

https://github.com/numpy/numpy/pull/93 The summary: * Tighten up date unit vs time unit casting rules, and integrate the NPY_CASTING enum deeper into the datetime conversions * Determine a unit when converting from a string array, similar to when converting from lists of strings * Switch local/utc handling to a timezone= parameter, which also accepts a datetime.tzinfo object. This, for example, enables the use of the pytz library with numpy.datetime64 * Remove the events metadata, make the old API functions raise exceptions, and rename the "frequency" metadata name to "timeunit" * Abstract the flexible dtype mechanism into a function, so that it can be more easily changed without having to add code to many places -Mark

1 1