Mailman 3 June 2011 - NumPy-Discussion

[job] Python Job at Hedge Fund
by Keith Goodman 07 Jun '11

07 Jun '11

We are looking for help to predict tomorrow's stock returns. The challenge is model selection in the presence of noisy data. The tools are ubuntu, python, cython, c, numpy, scipy, la, bottleneck, git. A quantitative background and experience or interest in model selection, machine learning, and software development are a plus. This is a full time position in Berkeley, California, two blocks from UC Berkeley. If you are interested send a CV or similar (or questions) to '.'.join(['htiek','scitylanayelekreb@namdoog','moc'][::-1])[::-1]

1 0

dealing with datetime UTC vs linear time
by Mark Wiebe 07 Jun '11

07 Jun '11

I've given the UTC leap-second issue a bit of thought, and here are some related requirements I've come up with: 1. The underlying data representation of datetime64 should be linearly related to time. 2. Default calendar functionality should be based on ISO 8601, and by reference be in UTC 3. It should be possible to unambiguously specify a time at any point in the past or the future, and get such a representation back from NumPy. One problem trying to resolve these requirements with respect to UTC is that either the underlying time has jump discontinuities, like in POSIX time_t, or minutes, hours, days, and weeks are sometimes one second longer or shorter. Additionally, it's not possible in general to specify UTC times with second resolution 6 or more months in advance. Here's what I propose NumPy's datetime do: A. Use a linear time representation, point 1. Conversion to/from ISO 8601 strings thus requires leap-second knowledge. B. Extend ISO8601 with a "TAI" suffix which is just like "Z" but has no leap-seconds and is 34 seconds ahead of an equivalent "Z" time at the present. This satisfies point 3. C. Add a metadata item which chooses between "UTC" and "TAI". For seconds and finer, converting between UTC and TAI units is safe, and for minutes and coarser, is unsafe. UTC involves leap-seconds and an ambiguity-resolution in the few cases that need it, while in TAI everything is linear. Local time zone-based formatting is only available for UTC datetimes. By default, all units will be UTC, but either TAI or UTC can be specified like 'M8[D:TAI]' or 'M8[h:UTC]'. Does this design sound reasonable to people? Cheers, Mark

3 5

error when numpy is imported
by akshar bhosale 06 Jun '11

06 Jun '11

[GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/admin/numpy-1.6.0b2/install/lib/python/numpy/__init__.py", line 137, in <module> import add_newdocs File "/home/admin/numpy-1.6.0b2/install/lib/python/numpy/add_newdocs.py", line 9, in <module> from numpy.lib import add_newdoc File "/homeadmin/numpy-1.6.0b2/install/lib/python/numpy/lib/__init__.py", line 13, in <module> from polynomial import * File "/home/admin/numpy-1.6.0b2/install/lib/python/numpy/lib/polynomial.py", line 17, in <module> from numpy.linalg import eigvals, lstsq File "/home/admin/numpy-1.6.0b2/install/lib/python/numpy/linalg/__init__.py", line 48, in <module> from linalg import * File "/home/dmin/numpy-1.6.0b2/install/lib/python/numpy/linalg/linalg.py", line 23, in <module> from numpy.linalg import lapack_lite ImportError: /home/external/Python-2.6/lib/python2.6/site-packages/numpy/linalg/lapack_lite.so: undefined symbol: _gfortran_concat_string We have intel mkl with which we are trying to build numpy and scipy. Kinly help

2 2

Re: [Numpy-discussion] k maximal elements
by RadimRehurek 06 Jun '11

06 Jun '11

> On Mon, Jun 6, 2011 at 6:57 AM, gary ruben <gruben(a)bigpond.net.au> wrote: > > I learn a lot by watching the numpy and scipy lists (today Olivier > > taught me about heapq :), but he may not have noticed that Python 2.4 > > added an nsmallest method) I needed indices of the selected elements as well (not just the k-smallest values themselves) some time ago and compared heapq vs. sort vs. numpy.argsort. Timing results (for my specific usecase) are at https://github.com/piskvorky/gensim/issues/5 . The results are as expected: argsort is a must, forming (value, index) tuples explicitly and then sorting that with sort/heapq kills performance. Best, Radim

1 0

Using NumPy iterators in C extension
by Irwin Zaid 06 Jun '11

06 Jun '11

Hi all, I am writing a C extension for NumPy that implements a custom function. I have a minor question about iterators, and their order of iteration, that I would really appreciate some help with. Here goes... My function takes a sequence of N-dimensional input arrays and a single (N+1)-dimensional output array. For each input array, it iterates over the N dimensions of that array and the leading N dimensions of the output array. It stores a result in the trailing dimension of the output array that is cumulative for the entire sequence of input arrays. Crucially, I need to be sure that the input and output elements always match during an iteration, otherwise the output array becomes incorrect. Meaning, is the kth element visited in the output array during a single iteration the same for each input array? So far, I have implemented this function using two iterators created with NpyIter_New(), one for an N-dimensional input array and one for the (N+1)-dimensional output array. I call NpyIter_RemoveAxis() on the output array iterator. I am using NPY_KEEPORDER for the NPY_ORDER flag, which I suspect is not what I want. How should I guarantee that I preserve order? Should I be using a different order flag, like NPY_CORDER? Or should I be creating a multi-array iterator with NpyIter_MultiNew for a single input array and the output array, though I don't see how to do that in my case. Thanks in advance for your help! I can provide more information if something is unclear. Best, Irwin

2 1

_sort module.
by Charles R Harris 06 Jun '11

06 Jun '11

Hi All, Numpy has a _sort module which contains *no* methods and whose only purpose is to modify the type descriptors by adding pointers to the sorting functions when it is loaded. Consequently _sort is imported early in numpy/core/__init__.py and is otherwise unused. This really doesn't seem right. I would like to make the sorting functions into a library or at least link them directly as part of the build process. I'd also like to move them into their own directory and break them up into several files in the process of adding more functions for order statistics. Thoughts? Chuck

2 1

k maximal elements
by Alex Ter-Sarkissov 06 Jun '11

06 Jun '11

I have a vector of positive integers length n. Is there a simple (i.e. without sorting/ranking) of 'pulling out' k larrgest (or smallest) values. Something like *sum(x[sum(x,1)>(max(sum(x,1)+min(sum(x,1))))/2,])* but smarter

5 6

[Repost] Adding the arrays returned in an array iterator
by srean 04 Jun '11

04 Jun '11

Bumping my question tentatively. I am fairly sure there is a good answer and for some reason it got overlooked. Regards srean ---------- Forwarded message ---------- From: srean <srean.list(a)gmail.com> Date: Fri, May 27, 2011 at 10:36 AM Subject: Adding the arrays in an array iterator To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> Hi List, I have to sum up an unknown number of ndarrays of the same size. These arrays, possibly thousands in number, are provided via an iterator. Right now I use python reduce with operator.add. Does that invoke the corresponding ufunc internally ? I want to avoid creating temporaries, which I suspect a naive invocation of reduce will create. With ufunc I know how to avoid making copies using the output parameter, but not sure how to make use of that in reduce. It is not essential that I use reduce though, so I would welcome idiomatic and efficient way of executing this. So far I have stayed away from building an ndarray object and summing across the relevant dimension. Is that what I should be doing ? Different invocations of this function has different number of arrays, so I cannot pre-compile away this into a numexpr. Thanks and regards srean

2 2

reinterpret complex <-> float
by Neal Becker 03 Jun '11

03 Jun '11

Can I arrange to reinterpret an array of complex of length N as an array of float of length 2N, and vice-versa? If so, how?

3 2

numpy input with genfromttxt()
by jgrub 03 Jun '11

03 Jun '11

Hello, im actually try to read in data with genfromtxt(), i want to read in numbers which are stored in a textfile like this: 0,000000 0,001221 0,001221 0,000000 1,278076 160,102539 4,000000E-7 0,000000 0,000000 0,002441 1,279297 160,000000 8,000000E-7 -0,001221 0,000000 0,001221 1,279297 159,897461 1,200000E-6 0,000000 0,000000 0,001221 1,279297 160,000000 1,600000E-6 -0,001221 0,000000 0,003662 1,278076 159,897461 2,000000E-6 0,000000 -0,001221 0,003662 1,279297 160,000000 my problem is that they are seperated with a comma so when i try to read them i just get a numpy array with NaN's. is there a short way to replace the "," with "." ? -- View this message in context: http://old.nabble.com/numpy-input-with-genfromttxt%28%29-tp31757790p3175779… Sent from the Numpy-discussion mailing list archive at Nabble.com.

5 7