Mailman 3 February 2009 - NumPy-Discussion

segfaults when passing ndarray subclass to ufunc with out=None
by Darren Dale March 8, 2009

March 8, 2009

I am seeing some really strange behavior when I try to pass an ndarray subclass and out=None to numpy's ufuncs. This example will reproduce the problem with svn numpy, the first print statement yields 1 as expected, the second yields "<type 'builtin_function_or_method'>" and the third yields a segmentation fault: import numpy as np class MyArray(np.ndarray): __array_priority__ = 20 def __new__(cls): return np.asarray(1).view(cls).copy() def __repr__(self): return 'my_array' __str__ = __repr__ def __mul__(self, other): return super(MyArray, self).__mul__(other) def __rmul__(self, other): return super(MyArray, self).__rmul__(other) mine = MyArray() print np.multiply(1, 1, None) x = np.multiply(mine, mine, None) print type(x) print x Darren

1 1

Numpy array in iterable
by Kim Hansen March 5, 2009

March 5, 2009

Hi Numpy discussions Quite often I find myself wanting to generate a boolean mask for fancy slicing of some array, where the mask itself is generated by checking if its value has one of several relevant values (corresponding to states) So at the the element level thsi corresponds to checking if element in iterable But I can't use the in operator on a numpy array: In [1]: test = arange(5) In [2]: states = [0, 2] In [3]: mask = test in states --------------------------------------------------------------------------- ValueError Traceback (most recent call last) C:\Documents and Settings\kha\<ipython console> in <module>() ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() I can however make my own utility function which works effectively the same way by iterating through the states In [4]: for i, state in enumerate(states): ...: if i == 0: ...: result = test == state ...: else: ...: result |= test == state ...: ...: In [5]: result Out[5]: array([ True, False, True, False, False], dtype=bool) However, I would have thought such an "array.is_in()" utility function was already available in the numpy package? But I can't find it, and I am curious to hear if it is there or if it just available in another form which I have simply overlooked. If it is not there I think it could be a nice extra utility funtion for the ndarray object. --Slaunger

3 10

loadtxt issues
by A B March 4, 2009

March 4, 2009

Hi, How do I write a loadtxt command to read in the following file and store each data point as the appropriate data type: 12|h|34.5|44.5 14552|bbb|34.5|42.5 Do the strings have to be read in separately from the numbers? Why would anyone use 'S10' instead of 'string'? dt = {'names': ('gender','age','weight','bal'), 'formats': ('i4', 'S4','f4', 'f4')} a = loadtxt("sample_data.txt", dtype=dt) gives ValueError: need more than 1 value to unpack I can do a = loadtxt("sample_data.txt", dtype="string") but can't use 'string' instead of S4 and all my data is read into strings. Seems like all the examples on-line use either numeric or textual input, but not both. Thanks.

4 6

PyArray_SETITEM with object arrays in Cython
by Wes McKinney March 4, 2009

March 4, 2009

Hello, I am writing some Cython code and have noted that the buffer interface offers very little speedup for PyObject arrays. In trying to rewrite the same code using the C API in Cython, I find I can't get PyArray_SETITEM to work, in a call like: PyArray_SETITEM(result, <void *> iterresult.dataptr, obj) where result is an ndarray of dtype object, and obj is a PyObject*. Anyone have some experience with this can offer pointers (no pun intended!)? Thanks, Wes

3 5

Question on lstsq and correlation coeff
by Anthony Kong March 4, 2009

March 4, 2009

Hi, all, It is probably a newbie question. I trying to use scipy/numpy in a finanical context. I want to compute the correlation coeff of two series (returns vs index returns). I tried two appoarches Firstly, from scipy.linalg import lstsq coeffs,a,b,c = lstsq(matrix, returns) # matrix contains index returns then I tried, import numpy as np cov = np.cov(idx1, returns) print cov.tolist() stddev_x = np.std(returns, ddof=1) stddev_y = np.std(idx1, ddof=1) print "cor = %s" % (cov.tolist()[:-1] /(stddev_x * stddev_y)) They differ from each other. As you can see from the numpy example, I am trying to find cor coeff for a sample. (ddof=1) So, my question is: is the discrepency caused by the fact that I am trying to use lstsq() on a 'sample popluation' (i.e. I am not regressing a full return series)? Is it correct to use lstsq() this way? Cheers, Anthony NOTICE This e-mail and any attachments are confidential and may contain copyright material of Macquarie Group Limited or third parties. If you are not the intended recipient of this email you should not read, print, re-transmit, store or act in reliance on this e-mail or any attachments, and should destroy all copies of them. Macquarie Group Limited does not guarantee the integrity of any emails or any attached files. The views or opinions expressed are the author's own and may not reflect the views or opinions of Macquarie Group Limited.

9 13

problem with assigning to recarrays
by Brian Gerke March 3, 2009

March 3, 2009

Hi- I'm quite new to numpy and to python in general, so I apologize if I'm missing something obvious, but I've come across some seemingly nasty behavior when trying to assign values to the fields of an indexed subarray of a numpy record array. Perhaps an example would explain it best. First, I make a boring record array: In [124]: r = rec.fromarrays([zeros(5), zeros(5)], names='field1,field2') This has five elements with two fields, all values are zero. Now I can change the values for field1 for a few of the array elements: In [125]: r[1].field1=1 In [126]: r[3].field1=1 Let's check and make sure that worked: In [127]: print r.field1 [ 0. 1. 0. 1. 0.] So far, so good. Now I want to change the value of field2 for those same elements: In [128]: r[where(r.field1 == 1.)].field2 = 1 Ok, so now the values of field 2 have been changed, for those elements right? In [129]: r.field2 Out[129]: array([ 0., 0., 0., 0., 0.]) Wait. What? That can't be right. Let's check again: In [130]: print r[where(r.field1 == 1.)].field2 [ 0. 0.] Ok, so it appears that I can *access* fields in this array with an array of indices, but I can't assign new values to fields so accessed. However, I *can* change the values if I use a scalar index. This is different from the behavior of ordinary arrays, for which I can reassign elements' values either way. Moreover, when I try to reassign record array fields by indexing with an array of indices, it would appear that nothing at all happens. This syntax is equivalent to the pass command. So, my question is this: is there some reason for this behavior in record arrays, which is unexpectedly different from the behavior of normal arrays, and rather confusing. If so, why does the attempt to assign values to fields of an indexed subarray not raise some kind of error, rather than doing nothing? I think it's unlikely that I've actually found a bug in numpy, but this behavior does not make sense to me. Thanks for any insights, Brian

3 6

Re: [Numpy-discussion] porting NumPy to Python 3
by James Watson March 2, 2009

March 2, 2009

Hi Ondrej, To get 2to3 to run without warnings, the following files require minor changes: - numpy/distutils/fcompiler/intel.py - numpy/distutils/misc_util.py - numpy/distutils/command/build_src.py - numpy/f2py/crackfortran.py - numpy/lib/function_base.py - numpy/linalg/lapack_lite/make_lite.py There are also other (possibly many, still working on this) files that require syntactic changes to run post 2to3. Is there anywhere specific I should upload these changes? Is this list appropriate? Is there a developer I can send these and future patches to? James. > Date: Tue, 3 Feb 2009 01:03:15 -0800 > From: Ondrej Certik <ondrej(a)certik.cz> > Subject: Re: [Numpy-discussion] porting NumPy to Python 3 > To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> > Message-ID: > <85b5c3130902030103w69180a6bm4143050b4b82bf1f(a)mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > Hi James, > > On Thu, Jan 29, 2009 at 2:11 AM, James Watson <watson.jim(a)gmail.com> wrote: >> Hi, >> >> I am interested in contributing to the port of NumPy to Python 3. Who >> I should coordinate effort with? >> >> I have started at the Python end of the problem (as opposed to >> http://www.scipy.org/Python3k), e.g. I have several patches to get >> 2to3 to work on NumPy's Python source code. > > I am sorry that noone has replied to your email. Could you please > upload your patches somewhere? > > Ondrej

5 7

RFR: #1008 Loss of precision in (complex) arcsinh & arctanh
by Pauli Virtanen March 2, 2009

March 2, 2009

http://scipy.org/scipy/numpy/ticket/1008 http://codereview.appspot.com/22054 -- Pauli Virtanen

4 4

Call for testing: full blas/lapack builds of numpy on windows 64 bits
by David Cournapeau March 2, 2009

March 2, 2009

Hi, That's a call for testing for 64 bits windows users out there: please try the following binary with the test suite: http://www.ar.media.kyoto-u.ac.jp/members/david/archives/numpy/numpy-1.3.0.… python -c "import numpy; numpy.test()" Report any crash. I am particularly interested in vista and higher versions without visual studio installed - it looks like those cause a lot of trouble wrt the C runtime. *THIS IS ONLY FOR TESTING* - it is still far from being stable. Details: This one is built with free compilers (gcc + gfortran 4.4), and BLAS/LAPACK built with gfortran. This should enable openmp usage too (4.4 supports openmp3.0). It also means supporting scipy is not too far away - there are still some issues with C++, but hopefully, those are compiler bugs, and all numpy issues will have been sorted out. cheers, David

9 15

Bilateral filter: bug corrected
by Nadav Horesh March 2, 2009

March 2, 2009

Attached here a code for bilateral filter: 1. The code is composed of a cython back end (bilateral_base.pyx) and a python front end (bilateral.py) 2. I do not have the tools to make windows binaries (I run it on gentoo linux) . 3. It is not hard to strip the cython code to get a pure (and slow) python implementation. 4. If someone finds the licensing inadequate, I have no problem to re-license. Any comments are (more then) welcome Nadav

3 20