Mailman 3 September 2011 - NumPy-Discussion

Preventing an ndarray subclass from returning new subclass instances for std(), etc
by Keith Hughitt 19 Sep '11

19 Sep '11

Hi all, For a ndarray-subclass I'm working on I extended the array_wrap and array_finalize functions<https://github.com/sunpy/sunpy/blob/master/sunpy/map/basemap.py#L145-165> so that operations like __sub__() would return a new instance of the subclass, instead of an ndarray (although I still haven't completely figured this out yet <http://mail.scipy.org/pipermail/scipy-user/2011-August/030179.html>.) One side-effect of this, however, is that now other operations which should *not* return a subclass (such as std) are returning one; def __array_finalize__(self, obj): """Finishes instantiation of the new map object""" if obj is None: return if hasattr(obj, 'header'): self.header = obj.header # preserve object properties properties = self.get_properties(obj.header) for attr, value in list(properties.items()): setattr(self, attr, getattr(obj, attr, value)) self.center = obj.center self.scale = obj.scale self.units = obj.units def __array_wrap__(self, out_arr, context=None): """Returns a wrapped instance of a Map object""" return np.ndarray.__array_wrap__(self, out_arr, context) I'm sure it is simply a coding error on my part, but so far I havne't been able to track it down. Any ideas? Thanks, Keith

6 8

Indexing by label in 3rd dimension
by Benjamin Landenberger 19 Sep '11

19 Sep '11

Hello list! I have an array *mask* of shape (a, b) and another array *intensities* of shape (N, a, b), where the values in *mask* range from 0 to N-1. It is somehow similar to label arrays in scipy.ndimage. Now I want to pick those entries from the first dimension of *intensities* which are given by *mask*. The returned array shall again wave shape (a, b). Can this be done with fancy indexing? Thank you, Ben

3 4

ANN: PyCPX 0.01, a numpy/cython wrapper for CPlex.
by Hoyt Koepke 19 Sep '11

19 Sep '11

Hello, I'm pleased to announce the first release of a wrapper I wrote for IBM's CPlex Optimizer Suite. It focuses on ease of use and seamless integration with numpy, and allows one to naturally specify constrained linear and quadratic optimization problems over real, boolean, and integer variables. The wrapper is written in cython/C++ for speed and released under the LGPL license. For documentation and examples, please see the website: http://www.stat.washington.edu/~hoytak/code/pycpx/index.html. Please send me any comments, questions and suggestions! I am quite open to feedback. Thanks, --Hoyt ++++++++++++++++++++++++++++++++++++++++++++++++ + Hoyt Koepke + University of Washington Department of Statistics + http://www.stat.washington.edu/~hoytak/ + hoytak(a)gmail.com ++++++++++++++++++++++++++++++++++++++++++

2 1

Bug in numpy std, etc. with other data structures?
by Skipper Seabold 18 Sep '11

18 Sep '11

Just ran into this. Any objections for having numpy.std and other functions in core/fromnumeric.py call asanyarray before trying to use the array's method? Other data structures like pandas and larry define their own std method, for instance, and this doesn't allow them to pass through. I'm inclined to say that the issue is with numpy, though maybe the data structures shouldn't shadow numpy array methods while altering the signature. I dunno. df = pandas.DataFrame(np.random.random((10,5))) np.std(df,axis=0) <snip> TypeError: std() got an unexpected keyword argument 'dtype' np.std(np.asanyarray(df),axis=0) array([ 0.30883352, 0.3133324 , 0.26517361, 0.26389029, 0.20022444]) Though I don't think this would work with larry yet. Pull request: https://github.com/numpy/numpy/pull/160 Skipper

5 8

lstsq: masked arrays, weights, scaling, and covariance
by Charles R Harris 17 Sep '11

17 Sep '11

Hi All, I'd like to start a discussion about modifications to lstsq to accommodate the new masked arrays and move weights, scaling, and covariance determination down to a lower common level. This is motivated by Travis' recent changes to polyfit as well as my own various polynomial fits that also allow weights. Also, once these features are pushed down to lstsq, it should be possible to push them down further into a c-wrapper for the LAPACK routines, which is where I really think they belong in the long run. Because missing values will effect the std/var/cov in the same way as weights of zero, I think support for missing values and weights go naturally together. Support for scaling and covariance are less closely tied, but they are both features I use all the time in practice and having them available will be useful. It might also be nice to change the return signature, though this would require a new function. I rather like the idea of returning the coefficients and a dictionary, where everything not a coefficient gets stuffed into the dictionary. In this regard see also Denis Laxalde's proposal <http://tinyurl.com/3sbwgxy>, something we might want to be consistent with. Thoughts? Chuck

3 3

Polynomial implementation in numpy.ma
by Stéfan van der Walt 17 Sep '11

17 Sep '11

Hi all, There were some failures in the polynomial tests earlier today, and while investigating I saw that numpy.ma implements its own root finder. It uses inversion of a Van der Monde matrix, which I believe may suffer from some numerical instability problems. Given that Charles has gone to some length to implement good polynomial root finders, I think it would be best to employ those instead, and simply pre-filter the data that comes from the masked array module, if possible. Regards Stéfan

2 4

[ANN] glumpy 0.2.0
by Nicolas Rougier 17 Sep '11

17 Sep '11

Hi folks, I am pleased to announce a new release of glumpy, a small python library for the (very) fast vizualization of numpy arrays, (mainly two dimensional) that has been designed with efficiency in mind. If you want to draw nice figures for inclusion in a scientific article, you’d better use matplotlib but if you want to have a sense of what’s going on in your simulation while it is running, then maybe glumpy can help you. Download and screenshots at: http://code.google.com/p/glumpy/ Nicolas

2 2

type-casting inconsistency with timedelta64
by Benjamin Root 16 Sep '11

16 Sep '11

Came across an oddity when experimenting with the datetime64 and timedelta64 objects. a = np.zeros((10,), dtype='l') b = np.datetime64('2010-12-20T14:23:56-0600') c = np.datetime64('2010-12-20T21:27:09-0600') a[0:1] = c - b # This works fine a[0] = c - b # This does not The second assignment throws the following error: TypeError: don't know how to convert scalar number to long Cheers, Ben Root P.S. - np.arange() can make ranges of datetime64 from string inputs, but np.linspace() can not (and does not take a dtype kwarg).

2 1

Difficulty using special types like timedelta and Decimal
by Benjamin Root 16 Sep '11

16 Sep '11

Certain numerical types in Python that support accurate fractional calculations such as timedelta and Decimal do not allow for multiplication or division by a floating point number, but do allow for use with an integer. This can cause difficulties with some functions such as np.gradient() which has a division by 2.0 as part of the algorithm. My question is: with Py3k's division always resulting in a floating point result regardless of the denominator's type, would it be possible to change some of these common denominators over to integers in order to better facilitate support for these special types? Obviously, I don't think we could expect full support of these types, but maybe a mechanism could be developed to better handle these? Thanks, Ben Root

2 3

can't use argmin for array of timedeltas
by Benjamin Root 16 Sep '11

16 Sep '11

I encountered something similar back in April or so and that it was fixed. The problem was that the minimum function was implemented as max(0 - a), and so this fails for timedelta objects that can't do this. This was fixed for min(), but apparently not for argmin(). Ben Root

1 1