Mailman 3 June 2012 - NumPy-Discussion

Are "min", "max" documented for scalars?
by Edward C. Jones 07 Jun '12

07 Jun '12

Python "max" and "min" have an interesting and _useful_ behavior when applied to numpy scalars and Python numbers. Here is a piece of pseudo-code: def max(a, b): if int(b) > int(a): return b else: return a The larger object is returned unchanged. If the two objects are equal, return the first unchanged. Is the behavior of "max", "min", "<", "<=", etc. for numpy scalar objects documented somewhere? keywords: greater than less than

1 0

Incrementing with advanced indexing: why don't repeated indexes repeatedly increment?
by John Salvatier 06 Jun '12

06 Jun '12

Hello, I've noticed that If you try to increment elements of an array with advanced indexing, repeated indexes don't get repeatedly incremented. For example: In [30]: x = zeros(5) In [31]: idx = array([1,1,1,3,4]) In [32]: x[idx] += [2,4,8,10,30] In [33]: x Out[33]: array([ 0., 8., 0., 10., 30.]) I would intuitively expect the output to be array([0,14, 0,10,30]) since index 1 is incremented by 2+4+8=14, but instead it seems to only increment by 8. What is numpy actually doing here? The authors of Theano noticed this behavior a while ago so they python loop through the values in idx (this kind of calculation is necessary for calculating gradients), but this is a bit slow for my purposes, so I'd like to figure out how to get the behavior I expected, but faster. I'm also not sure how to navigate the numpy codebase, where would I look for the code responsible for this behavior?

5 9

boolean indexing of structured arrays
by Benjamin Root 06 Jun '12

06 Jun '12

Not sure if this is a bug or not. I am using a fairly recent master branch. >>> # Setting up... >>> import numpy as np >>> a = np.zeros((10, 1), dtype=[('foo', 'f4'), ('bar', 'f4'), ('spam', 'f4')]) >>> a['foo'] = np.random.random((10, 1)) >>> a['bar'] = np.random.random((10, 1)) >>> a['spam'] = np.random.random((10, 1)) >>> a array([[(0.8748096823692322, 0.08278043568134308, 0.2463584989309311)], [(0.27129432559013367, 0.9645473957061768, 0.41787904500961304)], [(0.4902191460132599, 0.6772263646125793, 0.07460898905992508)], [(0.13542482256889343, 0.8646988868713379, 0.98673015832901)], [(0.6527929902076721, 0.7392181754112244, 0.5919206738471985)], [(0.11248272657394409, 0.5818713903427124, 0.9287213087081909)], [(0.47561103105545044, 0.48848700523376465, 0.7108170390129089)], [(0.47087424993515015, 0.6080209016799927, 0.6583810448646545)], [(0.08447299897670746, 0.39479559659957886, 0.13520188629627228)], [(0.7074970006942749, 0.8426893353462219, 0.19329732656478882)]], dtype=[('foo', '<f4'), ('bar', '<f4'), ('spam', '<f4')]) >>> b = (a['bar'] > 0.4) >>> b array([[False], [ True], [ True], [ True], [ True], [ True], [ True], [ True], [False], [ True]], dtype=bool) >>> # ---- Boolean indexing of structured array with a (10,1) boolean array ---- >>> a[b]['foo'] array([ 0.27129433, 0.49021915, 0.13542482, 0.65279299, 0.11248273, 0.47561103, 0.47087425, 0.707497 ], dtype=float32) >>> # ---- Boolean indexing of structured array with a (10,) boolean array ---- >>> a[b[:,0]]['foo'] array([[(0.27129432559013367, 0.9645473957061768, 0.41787904500961304)], [(0.4902191460132599, 0.6772263646125793, 0.07460898905992508)], [(0.13542482256889343, 0.8646988868713379, 0.98673015832901)], [(0.6527929902076721, 0.7392181754112244, 0.5919206738471985)], [(0.11248272657394409, 0.5818713903427124, 0.9287213087081909)], [(0.47561103105545044, 0.48848700523376465, 0.7108170390129089)], [(0.47087424993515015, 0.6080209016799927, 0.6583810448646545)], [(0.7074970006942749, 0.8426893353462219, 0.19329732656478882)]], dtype=[('foo', '<f4'), ('bar', '<f4'), ('spam', '<f4')]) So, if I index with a (10, 1) boolean array, I get back a (N,) shape result (regardless of whether I am accessing a field or not). But, if I index with a (10, ) boolean array, I get back a (N, 1) result. Note that other forms of indexing such as slicing and fancy indexing returns (N, 1) shaped results. Now, admittedly, this is actually consistent with boolean indexing of regular numpy arrays. I just wanted to make sure that this is intentional. This has caused some confusion for me recently when I (perhaps falsely) expected that the result from a boolean index of a structured array would result in a similarly structured array. The use-case was to modify an existing function by removing the unwanted "rows" with a simply boolean index statement instead of a slice. Cheers! Ben Root

1 0

Changes in PyArray_FromAny between 1.5.x and 1.6.x
by Mike Hansen 06 Jun '12

06 Jun '12

Hello, In trying to upgrade NumPy within Sage, we notices some differences in behavior between 1.5 and 1.6. In particular, in 1.5, we have sage: f = 0.5 sage: f.__array_interface__ {'typestr': '=f8'} sage: numpy.array(f) array(0.5) sage: numpy.array(float(f)) array(0.5) In 1.6, we get the following, sage: f = 0.5 sage: f.__array_interface__ {'typestr': '=f8'} sage: numpy.array(f) array(0.500000000000000, dtype=object) This seems to be do to the changes in PyArray_FromAny introduced in https://github.com/mwhansen/numpy/commit/2635398db3f26529ce2aaea4028a811884… . In particular, _array_find_type used to be used to query our __array_interface__ attribute, and it no longer seems to work. Is there a way to get the old behavior with the current code? --Mike

7 15

numpy.clip behavior at max and min of dtypes
by Edward C. Jones 05 Jun '12

05 Jun '12

Can the following function be written using numpy.clip? In some other way? Does numpy.clip satisfy condition 4 below? Does numpy.clip satisfy some closely related condition? Define a function clipcast: output = clipcast(arr, dtype=None, out=None) 1. All arrays have int or float dtypes. 2. Exactly one of the keyword arguments "dtype" and "out" must be used. If "dtype" is given, then output has that dtype. 3. "output" has the same shape as "arr". 4. Let ER be the set of all the real numbers that can be exactly represented by the output dtype. ER is finite and bounded. Let themin = min(ER) and themax = max(ER). For any real number x, define a function f(x) by If x is in ER, define f(x) = x. If x is between two consecutive numbers, u and v, in ER, then define f(x) = u or f(x) = v. Probably the choice would be made using a C cast. If x < themin, define f(x) = themin. If x > themax, define f(x) = themax. If x is an element of arr, say Arr[I], then output[I] == f(x) where I is any index that defines a single element of arr.

1 0

commit rights for Nathaniel
by Charles R Harris 05 Jun '12

05 Jun '12

Hi All, Numpy is approaching a time of transition. Ralf will be concentrating his efforts on Scipy and I will be cutting back on my work on Numpy. The 1.7 release looks to be delayed and I suspect that the Continuum Analytics folks will become increasingly dedicated to the big data push. We need new people to carry things forward and I think Nathaniel can pick up part of the load. Chuck

4 10

varargs for logical_or, etc
by Neal Becker 05 Jun '12

05 Jun '12

I think it's unfortunate that functions like logical_or are limited to binary. As a workaround, I've been using this: def apply_binary (func, *args): if len (args) == 1: return args[0] elif len (args) == 2: return func (*args) else: return func ( apply_binary (func, *args[:len(args)/2]), apply_binary (func, *args[(len(args))/2:])) Then for example: punc2 = np.logical_and (u % 5 == 4, apply_binary (np.logical_or, u/5 == 3, u/5 == 8, u/5 == 13))

4 3

1D array sorting ascending and descending by fields
by Patrick Redmond 05 Jun '12

05 Jun '12

Hi! I have a one-dimensional ndarray with two fields. I'd like to sort in descending order by field 'a', breaking ties by sorting in ascending order by field 'b'. I've found combinations of sorting and reversing followed by stable sorting that work, but there must be a straightforward way to do it. Your help is appreciated! Thank you, Patrick

4 6

nditer_buffer_flag branch (was: Add data memory allocation tracing facilities. (#284))
by Nathaniel Smith 05 Jun '12

05 Jun '12

On Tue, Jun 5, 2012 at 11:06 AM, Thouis (Ray) Jones wrote: > All of the failing tests seem to have been caused by the buffer copy bug, fixed in https://github.com/mwiebe/numpy/tree/nditer_buffer_flag (but not yet pulled into numpy). > > I also have a version that implements tracing, with pure C in the allocation functions writing to a dynamically allocated buffer, which must then be fetched proactively by Python. However, I think this version is a little nicer to use from the Python perspective. > > --- > Reply to this email directly or view it on GitHub: > https://github.com/numpy/numpy/pull/284#issuecomment-6121817 Speaking of which, Mark - what's the status of that nditer_buffer_flag branch? Should there be a pull request? -N

2 1

some typestrings not recognized anymore
by Ralf Gommers 05 Jun '12

05 Jun '12

Hi, Just ran into this: >>> np.__version__ '1.5.1' >>> np.empty((1,), dtype='>h2') # works in 1.6.2 too array([0], dtype=int16) >>> np.__version__ '1.7.0.dev-fd78546' >>> np.empty((1,), dtype='>h2') Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: data type ">h2" not understood Seems like a quite serious issue. It breaks scipy.io.netcdf for example. Ralf

4 4