Mailman 3 July 2017 - NumPy-Discussion

How to compare an array of arrays elementwise to None in
by Martin.Gfeller＠swisscom.com 19 Jul '17

19 Jul '17

Thank you for your help! Sebastian, I couldn't agree more with someone's bug being someone else's feature! A fast identity ufunc would be useful, though. Actually, numpy.frompyfunc(operator.is_,2,1) is much faster than the numpy.vectorize approach - only about 35% slower on quick measurement than the direct ==, as opposed to 62% slower with vectorize (with otypes hint). Robert, yes, that's what I already did provisionally. Eric, that is a nice puzzle - but I agree with Robert about understanding by code maintainers. Thanks again, and best regards, Martin On Mon, 17 Jul 2017 11:41 Sebastian Berg <sebastian(a)sipsolutions.net> write > Yes, I guess ones bug is someone elses feature :(, if it is very bad, we could delay the deprecation probably. For a solutions, maybe > we could add a ufunc for elementwise `is` on object arrays (dunno about the name, maybe `object_identity`. > Just some quick thoughts. > - Sebastian On Mon, 17 Jul 2017 at 17:45 Robert Kern <robert.kern(a)gmail.com> wrote: > Wrap the clumsiness up in a documented, tested utility function with a descriptive name and use that function everywhere instead. > Robert Kern On Mon, Jul 17, 2017 at 10:52 AM, Eric Wieser <wieser.eric+numpy(a)gmail.com> wrote: > Here's a hack that lets you keep using ==: > > class IsCompare: > __array_priority__ = 999999 # needed to make it work on either side of `==` > def __init__(self, val): self._val = val > def __eq__(self, other): return other is self._val > def __neq__(self, other): return other is not self._val > > a == IsCompare(None) # a is None > a == np.array(IsCompare(None)) # broadcasted a is None > >

2 1

How to compare an array of arrays elementwise to None in Numpy 1.13 (was easy before)?
by Martin.Gfeller＠swisscom.com 17 Jul '17

17 Jul '17

Dear all I have object array of arrays, which I compare element-wise to None in various places: >>> a = numpy.array([numpy.arange(5),None,numpy.nan,numpy.arange(6),None],dtype=numpy.object) >>> a array([array([0, 1, 2, 3, 4]), None, nan, array([0, 1, 2, 3, 4, 5]), None], dtype=object) >>> numpy.equal(a,None) FutureWarning: comparison to `None` will result in an elementwise object comparison in the future. So far, I always ignored the warning, for lack of an idea how to resolve it. Now, with Numpy 1.13, I have to resolve the issue, because it fails with: ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() It seem that the numpy.equal is applied to each inner array, returning a Boolean array for each element, which cannot be coerced to a single Boolean. The expression >>> numpy.vectorize(operator.is_)(a,None) gives the desired result, but feels a bit clumsy. Is there a cleaner, efficient way to do an element-wise (but shallow) comparison? Thank you and best regards, Martin Gfeller, Swisscom

4 4

Remote sprinting for scipy 2017
by David Cournapeau 14 Jul '17

14 Jul '17

Hi, Is there a numpy-specific communication channel for this year's sprints ? Could not see anything on the scipy sprints page Thanks, David

1 0

Adding out to bincount
by Jaime Fernández del Río 13 Jul '17

13 Jul '17

There is an ongoing discussion on #9397 <https://github.com/numpy/numpy/issues/9397> about adding an out= keyword argument to np.bincount(). Presently the design seems headed toward: - the new counts will be added to the contents of out, perhaps we need a better name than out, suggestions welcome - if minlength is specified and it doesn't match exactly the size of out, an error will be raised, - if any of the input indices is outside the bounds of the out array, an error will be raised, and - if the out array is not of the exact right type, i.e. np.double if weights are specified, np.intp if not, an error will be raised. If you have an opinion about this functionality please head over there and have your say. Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.

1 0

pytest and degrees of separation.
by Charles R Harris 13 Jul '17

13 Jul '17

Hi All, Just looking for opinions and feedback on the need to keep NumPy from having a hard nose/pytest dependency. The options as I see them are: 1. pytest is never imported until the tests are run -- current practice with nose 2. pytest is never imported unless the testfiles are imported -- what I would like 3. pytest is imported together when numpy is -- what we need to avoid. Currently the approach has been 1), but I think 2) makes more sense and allows more flexibility. Thoughts? Chuck

6 9

Generalized rectangle intersection. (Was: Array blitting)
by Mikhail V 10 Jul '17

10 Jul '17

disclaimer: I am not a past contributor to numpy and I don't know much about github, and what pull request means. So I just put the examples here. So in short, the proposal idea is to add a library function which calculates the intersection area of two rectangles, generalized for any dimensions. The need for this function comes quite often so I suppose it would be good to have such function in the library. Python function prototype: def box_clip(box1, box2, offset): -> (box_intersection, offset1, offset2) Here the rectangle is called "box". The function takes three equally sized arrays (or tuples) which denote the cartesian parameters of boxes and their relative position: - box1 - destination box size (only positive values) - box2 - source box size (only positive values) - offset - offset between box2 and box1 the boxes ( any values ) And returns an array (or tuple?) containing 3 arrays : - box_intersection : size of the intersection area - offset1 : offset in box1' coordinate system - offset2 : offset in box2' coordinate system Following are example of the full function with comments and usage examples, all tested with arrays and tuples as input and all seem to work correctly. #===== def box_clip(box1, box2, offset): L = len(box1) # amount of dimensions sizes_equal = ( L == len (box2) == len (offset) ) if not sizes_equal: print ("Error: input arrays must have equal size") return R = numpy.zeros((3, L)) # init result array for i in range (0,L): # take the i-th axis d = box1[i] # dest box size along i-th axis s = box2[i] # source box size along i-th axis o = offset[i] # offset along i-th axis left = max(0, o) # startpoint of the clipped area right = min(d, o+s) # endpoint of the clipped area r = right - left # size of the clipped area if r < 0: r = 0 # clamp negative size values R[0,i] = r # return the size of the clipped area R[1,i] = left # return the offset in respect to the destinatition box R[2,i] = left-o # return the offset in respect to the source box return R #===== Typical use cases: Example 1. Finding the intersection of two rectangles. E.g. for 2D rectangles defined in the tuple format (coordinate, size): rect1 = numpy.array ( ( (0, 5), (10, 20) ) ) rect2 = numpy.array ( ( (1, 5), (10, 20) ) ) R = box_clip( rect1[1], rect2[1], rect2[0] - rect1[0] ) > R [[ 9. 20.] # intersection size [ 1. 0.] # coordinate in rect1's origin [ 0. 0.]] # coordinate in rect2's origin E.g. to construct the rectangle object in the same input format (global coord, size) just sum the local rect1's coordinate (R[1]) and global rect1's coordinate: rect_x = numpy.array ( ( rect1[0] + R[1] , R[0] ) ) > rect_x [[ 1. 5.] [ 9. 20.]] Example 2. Use the function as a helper to find array slices for the array "blit" operation. This will need another intermediate function to convert between two cartesian points and the slice object: def p2slice(startpoint, endpoint): # point to slice conversion intervals = numpy.column_stack((startpoint, endpoint)) slices = [slice(*i) for i in intervals] return slices # exampe of blitting SRC array into the DEST array at a given offset W = 6; H = 6 w = 4; h = 1 DEST = numpy.ones([H,W], dtype = "uint8") SRC = numpy.zeros([h,w], dtype = "uint8") SRC[:]=8 offset = (5,4) R = box_clip(DEST.shape, SRC.shape, offset) DEST_slice = p2slice( R[1], R[1] + R[0] ) SRC_slice = p2slice( R[2], R[2] + R[0] ) DEST[DEST_slice] = SRC[SRC_slice] # blit >> DEST [[1 1 1 1 1 1] [1 1 1 1 1 1] [1 1 1 1 1 1] [1 1 1 1 1 1] [1 1 1 1 1 1] [1 1 1 1 8 8]] Notes: the function should be as general as possible. The input as box sizes and returning the intersection area size and both local offsets seems to be appropriate, since more often the rectangle objects are defined as (coordinate, size) tuple and in many cases the destination box is itself the origin (i.e. it's coordinate is 0,0,..) But of course there can be various variants for the output format and order. Regards, Mikhail V

3 3

reshape 2D array into 3D
by paul.carrico＠free.fr 10 Jul '17

10 Jul '17

Dear All I'm looking in a way to reshape a 2D matrix into a 3D one ; in my example I want to MOVE THE COLUMNS FROM THE 4TH TO THE 8TH IN THE 2ND PLANE (3rd dimension i guess) a = np.random.rand(5,8); print(a) I tried a = p.reshape(d, (2,5,4), ) but it is not what I'm expecting Nota : it looks like the following task (while I want to split it in 2 levels and not in 4), but I've not understood at all https://stackoverflow.com/questions/31686989/numpy-reshape-and-partition-2d… Thanks for your support Paul

3 3

Re: [Numpy-discussion] reshape 2D array into 3D
by Yarko Tymciurak 10 Jul '17

10 Jul '17

On Mon, Jul 10, 2017 at 8:46 AM Yarko Tymciurak <yarkot1(a)gmail.com> wrote: > On Mon, Jul 10, 2017 at 8:37 AM <paul.carrico(a)free.fr> wrote: > >> Thanks >> >> >> Nevertheless it does not work for me and I suspect the python/numpy >> releases :-( >> >> The server on which I'm working on is under Contos 7 that uses python 2.7 >> et numpy 1.7 from memory ; I tried to upgrade both of them (plus spyder) >> but it fails. >> > Question: Did you try to control the python & numpy versions by creating a virtualenv, or a conda env? I didn't want to impact the other solvers installed on, so I stopped >> >> Paul >> >> a = np.arange(40).reshape(5, 8); print(a) >> print("b =") >> b = np.lib.stride_tricks.as_strided(a, (2, 5, 4), (16, 32, 4)); print(b) >> >> [[ 0 1 2 3 4 5 6 7] >> [ 8 9 10 11 12 13 14 15] >> [16 17 18 19 20 21 22 23] >> [24 25 26 27 28 29 30 31] >> [32 33 34 35 36 37 38 39]] >> b = >> [[[ 0 4294967296 1 8589934592] >> [ 4 21474836480 5 25769803776] >> [ 8 38654705664 9 42949672960] >> [ 12 55834574848 13 60129542144] >> [ 16 73014444032 17 77309411328]] >> >> [[ 2 12884901888 3 17179869184] >> [ 6 30064771072 7 34359738368] >> [ 10 47244640256 11 51539607552] >> [ 14 64424509440 15 68719476736] >> [ 18 81604378624 19 85899345920]]] >> >> >> >> >> Le 2017-07-10 15:16, eat a écrit : >> >> Hi, >> >> On Mon, Jul 10, 2017 at 3:20 PM, <paul.carrico(a)free.fr> wrote: >> >>> Dear All >>> >>> I'm looking in a way to reshape a 2D matrix into a 3D one ; in my >>> example I want to *move the columns from the 4th to the 8th in the 2nd >>> plane* (3rd dimension i guess) >>> >>> a = np.random.rand(5,8); print(a) >>> >>> I tried >>> >>> a = p.reshape(d, (2,5,4), ) but it is not what I'm expecting >>> >>> >>> Nota : it looks like the following task (while I want to split it in 2 >>> levels and not in 4), but I've not understood at all >>> >>> >>> https://stackoverflow.com/questions/31686989/numpy-reshape-and-partition-2d… >>> >> Is this what you are looking for: >> import numpy as np >> >> a= np.arange(40).reshape(5, 8) >> >> a >> Out[]: >> array([[ 0, 1, 2, 3, 4, 5, 6, 7], >> [ 8, 9, 10, 11, 12, 13, 14, 15], >> [16, 17, 18, 19, 20, 21, 22, 23], >> [24, 25, 26, 27, 28, 29, 30, 31], >> [32, 33, 34, 35, 36, 37, 38, 39]]) >> >> np.lib.stride_tricks.as_strided(a, (2, 5, 4), (16, 32, 4)) >> Out[]: >> array([[[ 0, 1, 2, 3], >> [ 8, 9, 10, 11], >> [16, 17, 18, 19], >> [24, 25, 26, 27], >> [32, 33, 34, 35]], >> >> [[ 4, 5, 6, 7], >> [12, 13, 14, 15], >> [20, 21, 22, 23], >> [28, 29, 30, 31], >> [36, 37, 38, 39]]]) >> >> Regards, >> -eat >> >>> >>> Thanks for your support >>> >>> >>> Paul >>> >>> _______________________________________________ >>> NumPy-Discussion mailing list >>> NumPy-Discussion(a)python.org >>> https://mail.python.org/mailman/listinfo/numpy-discussion >>> >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion(a)python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion(a)python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> >

2 1

Reshape 2D array into 3D
by paul.carrico＠free.fr 10 Jul '17

10 Jul '17

Dear All I'm looking in a way to reshape a 2D matrix into a 3D one ; in my example I want to MOVE THE COLUMNS FROM THE 4TH TO THE 8TH IN THE 2ND PLANE (3rd dimension i guess) a = np.random.rand(5,8); print(a) I tried a = p.reshape(d, (2,5,4), ) but it is not what I'm expecting Nota : it looks like the following task (while I want to split it in 2 levels and not in 4), but I've not understood at all https://stackoverflow.com/questions/31686989/numpy-reshape-and-partition-2d… Thanks for your support Paul

2 1

Re: [Numpy-discussion] Scipy 2017 NumPy sprint
by Peter Creasey 09 Jul '17

09 Jul '17

> From: Marten van Kerkwijk <m.h.vankerkwijk(a)gmail.com> > > Though based on my experience with Quantity, I'd also argue that the > more annoying problems are not so much with `ndarray` itself, but > rather with the helper functions. Just to give an alternative view - as another astronomer I would say that concerns about the use of subclassing in Astropy is one of the reasons I rarely use it. To take an example, if I’m relying on the Quantity class to keep track of my units, then if I have an (N,3) array-like of positions in parsecs, that’s just one step away from going through the sausage-machine of scipy.spatial.cKDTree.query (to find distances to neighbours) and my units are toast. That probably sounds stronger than I meant - keeping track of units systematically is neither easy nor unimportant and I have enormous respect for the developers of Astropy and the community they have built around it. Peter Creasey

2 1