Mailman 3 July 2020 - NumPy-Discussion

mixed mode arithmetic
by Neal Becker 11 Jul '23

11 Jul '23

I've been browsing the numpy source. I'm wondering about mixed-mode arithmetic on arrays. I believe the way numpy handles this is that it never does mixed arithmetic, but instead converts arrays to a common type. Arguably, that might be efficient for a mix of say, double and float. Maybe not. But for a mix of complex and a scalar type (say, CDouble * Double), it's clearly suboptimal in efficiency. So, do I understand this correctly? If so, is that something we should improve?

4 6

Invalid value encoutered : how to prevent numpy.where to do this?
by Eric Emsellem 18 Feb '23

18 Feb '23

Dear all, I have a code using lots of "numpy.where" to make some constrained calculations as in: data = arange(10) result = np.where(data == 0, 0., 1./data) # or data1 = arange(10) data2 = arange(10)+1.0 result = np.where(data1 > data2, np.sqrt(data1-data2), np.sqrt(data2-data2)) which then produces warnings like: /usr/bin/ipython:1: RuntimeWarning: invalid value encountered in sqrt or for the first example: /usr/bin/ipython:1: RuntimeWarning: divide by zero encountered in divide How do I avoid these messages to appear? I know that I could in principle use numpy.seterr. However, I do NOT want to remove these warnings for other potential divide/multiply/sqrt etc errors. Only when I am using a "where", to in fact avoid such warnings! Note that the warnings only happen once, but since I am going to release that code, I would like to avoid the user to get such messages which are irrelevant here (because I am testing, with the where, when NOT to divide by zero or take a sqrt of a negative number). thanks! Eric

5 4

Documentation Team meeting - Monday June 8th
by Melissa Mendonça 04 Dec '22

04 Dec '22

Hi all! A reminder that on Monday, June 8, we have another documentation team meeting at 3PM UTC**. If you wish to join on Zoom, you need to use this link https://zoom.us/j/420005230 Here's the permanent hackmd document with the meeting notes: https://hackmd.io/oB_boakvRqKR-_2jRV-Qjg <https://www.google.com/url?q=https%3A%2F%2Fhackmd.io%2FoB_boakvRqKR-_2jRV-Q…> Hope to see you around (especially if you want to introduce yourself or discuss ideas for Google Season of Docs). ** You can click this link to get the correct time at your timezone: https://www.timeanddate.com/worldclock/fixedtime.html?msg=NumPy+Documentati… - Melissa

6 74

[Feature Request] Add alias of np.concatenate as np.concat
by Iordanis Fostiropoulos 10 May '22

10 May '22

In regard to Feature Request: https://github.com/numpy/numpy/issues/16469 It was suggested to sent to the mailing list. I think I can make a strong point as to why the support for this naming convention would make sense. Such as it would follow other frameworks that often work alongside numpy such as tensorflow. For backward compatibility, it can simply be an alias to np.concatenate I often convert portions of code from tf to np, it is as simple as changing the base module from tf to np. e.g. np.expand_dims -> tf.expand_dims. This is done either in debugging (e.g. converting tf to np without eager execution to debug portion of the code), or during prototyping, e.g. develop in numpy and convert in tf. I find myself more than at one occasion to getting syntax errors because of this particular function np.concatenate. It is unnecessarily long. I imagine there are more people that also run into the same problems. Pandas uses concat (torch on the other extreme uses simply cat, which I don't think is as descriptive).

7 6

Re: [Numpy-discussion] example reading binary Fortran file
by Neil Martinsen-Burrell 22 Jul '21

22 Jul '21

David Froger <david.froger.info <at> gmail.com> writes: > Hy,My question is about reading Fortran binary file (oh no this question > again...) I've posted this before, but I finally got it cleaned up for the Cookbook. For this purpose I use a subclass of file that has methods for reading unformatted Fortran data. See http://www.scipy.org/Cookbook/FortranIO/FortranFile. I'd gladly see this in numpy or scipy somewhere, but I'm not sure where it belongs. > program makeArray > implicit none > integer,parameter:: nx=10,ny=20 > real(4),dimension(nx,ny):: ux,uy,p > integer :: i,j > open(11,file='uxuyp.bin',form='unformatted') > do i = 1,nx > do j = 1,ny > ux(i,j) = real(i*j) > uy(i,j) = real(i)/real(j) > p (i,j) = real(i) + real(j) > enddo > enddo > write(11) ux,uy > write(11) p > close(11) > end program makeArray When I run the above program compiled with gfortran on my Intel Mac, I can read it back with:: >>> import numpy as np >>> from fortranfile import FortranFile >>> f=FortranFile('uxuyp.bin', endian='<') >>> uxuy = f.readReals(prec='f') # 'f' for default reals >>> len(uxuy) 400 >>> ux = np.array(uxuy[:200]).reshape((20,10)).T >>> uy = np.array(uxuy[200:]).reshape((20,10)).T >>> p = f.readReals('f').reshape((20,10)).T >>> ux array([[ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20.], [ 2., 4., 6., 8., 10., 12., 14., 16., 18., 20., 22., 24., 26., 28., 30., 32., 34., 36., 38., 40.], [ 3., 6., 9., 12., 15., 18., 21., 24., 27., 30., 33., 36., 39., 42., 45., 48., 51., 54., 57., 60.], [ 4., 8., 12., 16., 20., 24., 28., 32., 36., 40., 44., 48., 52., 56., 60., 64., 68., 72., 76., 80.], [ 5., 10., 15., 20., 25., 30., 35., 40., 45., 50., 55., 60., 65., 70., 75., 80., 85., 90., 95., 100.], [ 6., 12., 18., 24., 30., 36., 42., 48., 54., 60., 66., 72., 78., 84., 90., 96., 102., 108., 114., 120.], [ 7.,Proxy-Connection: keep-alive Cache-Control: max-age=0 14., 21., 28., 35., 42., 49., 56., 63., 70., 77., 84., 91., 98., 105., 112., 119., 126., 133., 140.], [ 8., 16., 24., 32., 40., 48., 56., 64., 72., 80., 88., 96., 104., 112., 120., 128., 136., 144., 152., 160.], [ 9., 18., 27., 36., 45., 54., 63., 72., 81., 90., 99., 108., 117., 126., 135., 144., 153., 162., 171., 180.], [ 10., 20., 30., 40., 50., 60., 70., 80., 90., 100., 110., 120., 130., 140., 150., 160., 170., 180., 190., 200.]]) >>> uy array([[ 1. , 0.5 , 0.33333334, 0.25 , 0.2 , 0.16666667, 0.14285715, 0.125 , 0.11111111, 0.1 , 0.09090909, 0.08333334, 0.07692308, 0.07142857, 0.06666667, 0.0625 , 0.05882353, 0.05555556, 0.05263158, 0.05 ], [ 2. , 1. , 0.66666669, 0.5 , 0.40000001, 0.33333334, 0.2857143 , 0.25 , 0.22222222, 0.2 , 0.18181819, 0.16666667, 0.15384616, 0.14285715, 0.13333334, 0.125 , 0.11764706, 0.11111111, 0.10526316, 0.1 ], [ 3. , 1.5 , 1. , 0.75 , 0.60000002, 0.5 , 0.42857143, 0.375 , 0.33333334, 0.30000001, 0.27272728, 0.25 , 0.23076923, 0.21428572, 0.2 , 0.1875 , 0.17647059, 0.16666667, 0.15789473, 0.15000001], [ 4. , 2. , 1.33333337, 1. , 0.80000001, 0.66666669, 0.5714286 , 0.5 , 0.44444445, 0.40000001, 0.36363637, 0.33333334, 0.30769232, 0.2857143 , 0.26666668, 0.25 , 0.23529412, 0.22222222, 0.21052632, 0.2 ], [ 5. , 2.5 , 1.66666663, 1.25 , 1. , 0.83333331, 0.71428573, 0.625 , 0.55555558, 0.5 , 0.45454547, 0.41666666, 0.38461539, 0.35714287, 0.33333334, 0.3125 , 0.29411766, 0.27777779, 0.2631579 , 0.25 ], [ 6. , 3. , 2. , 1.5 , 1.20000005, 1. , 0.85714287, 0.75 , 0.66666669, 0.60000002, 0.54545456, 0.5 , 0.46153846, 0.42857143, 0.40000001, 0.375 , 0.35294119, 0.33333334, 0.31578946, 0.30000001], [ 7. , 3.5 , 2.33333325, 1.75 , 1.39999998, 1.16666663, 1. , 0.875 , 0.77777779, 0.69999999, 0.63636363, 0.58333331, 0.53846157, 0.5 , 0.46666667, 0.4375 , 0.41176471, 0.3888889 , 0.36842105, 0.34999999], [ 8. , 4. , 2.66666675, 2. , 1.60000002, 1.33333337, 1.14285719, 1. , 0.8888889 , 0.80000001, 0.72727275, 0.66666669, 0.61538464, 0.5714286 , 0.53333336, 0.5 , 0.47058824, 0.44444445, 0.42105263, 0.40000001], [ 9. , 4.5 , 3. , 2.25 , 1.79999995, 1.5 , 1.28571427, 1.125 , 1. , 0.89999998, 0.81818181, 0.75 , 0.69230771, 0.64285713, 0.60000002, 0.5625 , 0.52941179, 0.5 , 0.47368422, 0.44999999], [ 10. , 5. , 3.33333325, 2.5 , 2. , 1.66666663, 1.42857146, 1.25 , 1.11111116, 1. , 0.90909094, 0.83333331, 0.76923078, 0.71428573, 0.66666669, 0.625 , 0.58823532, 0.55555558, 0.52631581, 0.5 ]]) >>> p array([[ 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21.], [ 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22.], [ 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23.], [ 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24.], [ 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25.], [ 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26.], [ 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27.], [ 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28.], [ 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29.], [ 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30.]]) Note that you have to provide the shape information for ux and uy because fortran writes them together as a stream of 400 numbers. -Neil

10 10

Why does fancy indexing work like this?
by Aaron Meurer 24 Aug '20

24 Aug '20

Why does fancy indexing have this behavior? >>> a = np.empty((0, 1, 2)) >>> b = np.empty((1, 1, 2)) >>> a[np.array([10, 10])] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index 10 is out of bounds for axis 0 with size 0 >>> a[:, np.array([10, 10])] array([], shape=(0, 2, 2), dtype=float64) >>> a[:, :, np.array([10, 10])] array([], shape=(0, 1, 2), dtype=float64) >>> b[np.array([10, 10])] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index 10 is out of bounds for axis 0 with size 1 >>> b[:, np.array([10, 10])] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index 10 is out of bounds for axis 1 with size 1 >>> b[:, :, np.array([10, 10])] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index 10 is out of bounds for axis 2 with size 2 As far as I can tell, the behavior is that if an array has a 0 dimension and an integer array index indexes an axis that isn't 0, there are no bounds checks. Why does it do this? It seems to be inconsistent with the behavior of shape () fancy indices (integer indices). I couldn't find any reference to this behavior in https://numpy.org/doc/stable/reference/arrays.indexing.html. Aaron Meurer

2 12

What is up with raw boolean indices (like a[False])?
by Aaron Meurer 21 Aug '20

21 Aug '20

I've been trying to figure out this behavior. It doesn't seem to be documented at https://numpy.org/doc/stable/reference/arrays.indexing.html >>> a = np.empty((2, 3)) >>> a.shape (2, 5) >>> a[True].shape (1, 2, 5) >>> a[False].shape (0, 2, 5) It seems like indexing with a raw boolean (True or False) adds an axis with a dimension 1 or 0, resp. Except it only works once: >>> a[:,False] array([], shape=(2, 0, 3), dtype=float64) >>> a[:,False, False] array([], shape=(2, 0, 3), dtype=float64) >>> a[:,False,True].shape (2, 0, 3) >>> a[:,True,False].shape (2, 0, 3) The docs say "A single boolean index array is practically identical to x[obj.nonzero()]". I have a hard time seeing this as an extension of that, since indexing by `np.nonzero(False)` or `np.nonzero(True)` *replaces* the given axis. >>> a[np.nonzero(True)].shape (1, 3) >>> a[np.nonzero(False)].shape (0, 3) I think at best this behavior should be documented. I'm trying to understand the motivation for it, or if it's even intentional. And in particular, why do multiple boolean indices not insert multiple axes? It would actually be useful to be able to generically add length 0 axes using an index, similar to how `newaxis` adds a length 1 axis. Aaron Meurer

2 11

a summary function to get a quick glimpse on the contents of a numpy array
by Peter Steinbach 01 Aug '20

01 Aug '20

Dear numpy devs and interested readers, as a day-to-day user, it occurred to me that having a quick look into the contents and extents of arrays is well doable with numpy. numpy offers a rich set of methods for this. However, very often I oversee myself and others that one just wants to see if the values of an array have a certain min/max or mean or how wide the range of values are. I hence sat down to write a summary function that returns a string of hand-packed summary statistics for a quick inspection. I propose to include it into numpy and would love to have your feedback on this idea before I submit a PR. Here is the core functionality: Examples -------- >>> a = np.random.normal(size=20) >>> print(summary(a)) min 25perc mean stdev median 75perc max -2.289870 -2.265757 -0.083213 1.115033 -0.162885 -2.217532 1.639802 >>> a = np.reshape(a, newshape=(4,5)) >>> print(summary(a,axis=1)) min 25perc mean stdev median 75perc max 0 -0.976279 -0.974090 0.293003 1.009383 0.466814 -0.969712 1.519695 1 -0.468854 -0.467739 0.184139 0.649378 -0.036762 -0.465510 1.303144 2 -2.289870 -2.276455 -0.324450 1.230031 -0.289008 -2.249625 1.111107 3 -1.782239 -1.777304 -0.485546 1.259598 -1.236190 -1.767434 1.639802 So you see, it is merely a tiny helper function that can aid practitioners and data scientists to get a quick insight on what an array contains. first off, here is the code: https://github.com/psteinb/numpy/blob/summary-function/numpy/lib/utils.py#L… I put it there as I am not sure at this point, if the community would appreciate such a function or not. Judging from the tests, lib/utils.py appears to a be place for undocumented functions. So to resolve this and prepare a proper PR, please let me know where this summary function could reside! Second, please give me your thoughts on the summary function's output? Should the number of digits be configurable? Should the columns be configurable? Is is ok to honor the axis parameter which is found in so many numpy functions? Last but not least, let me stress that this is my first time contribution to numpy. I love the library and would like to contribute something back. So bear with me, if my code violates best practices in your community for now. I'll bite my teeth into the formalities of a github PR once I get support from the community and the core devs. I think that a summary function would be a valuable addition to numpy! Best, Peter

2 1

Re: [Numpy-discussion] Documentation Team meeting - Monday August 3
by Melissa Mendonça 31 Jul '20

31 Jul '20

Thanks, you too! On Fri, Jul 31, 2020 at 12:21 PM olzhas robo.ai <o(a)roboai.net> wrote: > Thank you, Melissa! Have a great weekend :) > > On Fri, 31 Jul 2020 at 13:10, Melissa Mendonça <melissawm(a)gmail.com> > wrote: > >> Hi all! >> >> This is a reminder that our next Documentation Team meeting will be on *Monday, >> August 3* at 3PM UTC**. If you wish to join on Zoom, you need to use >> this link >> >> https://zoom.us/j/420005230 >> >> Here's the permanent hackmd document with the meeting notes (still being >> updated in the next few days!): >> >> https://hackmd.io/oB_boakvRqKR-_2jRV-Qjg >> <https://www.google.com/url?q=https%3A%2F%2Fhackmd.io%2FoB_boakvRqKR-_2jRV-Q…> >> >> Hope to see you around! >> >> ** You can click this link to get the correct time at your timezone: >> https://www.timeanddate.com/worldclock/fixedtime.html?msg=NumPy+Documentati… >> >> *** You can add the NumPy community calendar to your google calendar by >> clicking this link: >> https://calendar.google.com/calendar/r?cid=YmVya2VsZXkuZWR1X2lla2dwaWdtMjMy… >> >> - Melissa >> >

1 0

NumPy Development Meeting Today - Triage Focus
by Warren Weckesser 29 Jul '20

29 Jul '20

Hi all, Sorry for the short notice--Sebastian is off this week, and the rest of us forgot to send the email reminder. Our bi-weekly triage-focused NumPy development meeting is in 10 minutes (today, Wednesday, July 29th, at 11 am Pacific Time (18:00 UTC)). Everyone is invited to join in and edit the work-in-progress meeting topics and notes: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg I encourage everyone to notify us of issues or PRs that you feel should be prioritized or simply discussed briefly. Just comment on it so we can label it, or add your PR/issue to this weeks topics for discussion. Best regards Warren

1 0