Mailman 3 August 2020 - NumPy-Discussion

NumPy Development Meeting Today - Triage Focus
by Sebastian Berg 25 Aug '20

25 Aug '20

Hi all, Our bi-weekly triage-focused NumPy development meeting is tomorrow (Wednesday, August 26th) at 11 am Pacific Time (18:00 UTC). Everyone is invited to join in and edit the work-in-progress meeting topics and notes: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg I encourage everyone to notify us of issues or PRs that you feel should be prioritized or simply discussed briefly. Just comment on it so we can label it, or add your PR/issue to this weeks topics for discussion. Best regards Sebastian

1 0

Why does fancy indexing work like this?
by Aaron Meurer 24 Aug '20

24 Aug '20

Why does fancy indexing have this behavior? >>> a = np.empty((0, 1, 2)) >>> b = np.empty((1, 1, 2)) >>> a[np.array([10, 10])] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index 10 is out of bounds for axis 0 with size 0 >>> a[:, np.array([10, 10])] array([], shape=(0, 2, 2), dtype=float64) >>> a[:, :, np.array([10, 10])] array([], shape=(0, 1, 2), dtype=float64) >>> b[np.array([10, 10])] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index 10 is out of bounds for axis 0 with size 1 >>> b[:, np.array([10, 10])] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index 10 is out of bounds for axis 1 with size 1 >>> b[:, :, np.array([10, 10])] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index 10 is out of bounds for axis 2 with size 2 As far as I can tell, the behavior is that if an array has a 0 dimension and an integer array index indexes an axis that isn't 0, there are no bounds checks. Why does it do this? It seems to be inconsistent with the behavior of shape () fancy indices (integer indices). I couldn't find any reference to this behavior in https://numpy.org/doc/stable/reference/arrays.indexing.html. Aaron Meurer

2 12

Migrating code to eliminate references to numpy/core/include/numpy/npy_3kcompat.h (?)
by Otto Hirr 22 Aug '20

22 Aug '20

Greetings, tl;dr: Need to remove npy_3kcompat.h from any source code location since Python2 is no longer supported. Background: Ran into an error after compiling various apps regarding npy_PyErr_ChainExceptionsCause() being undefined at runtime. Poking around shows that this is related to npy_3kcompat.h, which seems to be for Python2 compatibility. (There is a differently named counterpart for Python3.) This seems to be used in a couple of numpy/core/src/multiarray files. Seems that since Python2 is no longer supported, cleaning up cruft of P2 will improve code and should eliminate the problem I ran into, (or else move an error report to something else...) Is this reasonable to eliminate? Regards, ..Otto (Still getting my feet wet in numpy, scipy, etc.)

2 1

What is up with raw boolean indices (like a[False])?
by Aaron Meurer 21 Aug '20

21 Aug '20

I've been trying to figure out this behavior. It doesn't seem to be documented at https://numpy.org/doc/stable/reference/arrays.indexing.html >>> a = np.empty((2, 3)) >>> a.shape (2, 5) >>> a[True].shape (1, 2, 5) >>> a[False].shape (0, 2, 5) It seems like indexing with a raw boolean (True or False) adds an axis with a dimension 1 or 0, resp. Except it only works once: >>> a[:,False] array([], shape=(2, 0, 3), dtype=float64) >>> a[:,False, False] array([], shape=(2, 0, 3), dtype=float64) >>> a[:,False,True].shape (2, 0, 3) >>> a[:,True,False].shape (2, 0, 3) The docs say "A single boolean index array is practically identical to x[obj.nonzero()]". I have a hard time seeing this as an extension of that, since indexing by `np.nonzero(False)` or `np.nonzero(True)` *replaces* the given axis. >>> a[np.nonzero(True)].shape (1, 3) >>> a[np.nonzero(False)].shape (0, 3) I think at best this behavior should be documented. I'm trying to understand the motivation for it, or if it's even intentional. And in particular, why do multiple boolean indices not insert multiple axes? It would actually be useful to be able to generically add length 0 axes using an index, similar to how `newaxis` adds a length 1 axis. Aaron Meurer

2 11

[Release] PyData/Sparse 0.11.0
by Hameer Abbasi 18 Aug '20

18 Aug '20

Hello, I’m happy to announce the release of PyData/Sparse 0.11.0, available to download via pip and conda-forge. PyData/Sparse is a library that provides sparse N-dimensional arrays for the PyData ecosystem. The official website and documentation is available at: https://sparse.pydata.org The sources and bug tracker: https://github.com/pydata/sparse The changelog for this release can be viewed at: https://sparse.pydata.org/en/0.11.0/changelog.html Best Regards, Hameer Abbasi -- Sent from Canary (https://canarymail.io)

1 0

NumPy Community Meeting Wednesday
by Sebastian Berg 18 Aug '20

18 Aug '20

Hi all, There will be a NumPy Community meeting Wednesday Agust 19th at 1pm Pacific Time (20:00 UTC). Everyone is invited and encouraged to join in and edit the work-in-progress meeting topics and notes at: https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both Best wishes Sebastian

1 0

Announcing 2020 Google Season of Docs Technical Writers for NumPy
by Melissa Mendonça 17 Aug '20

17 Aug '20

Hello all, I'm pleased to announce that NumPy was awarded two slots in the Google Season of Docs program (you can see the full results here: https://developers.google.com/season-of-docs/docs/participants). The selected projects are - "NumPy Documentation for Community Education", by Ryan Cooper (Proposal: https://developers.google.com/season-of-docs/docs/participants/project-nump… ) <https://developers.google.com/season-of-docs/docs/participants/project-nump…:> - "High level restructuring and end user focus", by kubedoc (Proposal: https://developers.google.com/season-of-docs/docs/participants/project-nump… ) We appreciate all projects that were submitted and thank all participants for their efforts in putting together their proposals. Also, if you wish to contribute documentation to NumPy on a volunteer basis, you are welcome to do so! Cheers, - Melissa

1 0

Experimental `like=` attribute for array creation functions
by Sebastian Berg 17 Aug '20

17 Aug '20

Hi all, as a heads up that Peter Entschev has a PR open to add `like=` to most array creation functions, my current plan is to merge it soon as a preliminary API and bring it up again before the actual release (in a few months). This allows overriding for array-likes, e.g. it will allow: arr = np.asarray([3], like=dask_array) type(arr) is dask.array.Array This was proposed in NEP 35: https://numpy.org/neps/nep-0035-array-creation-dispatch-with-array-function… Although that has not been accepted as of now, the PR is: https://github.com/numpy/numpy/pull/16935 This was discussed in a smaller group, and is an attempt to see how we can make the array-function protocol viable to allow packages such as sklearn to work with non-NumPy arrays. As of now, this would be experimental and can revisit it before the actual NumPy release. We should probably discuss accepting NEP 35 more. At this time, I hope that we can put in the functionality to facilitate this discussion, rather the other way around. If anyone feels nervous about this step, I would be happy to document that we will not include it in the next release unless the NEP is accepted first, or at least hide it behind an environment variable. Cheers, Sebastian

7 19

NEP Procedure Discussion
by Peter Andreas Entschev 16 Aug '20

16 Aug '20

Hi all, During the discussion about NEP-35, there have been lots of discussions around the NEP process itself. In the interest of allowing people who are mostly interested in this discussion and to avoid drifting so much off-topic in that thread, I'm starting this new thread to discuss the NEP procedure. A few questions that have been raised so far: - Is the NEP Template [1] a guideline to be strictly followed or a suggestion for authors? - Who should decide when a NEP is sufficiently clear? - Should a NEP PR be merged at all until it's sufficiently clear or should it only be merged even in Draft state only after it's sufficiently clear? - What parts of the NEP are necessary to be clear for everyone? Just Abstract? Motivation and Scope? Everything, including the real technical details of implementation? - Would it be possible to have proof-readers -- preferably people who are not at all involved in the NEP's topic? Please feel free to comment on that and add any major points I might have missed. Best, Peter [1] https://github.com/numpy/numpy/blob/master/doc/neps/nep-template.rst

4 3

Use of booleans in slices
by Aaron Meurer 14 Aug '20

14 Aug '20

I noticed that np.bool_.__index__() gives a DeprecationWarning >>> np.bool_(True).__index__() __main__:1: DeprecationWarning: In future, it will be an error for 'np.bool_' scalars to be interpreted as an index 1 This is good, because booleans don't actually act like integers in indexing contexts. However, raw Python bools also allow __index__() >>> True.__index__() 1 A consequence of this is that NumPy slices allow booleans, as long as they are the Python type (if you use the NumPy bool_ type you get the deprecation warning). >>> a = np.arange(10) >>> a[True:] array([1, 2, 3, 4, 5, 6, 7, 8, 9]) Should this behavior also be considered deprecated? Presumably deprecating bool.__index__() in Python is a no-go, but it could be deprecated in NumPy contexts (in the pure Python collections, booleans don't have a special indexing meaning anyway). Interestingly, places that use a shape don't allow booleans (I guess they don't necessarily use __index__()?) >>> np.empty((True,)) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: an integer is required Aaron Meurer

2 1