Mailman 3 May 2022 - NumPy-Discussion

mixed mode arithmetic
by Neal Becker July 11, 2023

July 11, 2023

I've been browsing the numpy source. I'm wondering about mixed-mode arithmetic on arrays. I believe the way numpy handles this is that it never does mixed arithmetic, but instead converts arrays to a common type. Arguably, that might be efficient for a mix of say, double and float. Maybe not. But for a mix of complex and a scalar type (say, CDouble * Double), it's clearly suboptimal in efficiency. So, do I understand this correctly? If so, is that something we should improve?

4 6

Invalid value encoutered : how to prevent numpy.where to do this?
by Eric Emsellem Feb. 18, 2023

Feb. 18, 2023

Dear all, I have a code using lots of "numpy.where" to make some constrained calculations as in: data = arange(10) result = np.where(data == 0, 0., 1./data) # or data1 = arange(10) data2 = arange(10)+1.0 result = np.where(data1 > data2, np.sqrt(data1-data2), np.sqrt(data2-data2)) which then produces warnings like: /usr/bin/ipython:1: RuntimeWarning: invalid value encountered in sqrt or for the first example: /usr/bin/ipython:1: RuntimeWarning: divide by zero encountered in divide How do I avoid these messages to appear? I know that I could in principle use numpy.seterr. However, I do NOT want to remove these warnings for other potential divide/multiply/sqrt etc errors. Only when I am using a "where", to in fact avoid such warnings! Note that the warnings only happen once, but since I am going to release that code, I would like to avoid the user to get such messages which are irrelevant here (because I am testing, with the where, when NOT to divide by zero or take a sqrt of a negative number). thanks! Eric

5 4

Documentation Team meeting - Monday June 8th
by Melissa Mendonça Dec. 3, 2022

Dec. 3, 2022

Hi all! A reminder that on Monday, June 8, we have another documentation team meeting at 3PM UTC**. If you wish to join on Zoom, you need to use this link https://zoom.us/j/420005230 Here's the permanent hackmd document with the meeting notes: https://hackmd.io/oB_boakvRqKR-_2jRV-Qjg <https://www.google.com/url?q=https%3A%2F%2Fhackmd.io%2FoB_boakvRqKR-_2jRV-Q…> Hope to see you around (especially if you want to introduce yourself or discuss ideas for Google Season of Docs). ** You can click this link to get the correct time at your timezone: https://www.timeanddate.com/worldclock/fixedtime.html?msg=NumPy+Documentati… - Melissa

6 74

An extension of the .npy file format
by Michael Siebert Aug. 25, 2022

Aug. 25, 2022

Dear all, originally, I have planned to make an extension of the .npy file format a dedicated follow-up pull request, but I have upgraded my current request instead, since it was not as difficult to implement as I initially thought and probably a more straight-forward solution: https://github.com/numpy/numpy/pull/20321/ What is this pull request about? It is about appending to Numpy .npy files. Why? I see two main use cases: 1. creating .npy files larger than the main memory. They can, once finished, be loaded as memory maps 2. creating binary log files, which can be processed very efficiently without parsing Are there not other good file formats to do this? Theoretically yes, but practically they can be pretty complex and with very little tweaking .npy could do efficient appending too. Use case 1 is already covered by the Pip/Conda package npy-append-array I have created and getting the functionality directly into Numpy was the original goal of the pull request. This would have been possible without introducing a new file format version, just by adding some spare space in the header. During the pull request discussion it turned out that rewriting the header after each append would be desirable in case the writing program crashes to minimize data loss. Use case 2 however would highly profit from a new file format version as it would make rewriting the header unnecessary: since efficient appending can only take place along one axis, setting shape[-1] = -1 in case of Fortran order or shape[0] = -1 otherwise (default) in the .npy header on file creation could indicate that the array size is determined by the file size: when np.load (typically with memory mapping on) gets called, it constructs the ndarray with the actual shape by replacing the -1 in the constructor call. Otherwise, the header is not modified anymore, neither on append nor on file write finish. Concurrent appends to a single file would not be advisable and should be channeled through a single AppendArray instance. Concurrent reads while writes take place however should work relatively smooth: every time an np.load (ideally with mmap) is called, the ndarray would provide access to all data written until that time. Currently, my pull request provides: 1. A definition of .npy version 4.0 that supports -1 in the shape 2. implementations for fortran order and non-fortran order (default) including test cases 3. Updated np.load 4. The AppendArray class that does the actual appending Although there is a certain hassle with introducing a new .npy version, the changes themselves are very small. I could also implement a fallback mode for older Numpy installations, if someone is interested. What do you think about such a feature, would it make sense? Anyone available for some more code review? Best from Berlin, Michael PS thank you so far, I could improve my npy-append-array module as well and from what I have seen so far the Numpy code readability exceeded my already high expectations.

5 7

NumPy community meeting
by Inessa Pawson July 31, 2022

July 31, 2022

The next NumPy community meeting will be held this Wednesday, May 25th at 18:00 (6 pm) UTC. Join us via Zoom: https://berkeley.zoom.us/j/762261535 Everyone is welcome and encouraged to attend. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy

1 5

copy="never" discussion and no deprecation cycle?
by Sebastian Berg July 21, 2022

July 21, 2022

Hi all, (sorry for the length, details/discussion below) On the triage call, there seemed a preference to just try to skip the deprecation and introduce `copy="never"`, `copy="if_needed"`, and `copy="always"` (i.e. string options for the `copy` keyword argument). Strictly speaking, this is against the typical policy (one year of warning/errors). But nobody could think of a reasonable chance that anyone actually uses it. (For me just "policy" will be enough of an argument to just take it slow.) BUT: If nobody has *any* concerns at all, I think we may just end up introducing the change right away. The PR is: https://github.com/numpy/numpy/pull/19173 ## The Feature There is the idea to add `copy=never` (or similar). This would modify the existing `copy` argument to make it a 3-way decision: * `copy=always` or `copy=True` to force a copy * `copy=if_needed` or `copy=False` to prefer no-copy behavior * `copy=never` to error when no-copy behavior is not possible (this ensures that a view is returned) this would affect the functions: * np.array(object, copy=...) * arr.astype(new_dtype, copy=...) * np.reshape(arr, new_shape, copy=...), and the method arr.reshape() * np.meshgrid and possibly Where `reshape` currently does not have the option and would benefit by allowing for `arr.reshape(-1, copy=never)`, which would guarantee a view. ## The Options We have three options that are currently being discussed: 1. We introduce a new `np.CopyMode` or `np.<something>.Copy` Enum with values `np.CopyMode.NEVER`, `np.CopyMode.IF_NEEDED`, and `np.CopyMode.ALWAYS` * Plus: No compatibility concerns * Downside(?): This would be a first in NumPy, and is untypical API due to that. 2. We introduce `copy="never"`, `copy="if_needed"` and `copy="always"` as strings (all other strings will be a `TypeError`): * Problem: `copy="never"` currently means `copy=True` (the opposite) Which means new code has to take care when it may run on older NumPy versions. And in theory could make old code return the wrong thing. * Plus: Strings are the typical for options in NumPy currently. 3. Same as 2. But we take it very slow: Make strings an error right now and only introduce the new options after two releases as per typical deprecation policy. ## Discussion We discussed it briefly today in the triage call and we were leaning towards strings. I was honestly expecting to converge to option 3 to avoid compatibility issues (mainly surprises with `copy="never"` on older versions). But considering how weird it is to currently pass `copy="never"`, the question was whether we should not change it with a release note. The probability of someone currently passing exactly one of those three (and no other) strings seems exceedingly small. Personally, I don't have a much of an opinion. But if *nobody* voices any concern about just changing the meaning of the string inputs, I think the current default may be to just do it. Cheers, Sebastian

12 28

NumPy Newcomers' Hour – Thursday, March 24th
by Inessa Pawson July 10, 2022

July 10, 2022

Our next Newcomers' Hour will be held tomorrow, March 24th, at 4 pm UTC. We have no agenda this time. Stop by to ask questions or just to say hi. Join the meeting via Zoom: https://us02web.zoom.us/j/87192457898 Cheers, Inessa Inessa Pawson NumPy Contributor Experience Lead

1 4

Changes to `np.loadtxt`
by Sebastian Berg June 29, 2022

June 29, 2022

Hi all, just a brief heads up that: https://github.com/numpy/numpy/pull/20580 is now merged. This moves `np.loadtxt` to C. Mainly making it much faster. There are also some other improvements and changes though: * It now supports `quotechar='"'` to support Excel dialect CSV. * Parsing some numbers is stricter (e.g. removed support for `_` or hex float parsing by default). * `max_rows` now actually counts rows and not lines. A warning is given if this makes a difference (blank lines). * Some exception will change, parsing failures now (almost) always give an informative `ValueError`. * `converters=callable` is now valid to provide a single converter for all columns. Please test, and let us know if there is any issue or followup you would like to see. We do have possible followups planned * Consider deprecating the `encoding="bytes"` default which exists for Python 2 compatibility. * Consider renaming `skip_rows` to the more precise `skip_lines`. Moving to C unlocks possible further improvement, such as full `csv.Dialect` support. We do not have this on the roadmap, but such contributions are possible now. Similarly, it should be possible to rewrite `genfromtxt` based on this work. Cheers, Sebastian

1 1

Numpy binary wheels and CI for win/arm64 platform
by Niyas Sait June 22, 2022

June 22, 2022

Hello, Numpy v1.21.2 <https://github.com/numpy/numpy/releases/tag/v1.21.2> added support for windows/arm64 platforms but we still don't have any systems in place to produce binary wheels or test win/arm64 packages. I think it will be good to start looking into this. CPython has an official buildbot worker running for win/arm64 and official python support for the platform will be available from the 3.11 release. It is not yet clear to me how the build and CI system for numpy is deployed and how to enable support for a new platform like win/arm64. One of the main issues in supporting win/arm64 build would be due to the lack of win/arm64 VMs available on the cloud. But I see we have been producing binary wheels for Apple M1 platforms on pypi and conda repository for some time which also lacks the cloud VM support. I think we could take some learnings from Apple M1 support and look at how a similar strategy can be used for win/arm64. I would like to hear if anyone has any thoughts on this topic. Also, any pointers to understand numpy wheel generation and CI flow for similar platforms would be helpful as well. Thanks, Niyas

4 9

next NumPy Newcomers Hour
by Inessa Pawson June 5, 2022

June 5, 2022

The next NumPy Newcomers Hour will be held this Thursday, June 2nd at 4 pm UTC. Sebastian Berg, one of the NumPy core developers, will talk about deprecations in NumPy. Join us via Zoom: https://us02web.zoom.us/j/87192457898 Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy https://numpy.org/ Twitter: @inessapawson

1 1

overhead of numpy scalars
by Pieter Eendebak June 1, 2022

June 1, 2022

Hi everyone, Operations such as np.sqrt or np.cos on numpy scalars are much slower than operations on an array of size 1 (benchmark below). The reason seems to be that a numpy scalar (e.g. np.float64(1.1)) is converted internally to an array, an array is created for the result of the operation and the result is converted back to a scalar (so three object constructions in total). Operating directly on an array creates an array for the result, which is returned (so one object construction in total). Has anyone looked into improving this performance before or has ideas how to tackle this? With kind regards, Pieter Eendebak Example benchmark: import numpy as np from numpy import sqrt import time v=np.array([1.1]) t0=time.perf_counter() x=np.float64(v) for kk in range(1_000_000): w1=sqrt(x) dt1=time.perf_counter()-t0 print(dt1) t0=time.perf_counter() x=v for kk in range(1_000_000): w2=sqrt(x) dt2=time.perf_counter()-t0 print(dt2) print(f'factor {dt1/dt2:.2f} faster') assert(float(w1)==float(w2)) Results in 1.0878291579974757 0.5115369699997245 factor 2.13 faster

2 2

NumPy Development Meeting Wednesday - Triage Focus
by Sebastian Berg May 30, 2022

May 30, 2022

Hi all, Our bi-weekly triage-focused NumPy development meeting is Wednesday, June 1st at 16:00 UTC (9:00am Pacific Time). Everyone is invited to join in and edit the work-in-progress meeting topics and notes: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg I encourage everyone to notify us of issues or PRs that you feel should be prioritized, discussed, or reviewed. A calendar to can be found at: https://scientific-python.org/calendars/ please make sure to not import the static `ics` but the url. Best regards Sebastian

1 0

NumPy 1.23.0rc2 released
by Charles R Harris May 30, 2022

May 30, 2022

Hi All, On behalf of the NumPy team, I'm pleased to announce the release of NumPy 1.23.0rc2. The NumPy 1.23.0 release continues the ongoing work to improve the handling and promotion of dtypes, increase the execution speed, clarify the documentation, and expire old deprecations. The highlights are: - Implementation of ``loadtxt`` in C, greatly improving its performance. - Exposing DLPack at the Python level for easy data exchange. - Changes to the promotion and comparisons of structured dtypes. - Improvements to f2py. The Python versions supported in this release are 3.8-3.10, 3.11 will be supported when it comes out. Note that 32 bit wheels are only provided for Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit OpenBLAS. Wheels can be downloaded from PyPI <https://pypi.org/project/numpy/1.23.0rc2/>; source archives, release notes, and wheel hashes are available on Github <https://github.com/numpy/numpy/releases/tag/v1.23.0rc2>. *Contributors* A total of 150 people contributed to this release. People with a "+" by their names contributed a patch for the first time. - @DWesl - @GalaxySnail + - @code-review-doctor + - @h-vetinari - Aaron Meurer - Alexander Shadchin - Alexandre de Siqueira - Allan Haldane - Amrit Krishnan - Andrei Batomunkuev - Andrew J. Hesford + - Andrew Murray + - Andrey Andreyevich Bienkowski + - André Elimelek de Weber + - Andy Wharton + - Arryan Singh - Arushi Sharma - Bas van Beek - Bharat Raghunathan - Bhavuk Kalra + - Brigitta Sipőcz - Brénainn Woodsend + - Burlen Loring + - Caio Agiani + - Charles Harris - Chiara Marmo - Cornelius Roemer + - Dahyun Kim + - Damien Caliste - David Prosin + - Denis Laxalde - Developer-Ecosystem-Engineering - Devin Shanahan + - Diego Wang + - Dimitri Papadopoulos Orfanos - Ding Liu + - Diwakar Gupta + - Don Kirkby + - Emma Simon + - Eric Wieser - Evan Miller + - Evgeni Burovski - Evgeny Posenitskiy + - Ewout ter Hoeven + - Felix Divo - Francesco Andreuzzi + - Ganesh Kathiresan - Gaëtan de Menten - Geoffrey Gunter + - Hans Meine - Harsh Mishra + - Henry Schreiner - Hood Chatham + - Ilhan Polat - Inessa Pawson - Isuru Fernando - Ivan Gonzalez + - Ivan Meleshko + - Ivan Yashchuk + - Janus Heide + - Jarrod Millman - Jason Thai + - Jeremy Volkman + - Jesús Carrete Montaña + - Jhong-Ken Chen (陳仲肯) + - John Kirkham - John-Mark Gurney + - Jonathan Deng + - Joseph Fox-Rabinovitz - Jouke Witteveen + - Junyan Ou + - Jérôme Richard + - Kassian Sun + - Kazuki Sakamoto + - Kenichi Maehashi - Kevin Sheppard - Kilian Lieret + - Kushal Beniwal + - Leo Singer - Logan Thomas + - Lorenzo Mammana + - Margret Pax - Mariusz Felisiak + - Markus Mohrhard + - Mars Lee - Marten van Kerkwijk - Masamichi Hosoda + - Matthew Barber - Matthew Brett - Matthias Bussonnier - Matthieu Darbois - Matti Picus - Melissa Weber Mendonça - Michael Burkhart + - Morteza Mirzai + - Motahhar Mokf + - Muataz Attaia + - Muhammad Motawe + - Mukulika Pahari - Márton Gunyhó + - Namami Shanker + - Nihaal Sangha + - Niyas Sait - Omid Rajaei + - Oscar Gustafsson + - Ovee Jawdekar + - P. L. Lim + - Pamphile Roy + - Pantelis Antonoudiou + - Pearu Peterson - Peter Andreas Entschev - Peter Hawkins - Pierre de Buyl - Pieter Eendebak + - Pradipta Ghosh + - Rafael Cardoso Fernandes Sousa + - Raghuveer Devulapalli - Ralf Gommers - Raphael Kruse - Raúl Montón Pinillos - Robert Kern - Rohit Goswami - Ross Barnowski - Ruben Garcia + - Sadie Louise Bartholomew + - Saswat Das + - Sayed Adel - Sebastian Berg - Serge Guelton - Simon Surland Andersen + - Siyabend Ürün + - Somasree Majumder + - Soumya + - Stefan van der Walt - Stefano Miccoli + - Stephan Hoyer - Stephen Worsley + - Tania Allard - Thomas Duvernay + - Thomas Green + - Thomas J. Fan - Thomas Li + - Tim Hoffmann - Ting Sun + - Tirth Patel - Toshiki Kataoka - Tyler Reddy - Warren Weckesser - Yang Hau - Yoon, Jee Seok + Cheers, Charles Harris Reply allReplyForward <https://drive.google.com/u/0/settings/storage?hl=en&utm_medium=web&utm_sour…> <https://www.google.com/intl/en/policies/terms/> <https://www.google.com/intl/en/policies/privacy/> <https://www.google.com/gmail/about/policy/>

1 0

numpy.sort: Ability to reverse-sort a subset of columns?
by James Anderson May 27, 2022

May 27, 2022

Near as I can tell the numpy.sort function doesn’t support requesting a descending sort on specific columns. (Docs: https://numpy.org/doc/stable/reference/generated/numpy.sort.html) I wanted to check I’m not missing how this is properly done or if should I file this as a feature request. There are many reasons one would need to choose sorting in ascending/descending order on a column by column basis. My first thought is another parameter to sort which is an array of bools to indicate sort order. An alternative is to prepend ‘-‘ to column names that should be sorted in descending order. However the second idea may conflict with people's names for columns. Thanks, James Anderson Example: In my case I have an sweep line algorithm that needs to walk an ndarray with the following dtype row-by-row: active_dtype = np.dtype([('Value', 'f4'), ('ID', 'u8'), ('Active', 'u1')]) I need to sort one column (Value) in ascending order, but another column (Active) in descending order. (To handle the degenerate case where a rectangle has a zero width/height). For full context the code below takes a rectangle array (rect_array) and returns sorted arrays indicating where rectangle boundaries start and stop on each axis. In this example one can invert the meaning of ‘Active’ to work around the lack of API support. The intent is just to illustrate how I was hoping I had expected to use numpy.sort. x_sweep_array = np.empty(len(rect_array) * 2, dtype=cls.active_dtype) y_sweep_array = np.empty(len(rect_array) * 2, dtype=cls.active_dtype) for i, rect in enumerate(rect_array): x_sweep_array[i * 2] = (rect['MinX'], rect['ID'], True) x_sweep_array[(i * 2) + 1] = (rect['MaxX'], rect['ID'], False) y_sweep_array[i * 2] = (rect['MinY'], rect['ID'], True) y_sweep_array[(i * 2) + 1] = (rect['MaxY'], rect['ID'], False) x_sweep_array = np.sort(x_sweep_array, order=('Value', 'Active')) # <- Cannot reverse sort Active y_sweep_array = np.sort(y_sweep_array, order=('Value', 'Active')) return (x_sweep_array, y_sweep_array)

1 0

NEP 29 and the faster Python release cadence
by Aaron Meurer May 27, 2022

May 27, 2022

Was the faster CPython release cadence (PEP 602 https://peps.python.org/pep-0602/) ever discussed in relation to NEP 29 (https://numpy.org/neps/nep-0029-deprecation_policy.html)? NEP 29 currently says: "The current Python release cadence is 18 months so a 42 month window ensures that there will always be at least two minor versions of Python in the window." However, as of PEP 602, the release cadence of CPython is now 12 months, not 18 months. This schedule now means supporting 3-4 versions of Python at once, rather than 2-3 versions (for those who don't want to do the math, 48/18 is 2.3 and 48/12 is 3.5). PEP 602 started with Python 3.8, which was released in October, 2019. Python 3.8 is currently the oldest version supported in NEP 29, so we are about to enter the full cycle of the new cadence. In particular, when Python 3.11 is released in October of this year, the NEP 29 prescription will imply supporting 4 Python versions: 3.8, 3.9, 3.10, and 3.11, up until April, 2023 when 3.8 support would be dropped (and continuing like this: 3 versions between April and October; 4 versions between October and April). Furthermore, the NEP seems to imply that the exact CPython release schedule is not known: "The window is extended 6 months beyond the anticipated two-release interval for Python to provide resilience against small fluctuations / delays in its release schedule." I'm not sure how true this was before, but PEP 602 pretty clearly prescribes a Python release in October of every year. Delays presumably might still be possible, but I don't see why it's necessary to build that possibility into the default period in the NEP given that it should be an unusual situation. Personally I would prefer not ever supporting 4 simultaneous Python versions (I rather wish Python hadn't increased their cadence, but that ship has sailed). I don't know if it's possible to update the NEP to adjust for the new cadence. Even if it isn't, the text is clearly out of date and should be updated. I wasn't on this list back in 2019, so my apologies if this has already been discussed. Aaron Meurer

7 14

GSoC Student Announcement 2022
by Rohit Goswami May 27, 2022

May 27, 2022

NumPy as a sub-organization under the Python Software Foundation organization will be mentoring a summer student this year funded by the Google Summer of Code program. Ralf facilitated the application process and I will be the primary mentor this year (with Gagandeep for support). Please look forward to work done by our mentee, Namami Shankar who will be working on F2PY and in particular our front-end. His proposal entitled "Restructuring the F2PY frontend and replacing the build system" can be found here: https://summerofcode.withgoogle.com/media/user/a7128bebaf4c/proposal/bMbhEN…

3 2

NumPy 1.23.0rc1 released
by Charles R Harris May 26, 2022

May 26, 2022

Hi All, On behalf of the NumPy team, I'm pleased to announce the release of NumPy 1.23.0rc1. The NumPy 1.23.0 release continues the ongoing work to improve the handling and promotion of dtypes, increase the execution speed, clarify the documentation, and expire old deprecations. The highlights are: - Implementation of ``loadtxt`` in C, greatly improving its performance. - Exposing DLPack at the Python level for easy data exchange. - Changes to the promotion and comparisons of structured dtypes. - Improvements to f2py. The Python versions supported in this release are 3.8-3.10, 3.11 will be supported when it comes out. Note that 32 bit wheels are only provided for Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit OpenBLAS. Wheels can be downloaded from PyPI <https://pypi.org/project/numpy/1.23.0rc1/>; source archives, release notes, and wheel hashes are available on Github <https://github.com/numpy/numpy/releases/tag/v1.23.0rc1>. *Contributors* A total of 150 people contributed to this release. People with a "+" by their names contributed a patch for the first time. - @DWesl - @GalaxySnail + - @code-review-doctor + - @h-vetinari - Aaron Meurer - Alexander Shadchin - Alexandre de Siqueira - Allan Haldane - Amrit Krishnan - Andrei Batomunkuev - Andrew J. Hesford + - Andrew Murray + - Andrey Andreyevich Bienkowski + - André Elimelek de Weber + - Andy Wharton + - Arryan Singh - Arushi Sharma - Bas van Beek - Bharat Raghunathan - Bhavuk Kalra + - Brigitta Sipőcz - Brénainn Woodsend + - Burlen Loring + - Caio Agiani + - Charles Harris - Chiara Marmo - Cornelius Roemer + - Dahyun Kim + - Damien Caliste - David Prosin + - Denis Laxalde - Developer-Ecosystem-Engineering - Devin Shanahan + - Diego Wang + - Dimitri Papadopoulos Orfanos - Ding Liu + - Diwakar Gupta + - Don Kirkby + - Emma Simon + - Eric Wieser - Evan Miller + - Evgeni Burovski - Evgeny Posenitskiy + - Ewout ter Hoeven + - Felix Divo - Francesco Andreuzzi + - Ganesh Kathiresan - Gaëtan de Menten - Geoffrey Gunter + - Hans Meine - Harsh Mishra + - Henry Schreiner - Hood Chatham + - Ilhan Polat - Inessa Pawson - Isuru Fernando - Ivan Gonzalez + - Ivan Meleshko + - Ivan Yashchuk + - Janus Heide + - Jarrod Millman - Jason Thai + - Jeremy Volkman + - Jesús Carrete Montaña + - Jhong-Ken Chen (陳仲肯) + - John Kirkham - John-Mark Gurney + - Jonathan Deng + - Joseph Fox-Rabinovitz - Jouke Witteveen + - Junyan Ou + - Jérôme Richard + - Kassian Sun + - Kazuki Sakamoto + - Kenichi Maehashi - Kevin Sheppard - Kilian Lieret + - Kushal Beniwal + - Leo Singer - Logan Thomas + - Lorenzo Mammana + - Margret Pax - Mariusz Felisiak + - Markus Mohrhard + - Mars Lee - Marten van Kerkwijk - Masamichi Hosoda + - Matthew Barber - Matthew Brett - Matthias Bussonnier - Matthieu Darbois - Matti Picus - Melissa Weber Mendonça - Michael Burkhart + - Morteza Mirzai + - Motahhar Mokf + - Muataz Attaia + - Muhammad Motawe + - Mukulika Pahari - Márton Gunyhó + - Namami Shanker + - Nihaal Sangha + - Niyas Sait - Omid Rajaei + - Oscar Gustafsson + - Ovee Jawdekar + - P. L. Lim + - Pamphile Roy + - Pantelis Antonoudiou + - Pearu Peterson - Peter Andreas Entschev - Peter Hawkins - Pierre de Buyl - Pieter Eendebak + - Pradipta Ghosh + - Rafael Cardoso Fernandes Sousa + - Raghuveer Devulapalli - Ralf Gommers - Raphael Kruse - Raúl Montón Pinillos - Robert Kern - Rohit Goswami - Ross Barnowski - Ruben Garcia + - Sadie Louise Bartholomew + - Saswat Das + - Sayed Adel - Sebastian Berg - Serge Guelton - Simon Surland Andersen + - Siyabend Ürün + - Somasree Majumder + - Soumya + - Stefan van der Walt - Stefano Miccoli + - Stephan Hoyer - Stephen Worsley + - Tania Allard - Thomas Duvernay + - Thomas Green + - Thomas J. Fan - Thomas Li + - Tim Hoffmann - Ting Sun + - Tirth Patel - Toshiki Kataoka - Tyler Reddy - Warren Weckesser - Yang Hau - Yoon, Jee Seok + Cheers, Charles Harris

1 0

Enhancement for AArch64 SVE instruction set
by kawakami.k＠fujitsu.com May 26, 2022

May 26, 2022

Hello, I am working on speeding up NumPy with the AArch64 SVE instruction set. I could not find a numpy implementation for SVE. Is there already a test implementation or discussion about SVE support?

3 5

Dropping the pdf documentation.
by Charles R Harris May 25, 2022

May 25, 2022

Hi All, This is a proposal to drop the generation of pdf documentation and only generate the html version. This is a one way change due to the difficulty maintaining/fixing the pdf versions. See minimal discussion here <https://github.com/numpy/numpy/issues/21557#issuecomment-1133920412>. Chuck

11 21

normalization of Polynomial
by Pieter Eendebak May 24, 2022

May 24, 2022

Hi, The new Polynomial objects includes a domain and window. How can I normalize such an object to a standard domain and window? There is the .convert method, but it returns a strange format. A minimal example: import numpy as np p =np.polynomial.Polynomial([1, 1]) x=np.linspace(0, 3, 15); y=p(x) y[1]+=.1 q=np.polynomial.Polynomial.fit(x, y, 1) print(f'p: {repr(p)}') print(f'fitted q: {repr(q)}') print(f'fitted q: {str(q)}') qc=q.convert() print(f'q.convert(): str {qc}') print(f'q.convert(): {repr(qc)}') The output of repr(qc) is: Polynomial([Polynomial([1.02166667, 0.99 ], domain=[-1., 1.], window=[-1., 1.])], dtype=object, domain=[-1., 1.], window=[-1., 1.]) but I am looking for: Polynomial([1.02166667, 0.99 ], domain=[-1., 1.], window=[-1., 1.]) With kind regards, Pieter Eendebak

1 0

Maintenance/1.23.x is branched
by Charles R Harris May 23, 2022

May 23, 2022

Hi All, The 1.23.x maintenance branch has been made and main is now open for 1.24.0 development. Chuck

1 0

NumPy 1.22.4 released
by Charles R Harris May 21, 2022

May 21, 2022

Hi All, On behalf of the NumPy team, I'm pleased to announce the release of NumPy 1.22.4. NumPy 1.22.4 is a maintenance release that fixes bugs discovered after the 1.22.3 release. In addition, the wheels for this release are built using the recently released Cython 0.29.30, which should fix the reported problems with debugging <https://github.com/numpy/numpy/issues/21008>. The Python versions supported in this release are 3.8-3.10. Wheels can be downloaded from PyPI <https://pypi.org/project/numpy/1.22.4>; source archives, release notes, and wheel hashes are available on Github <https://github.com/numpy/numpy/releases/tag/v1.22.4>. Note that the Mac wheels are based on OS X 10.15 rather than 10.9 that was used in previous NumPy release cycles. *Contributors* A total of 12 people contributed to this release. People with a "+" by their names contributed a patch for the first time. - Alexander Shadchin - Bas van Beek - Charles Harris - Hood Chatham - Jarrod Millman - John-Mark Gurney + - Junyan Ou + - Mariusz Felisiak + - Ross Barnowski - Sebastian Berg - Serge Guelton - Stefan van der Walt *Pull requests merged* A total of 22 pull requests were merged for this release. - #21191: TYP, BUG: Fix ``np.lib.stride_tricks`` re-exported under the... - #21192: TST: Bump mypy from 0.931 to 0.940 - #21243: MAINT: Explicitly re-export the types in ``numpy._typing`` - #21245: MAINT: Specify sphinx, numpydoc versions for CI doc builds - #21275: BUG: Fix typos - #21277: ENH, BLD: Fix math feature detection for wasm - #21350: MAINT: Fix failing simd and cygwin tests. - #21438: MAINT: Fix failing Python 3.8 32-bit Windows test. - #21444: BUG: add linux guard per #21386 - #21445: BUG: Allow legacy dtypes to cast to datetime again - #21446: BUG: Make mmap handling safer in frombuffer - #21447: BUG: Stop using PyBytesObject.ob_shash deprecated in Python 3.11. - #21448: ENH: Introduce numpy.core.setup_common.NPY_CXX_FLAGS - #21472: BUG: Ensure compile errors are raised correctly - #21473: BUG: Fix segmentation fault - #21474: MAINT: Update doc requirements - #21475: MAINT: Mark ``npy_memchr`` with ``no_sanitize("alignment")`` on clang - #21512: DOC: Proposal - make the doc landing page cards more similar... - #21525: MAINT: Update Cython version to 0.29.30. - #21536: BUG: Fix GCC error during build configuration - #21541: REL: Prepare for the NumPy 1.22.4 release. - #21547: MAINT: Skip tests that fail on PyPy. Cheers, Charles Harris

1 0

New stable documentation
by Charles R Harris May 21, 2022

May 21, 2022

Hi All, I've put up new stable documentation for NumPy 1.22.4. I'd appreciate it if those familiar with how they want the documentation to look could take a look at it so that fixes can be made while I'm still in the documentation state. Chuck

4 5

ANN: SciPy 1.8.1
by Tyler Reddy May 18, 2022

May 18, 2022

Hi all, On behalf of the SciPy development team, I'm pleased to announce the release of SciPy 1.8.1, which is a bug fix release that restores Pythran usage on Windows. Sources and binary wheels can be found at: https://pypi.org/project/scipy/ and at: https://github.com/scipy/scipy/releases/tag/v1.8.1 One of a few ways to install this release with pip: pip install scipy==1.8.1 ===================== SciPy 1.8.1 Release Notes ===================== SciPy 1.8.1 is a bug-fix release with no new features compared to 1.8.0. Notably, usage of Pythran has been restored for Windows builds/binaries. Authors ====== * Henry Schreiner * Maximilian Nöthe * Sebastian Berg (1) * Sameer Deshmukh (1) + * Niels Doucet (1) + * DWesl (4) * Isuru Fernando (1) * Ralf Gommers (4) * Matt Haberland (1) * Andrew Nelson (1) * Dimitri Papadopoulos Orfanos (1) + * Tirth Patel (3) * Tyler Reddy (46) * Pamphile Roy (7) * Niyas Sait (1) + * H. Vetinari (2) * Warren Weckesser (1) A total of 17 people contributed to this release. People with a "+" by their names contributed a patch for the first time. This list of names is automatically generated, and may not be fully complete. Issues closed for 1.8.1 ------------------------------- * `#15258 <https://github.com/scipy/scipy/issues/15258>`__: BUG: sparse \`dot\` method should accept scalars * `#15433 <https://github.com/scipy/scipy/issues/15433>`__: BUG: optimize: minimize: \`ValueError\` when \`np.all(lb==ub)\` * `#15539 <https://github.com/scipy/scipy/issues/15539>`__: BUG: Questionable macOS wheel contents * `#15543 <https://github.com/scipy/scipy/issues/15543>`__: REL: list contributors using GitHub handles * `#15552 <https://github.com/scipy/scipy/issues/15552>`__: BUG: MacOS universal2 wheels have two gfortran shared libraries,... * `#15636 <https://github.com/scipy/scipy/issues/15636>`__: BUG: DOCS incorrect \`source\` link on docs * `#15678 <https://github.com/scipy/scipy/issues/15678>`__: BUG: scipy.stats.skew does not work with scipy.stats.bootstrap * `#16174 <https://github.com/scipy/scipy/issues/16174>`__: Failure of \`TestCorrelateComplex.test_rank0\` in CI with NumPy... Pull requests for 1.8.1 ------------------------------ * `#15167 <https://github.com/scipy/scipy/pull/15167>`__: CI: make sure CI stays on VS2019 unless changed explicitly * `#15306 <https://github.com/scipy/scipy/pull/15306>`__: Revert "BLD Respect the --skip-build flag in setup.py" * `#15504 <https://github.com/scipy/scipy/pull/15504>`__: MAINT: np.all(lb == ub) for optimize.minimize * `#15530 <https://github.com/scipy/scipy/pull/15530>`__: REL: prep for SciPy 1.8.1 * `#15531 <https://github.com/scipy/scipy/pull/15531>`__: [BUG] Fix importing scipy.lib._pep440 * `#15558 <https://github.com/scipy/scipy/pull/15558>`__: CI: re-enable Pythran in Azure Windows CI jobs * `#15566 <https://github.com/scipy/scipy/pull/15566>`__: BUG: fix error message * `#15580 <https://github.com/scipy/scipy/pull/15580>`__: BUG: Avoid C Preprocessor symbol in _hypotests_pythran.py. * `#15614 <https://github.com/scipy/scipy/pull/15614>`__: REL: filter out @ in authors name and add count * `#15637 <https://github.com/scipy/scipy/pull/15637>`__: DOC, MAINT: fix links to wrapped functions and SciPy's distributions * `#15669 <https://github.com/scipy/scipy/pull/15669>`__: BUG: stats: fix a bug in UNU.RAN error handler * `#15691 <https://github.com/scipy/scipy/pull/15691>`__: MAINT: stats: bootstrap: fix bug with \`method="BCa"\` when \`statistic\`... * `#15798 <https://github.com/scipy/scipy/pull/15798>`__: MAINT,BUG: stats: update to UNU.RAN 1.9.0 * `#15870 <https://github.com/scipy/scipy/pull/15870>`__: TST: signal: Convert a test with 'assert_array_less' to 'less... * `#15910 <https://github.com/scipy/scipy/pull/15910>`__: make sure CI stays on VS2019 unless changed explicitly * `#15926 <https://github.com/scipy/scipy/pull/15926>`__: MAINT: 1.8.1 backports/prep * `#16035 <https://github.com/scipy/scipy/pull/16035>`__: BUG: allow scalar input to the \`.dot\` method of sparse matrices * `#16041 <https://github.com/scipy/scipy/pull/16041>`__: MAINT: add include dir explicitly for PROPACK to build with classic... * `#16139 <https://github.com/scipy/scipy/pull/16139>`__: WIP, BLD, MAINT: git security/version shim * `#16152 <https://github.com/scipy/scipy/pull/16152>`__: TST: Fortify invalid-value warning filters to small changes in... * `#16155 <https://github.com/scipy/scipy/pull/16155>`__: MAINT: correct wrong license of Biasedurn * `#16158 <https://github.com/scipy/scipy/pull/16158>`__: MAINT: better UNU.RAN licensing information * `#16163 <https://github.com/scipy/scipy/pull/16163>`__: MAINT: update UNU.RAN copyright information * `#16172 <https://github.com/scipy/scipy/pull/16172>`__: CI: pin Pip to 22.0.4 to avoid issues with \`--no-build-isolation\` * `#16175 <https://github.com/scipy/scipy/pull/16175>`__: TST: fix test failure due to changes in numpy scalar behavior. Checksums ========= MD5 ~~~ fb1d0c039789080b79eb7f5cfbe2ed6c scipy-1.8.1-cp310-cp310-macosx_10_9_x86_64.whl 5cd8466411b660852075ded9609ceb49 scipy-1.8.1-cp310-cp310-macosx_12_0_arm64.whl 1f5e72c7d91fda43cddcbd3f272b9b37 scipy-1.8.1-cp310-cp310-macosx_12_0_universal2.macosx_10_9_x86_64.whl 9cc2f082eeea8b9a75dccfe9d21d2cec scipy-1.8.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 2cdec5543380dc0c39b061f28893d448 scipy-1.8.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl e8965ab9b1665d1afab0acc6ff716294 scipy-1.8.1-cp310-cp310-win_amd64.whl 2806a86294cff1208ce008515cc114a0 scipy-1.8.1-cp38-cp38-macosx_10_9_x86_64.whl 6fa1abe98efc2b70de1b166a469bb253 scipy-1.8.1-cp38-cp38-macosx_12_0_arm64.whl 4996536b5017a35942c04ac452645983 scipy-1.8.1-cp38-cp38-macosx_12_0_universal2.macosx_10_9_x86_64.whl 7c626d9c3eb2fcddc0be78b27d75f6a9 scipy-1.8.1-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl 8291385cb9fed5df2b99447d67c3d2af scipy-1.8.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl dbd563d80b403c0e0efaebca344f2336 scipy-1.8.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 6925e10377659b20a5499202df4ec2d8 scipy-1.8.1-cp38-cp38-win32.whl 821c51f0d16b0b33e2fb9600db83f41e scipy-1.8.1-cp38-cp38-win_amd64.whl ab397feec1d9479b3d603e0314e8876a scipy-1.8.1-cp39-cp39-macosx_10_9_x86_64.whl 251ac02266cf7b1f39a479e29c1b2b31 scipy-1.8.1-cp39-cp39-macosx_12_0_arm64.whl cec549c3321d98233d5e25b99bf8a7de scipy-1.8.1-cp39-cp39-macosx_12_0_universal2.macosx_10_9_x86_64.whl 21cba6f710c4065cf602af55e661928e scipy-1.8.1-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl c44c79b4447cd9f7b77a6172e1d6a71a scipy-1.8.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 44c33e63e2e810891bfd7aebcd87bf4d scipy-1.8.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 9a7e5cd1440b4d506acc2105501d8643 scipy-1.8.1-cp39-cp39-win32.whl 12952c28f9b1b045598f04628d97c64b scipy-1.8.1-cp39-cp39-win_amd64.whl df5ce79288fc457238aeef18e8f70dfc scipy-1.8.1.tar.gz 15c3e57656dbd1626ae33971f2e80ab1 scipy-1.8.1.tar.xz 51a26cfa0ca569cffd940138814ba23d scipy-1.8.1.zip SHA256 ~~~~~~ 65b77f20202599c51eb2771d11a6b899b97989159b7975e9b5259594f1d35ef4 scipy-1.8.1-cp310-cp310-macosx_10_9_x86_64.whl e013aed00ed776d790be4cb32826adb72799c61e318676172495383ba4570aa4 scipy-1.8.1-cp310-cp310-macosx_12_0_arm64.whl 02b567e722d62bddd4ac253dafb01ce7ed8742cf8031aea030a41414b86c1125 scipy-1.8.1-cp310-cp310-macosx_12_0_universal2.macosx_10_9_x86_64.whl 1da52b45ce1a24a4a22db6c157c38b39885a990a566748fc904ec9f03ed8c6ba scipy-1.8.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl a0aa8220b89b2e3748a2836fbfa116194378910f1a6e78e4675a095bcd2c762d scipy-1.8.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 4e53a55f6a4f22de01ffe1d2f016e30adedb67a699a310cdcac312806807ca81 scipy-1.8.1-cp310-cp310-win_amd64.whl 28d2cab0c6ac5aa131cc5071a3a1d8e1366dad82288d9ec2ca44df78fb50e649 scipy-1.8.1-cp38-cp38-macosx_10_9_x86_64.whl 6311e3ae9cc75f77c33076cb2794fb0606f14c8f1b1c9ff8ce6005ba2c283621 scipy-1.8.1-cp38-cp38-macosx_12_0_arm64.whl 3b69b90c9419884efeffaac2c38376d6ef566e6e730a231e15722b0ab58f0328 scipy-1.8.1-cp38-cp38-macosx_12_0_universal2.macosx_10_9_x86_64.whl 6cc6b33139eb63f30725d5f7fa175763dc2df6a8f38ddf8df971f7c345b652dc scipy-1.8.1-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl 9c4e3ae8a716c8b3151e16c05edb1daf4cb4d866caa385e861556aff41300c14 scipy-1.8.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 23b22fbeef3807966ea42d8163322366dd89da9bebdc075da7034cee3a1441ca scipy-1.8.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 4b93ec6f4c3c4d041b26b5f179a6aab8f5045423117ae7a45ba9710301d7e462 scipy-1.8.1-cp38-cp38-win32.whl 70ebc84134cf0c504ce6a5f12d6db92cb2a8a53a49437a6bb4edca0bc101f11c scipy-1.8.1-cp38-cp38-win_amd64.whl f3e7a8867f307e3359cc0ed2c63b61a1e33a19080f92fe377bc7d49f646f2ec1 scipy-1.8.1-cp39-cp39-macosx_10_9_x86_64.whl 2ef0fbc8bcf102c1998c1f16f15befe7cffba90895d6e84861cd6c6a33fb54f6 scipy-1.8.1-cp39-cp39-macosx_12_0_arm64.whl 83606129247e7610b58d0e1e93d2c5133959e9cf93555d3c27e536892f1ba1f2 scipy-1.8.1-cp39-cp39-macosx_12_0_universal2.macosx_10_9_x86_64.whl 93d07494a8900d55492401917a119948ed330b8c3f1d700e0b904a578f10ead4 scipy-1.8.1-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl d3b3c8924252caaffc54d4a99f1360aeec001e61267595561089f8b5900821bb scipy-1.8.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 70de2f11bf64ca9921fda018864c78af7147025e467ce9f4a11bc877266900a6 scipy-1.8.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 1166514aa3bbf04cb5941027c6e294a000bba0cf00f5cdac6c77f2dad479b434 scipy-1.8.1-cp39-cp39-win32.whl 9dd4012ac599a1e7eb63c114d1eee1bcfc6dc75a29b589ff0ad0bb3d9412034f scipy-1.8.1-cp39-cp39-win_amd64.whl 9e3fb1b0e896f14a85aa9a28d5f755daaeeb54c897b746df7a55ccb02b340f33 scipy-1.8.1.tar.gz 0140dd24d14ce194fdb3df496c2190f0896ca84ac98bf5d69c3b3fb7bde1a1e9 scipy-1.8.1.tar.xz 0d27d1a8644ca83b86a370bcc8dfb73bd7d200843d0c327fef1127e8be95512d scipy-1.8.1.zip

1 0

ndarray shape permutation
by Paul Korir May 18, 2022

May 18, 2022

Hellos, I would like to propose `numpy.ndarray.permute_shape(<permuted_shape>)` method to predictably permute the shape of an ndarray. In my opinion, the current alternatives (`swapaxes`, `transform`, `moveaxes` and friends) are counterintuitive and rely on referring to the axis indices. It would be abundantly helpful to have something like reshape but which moves the data around (unlike reshape, which doesn't). Scenario: structural biology uses MRC files, which define a number of fields that describe a 3D volume. There is a field which describes the dimensions of the 3D image and another which associates each image axis with a physical axis. There are six such assignments between the image shape and the axes assignments (if we keep the shape fixed we can assign XYZ, XZY, YXZ, YZX, ZXY, ZYX to the columns, rows and stacks) and working out how to correctly transform the data is generally non-trivial. I've written a package (https://github.com/emdb-empiar/maptools) which does this but I use swapaxes to reorient the 3D image. It would be so much easier to use `numpy.ndarray.permute_shape(<permuted_shape>)` instead. Any thoughts? Also, any helpful hints on how to get started with such a contribution would be helpful. Paul

5 6

ENH: Support Random Unit Vector
by saroad2＠gmail.com May 17, 2022

May 17, 2022

### Proposed new feature or change: Description ======== Often you want to randomize a "direction" that doesn't have "size". This can be useful when: * You want to randomize the direction of an NPC walk developing in a 2D or 3D games, where its speed is a predefined constant. * You're developing particles simulation with all particles moving in random directions * etc. This random direction should be an `n`th dimensional unit vector which is randomize **uniformly** from the unit sphere. That means that sections of the unit sphere with equal areas should have the same chance of getting a vector from. Implementation =========== If one wants to randomize `n`th dimensional direction uniformly, one must do the following: 1. Generate `n` components of the vector using the standard normal distribution 2. Find the norm the generated vector 3. Divide the vector by its norm In simple Python code: ```python import numpy as np def sphere_uniform(n: int): vec = np.random.normal(size=n) norm = np.sqrt(np.sum(vec ** 2)) return vec / norm ``` This implementation suggestion is based on [this blog post](https://towardsdatascience.com/the-best-way-to-pick-a-unit-vector-7bd…. I'm not sure if there's a faster way of implementing it using C++ and Cython, but I'm confident that someone here will know. Why Implement It in Numpy? =================== I believe that random unit vectors are common enough to be a part of Numpy. Scientists, gamers and engineers use random directions all the time, and I believe there is no reason why Numpy won't provide this ability.

2 1

NumPy Development Meeting Wednesday - Triage Focus
by Sebastian Berg May 17, 2022

May 17, 2022

Hi all, Our bi-weekly triage-focused NumPy development meeting is Wednesday, May 18th at 16:00 UTC (9:00am Pacific Time). Everyone is invited to join in and edit the work-in-progress meeting topics and notes: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg I encourage everyone to notify us of issues or PRs that you feel should be prioritized, discussed, or reviewed. A calendar to can be found at: https://scientific-python.org/calendars/ please make sure to not import the static `ics` but the url. Best regards Sebastian

1 0

Interesting security project
by Charles R Harris May 14, 2022

May 14, 2022

Hi All, There is a project underway to secure open source software <https://venturebeat.com/2022/05/13/how-much-will-it-cost-to-secure-open-sou…>. This doesn't affect us (yet), but is of interest as it shows the growing importance of open source. Chuck

1 0

Fixing/changing structured dtype casting safety, promotion, and comparisons
by Sebastian Berg May 11, 2022

May 11, 2022

Hi all, This was probably announced before, but I wanted to mention it again, with https://github.com/numpy/numpy/pull/19226 as the last step the casting safety, promotion, and comparisons of structured (void) dtypes will be updated to fix some bugs and fit better with changes that happened a few years ago already. The most interesting change is probably to comparisons: It is now possible to compare sturctured dtypes when included dtypes mismatch: arr1 = np.array([1, 2, 3], dtype="i,i") arr2 = np.array([1, 2, 1], dtype="i,f8") # note the f8 arr1 == arr2 # Will now return [True, True, False] while previously this returned `False` with a `FutureWarning` (although that warning was not quite correct. Comparisons will now always generally succeed if the two dtypes can be promoted. The other change to `==` and `!=` is that in cases where promotion is not possible, a `TypeError` will now be given for structured dtype comparisons. (Previously, also `False` with a `FutureWarning`) These changes align with the changes in promotion for structured dtypes. Structured dtypes will now correctly promote all included dtypes if the number of fields, names, and titles match exactly. Further, structured dtypes will remove any unnecessary empty space when promoted. The result is considered "canonical" with the removed padding. In alignment with the above changes to the promotion logic, the casting safety has been updated: * "equiv" enforces matching names and titles. The itemsize is allowed to differ due to padding. * "safe" allows mismatching field names and titles * The cast safety is limited by the cast safety of each included field. * The order of fields is used to decide cast safety of each individual field. Previously, the field names were used and only unsafe casts were possible when names mismatched. The main important change here is that name mismatches are now considered "safe" casts. The cast itself already worked using the field order. Cheers, Sebastian

1 0

[Feature Request] Add alias of np.concatenate as np.concat
by Iordanis Fostiropoulos May 10, 2022

May 10, 2022

In regard to Feature Request: https://github.com/numpy/numpy/issues/16469 It was suggested to sent to the mailing list. I think I can make a strong point as to why the support for this naming convention would make sense. Such as it would follow other frameworks that often work alongside numpy such as tensorflow. For backward compatibility, it can simply be an alias to np.concatenate I often convert portions of code from tf to np, it is as simple as changing the base module from tf to np. e.g. np.expand_dims -> tf.expand_dims. This is done either in debugging (e.g. converting tf to np without eager execution to debug portion of the code), or during prototyping, e.g. develop in numpy and convert in tf. I find myself more than at one occasion to getting syntax errors because of this particular function np.concatenate. It is unnecessarily long. I imagine there are more people that also run into the same problems. Pandas uses concat (torch on the other extreme uses simply cat, which I don't think is as descriptive).

7 6

NumPy community meeting – Wednesday, March 30th, 2022 at 18:00 (6 pm) UTC
by Inessa Pawson May 9, 2022

May 9, 2022

Our next NumPy community meeting is scheduled for Wednesday, March 30th at 18:00 (6 pm) UTC. Join us via Zoom: https://berkeley.zoom.us/j/762261535 As always, everyone is welcome and encouraged to attend. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy

1 2

Automatic formatters for only changed lines
by Trevor Gross May 8, 2022

May 8, 2022

Hey all, Matti pointed out to me that discussion of an autoformatter has come up before, with the decision that an entire codebase rewrite would be undesirable (ref: https://mail.python.org/pipermail/scipy-dev/2020-January/023927.html). As a result, tools/linter.py was added to only validate the changed lines. There is better support for autoformatters that only work on changed lines now, and I'd like to propose adding one. I already have a pre-commit demo implemented here, with an example run: https://github.com/numpy/numpy/pull/21449. It applies Black to changed python file lines plus flake8 to validate them (changed lines only), and clang-format with the pep7 config to any changed c/c++ lines. Pre-commit has the easy interface to run on staged files (default) all files, specific files, or files changed between --from-ref and --to-ref. I don't necessarily know that this should be enforced on CI off the bat (though in the future it could be), but at least having the tool available and easily usable would help to incrementally improve codebase style, and remove the need for formatting by hand. Any thoughts? - Trevor

10 14

ndarray repr - put shape & type first
by Ilya Kamenshchikov May 7, 2022

May 7, 2022

Hello, I wanted to discuss a pain point I've experienced while debugging numpy code. When dealing with e.g. transformed image arrays or other non-trivial ndarrays in debugger, I'm swamped by a bunch of numbers in their repr that don't help me at all. What I really care about is *shape and dtype*, as they are essentially what distinguishes complete type of numpy arrays. Example of not helpful output: ``` array([[[0.49113064, 0.42102904, 0.62108876], [0.25435884, 0.18665439, 0.53790145], ``` By counting the [ brackets, I can at least get the number of dimensions, but usefulness stops here. Could repr of an array start with something like: `array(*shape=[32,32,3], dtype=float, data=*([[[0.49113064, ...)` I know one invariant that repr likes to keep is that you can post repr and it should be the initialisation of the represented object, but given that we replace long number sequence with ..., this is already not the case. Short term: Perhaps a plugin / change to IDE could do what I ask, please let me know if something like this already exists :) Long term: I think that behavior will be more useful than what is currently there. It could also be conditionally there once we anyhow need to use ... for too long arrays. Best Regards, -- Ilya Kamen

2 1

Numpy-related jobs, grants, etc
by Renato Fabbri May 4, 2022

May 4, 2022

This is not only for me, but a question for the community in general. Do we have an open knowledge base of budget (interested companies or individuals, public funding) for developing numpy-related packages? Explaining a bit with a specific case... In the last few years, I have many times thought about the possibility of working with Numpy: finding payment for developing libraries using Numpy or even Numpy internals. Especially, I have written a music python library (called 'music' on PyPI). It holds many useful things, equal-loudness contours, musical notes, localization, HRTF, LUT, singing, glissandi, tremolos & vibratos with transitions, etc etc. I believe it is something well done, which involved scientific research and over a decade of experience with the algorithms, but it still lacks a better organization of the functionalities, better documentation, and better maintenance. To reach that point, one would probably have to be paid to take the necessary time. Beyond that, I know it is useful for making art/music, teaching, learning music, learning music-related signal processing and programming, etc. Maybe the most important point: this is probably the case with other libraries/projects. ((( On the other side, we as a company (here at Modena/Italy) may look into (public, private, own) funding to develop Numpy (internals, libraries). We would also enjoy developing Numpy-related libraries (internals?) if someone needs or wants them. ))) Should this be posted in another list/forum? Any thoughts? R. -- Renato Fabbri --

2 1

Discussion: History & Possible Deprecation/Removal of numpy.linalg.{eig(), eigvals()}?
by Leo Fang May 3, 2022

May 3, 2022

Hi, I am catching up with an assigned task that I slipped. I am wondering a few things about the numpy.linalg.eig() API (eigvals() is also included in this discussion, but for simplicity let's focus on eig()). AFAIU the purpose of eig() is for handling general eigen-systems which have left and right eigenvectors (that could differ beyond transpose/conjugate). For this, I have two questions: 1. What's the history of NumPy adding this eig() routine? My guess would be NumPy added it because Matlab had it, but I couldn't tell from a quick commit search. 2. Would it be possible to deprecate/remove this API? From past discussions with users and developers, I found: - numpy.linalg.eig() only returns the right eigenvectors, so it's not very useful: * When the left eigenvectors are not needed --- which is the majority of the use cases --- eigh() is the right routine, but most users (especially those transitioning from Matlab) don't know it's there * When the left eigenvectors are needed, users have to use scipy.linalg.eig() anyway - A general support for eig() could be challenging (numerical issues), and doing SVDs instead is usually the better way out (in terms of both numerical stability and mathematical sense). Thanks, Leo

3 2

Types in pure Python sources
by Sergei Lebedev May 3, 2022

May 3, 2022

Hi everyone, I've noticed that NumPy currently users separate type stub files for specifying types for both pure Python and native modules. For example the (untyped) implementation of np.core._asarray is in [1], but the types are in [2]. This works fine for type checkers, and if anything, improves type checking performance, because type stubs are much smaller than their pure Python counterparts. It does, however, create problems for IDEs which now have to either work with untyped APIs or manually merge them with type stubs to provide better coding assistance. Is there a reason for type stubs being separate (other than historical [3])? If not, how should I go about starting a conversation upstream about this? Sergei [1]: https://github.com/numpy/numpy/blob/main/numpy/core/_asarray.py [2]: https://github.com/numpy/numpy/blob/main/numpy/core/_asarray.pyi [3]: https://github.com/numpy/numpy-stubs

3 4

NumPy Development Meeting Wednesday - Triage Focus
by Sebastian Berg May 3, 2022

May 3, 2022

Hi all, Our bi-weekly triage-focused NumPy development meeting is Wednesday, May 4th at 16:00 UTC (9:00am Pacific Time). Everyone is invited to join in and edit the work-in-progress meeting topics and notes: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg I encourage everyone to notify us of issues or PRs that you feel should be prioritized, discussed, or reviewed. A calendar to can be found at: https://scientific-python.org/calendars/ please make sure to not import the static `ics` but the url. Best regards Sebastian

2 1