Mailman 3 May 2022 - NumPy-Discussion

overhead of numpy scalars
by Pieter Eendebak 01 Jun '22

01 Jun '22

Hi everyone, Operations such as np.sqrt or np.cos on numpy scalars are much slower than operations on an array of size 1 (benchmark below). The reason seems to be that a numpy scalar (e.g. np.float64(1.1)) is converted internally to an array, an array is created for the result of the operation and the result is converted back to a scalar (so three object constructions in total). Operating directly on an array creates an array for the result, which is returned (so one object construction in total). Has anyone looked into improving this performance before or has ideas how to tackle this? With kind regards, Pieter Eendebak Example benchmark: import numpy as np from numpy import sqrt import time v=np.array([1.1]) t0=time.perf_counter() x=np.float64(v) for kk in range(1_000_000): w1=sqrt(x) dt1=time.perf_counter()-t0 print(dt1) t0=time.perf_counter() x=v for kk in range(1_000_000): w2=sqrt(x) dt2=time.perf_counter()-t0 print(dt2) print(f'factor {dt1/dt2:.2f} faster') assert(float(w1)==float(w2)) Results in 1.0878291579974757 0.5115369699997245 factor 2.13 faster

2 2

NumPy Development Meeting Wednesday - Triage Focus
by Sebastian Berg 31 May '22

31 May '22

Hi all, Our bi-weekly triage-focused NumPy development meeting is Wednesday, June 1st at 16:00 UTC (9:00am Pacific Time). Everyone is invited to join in and edit the work-in-progress meeting topics and notes: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg I encourage everyone to notify us of issues or PRs that you feel should be prioritized, discussed, or reviewed. A calendar to can be found at: https://scientific-python.org/calendars/ please make sure to not import the static `ics` but the url. Best regards Sebastian

1 0

NumPy 1.23.0rc2 released
by Charles R Harris 30 May '22

30 May '22

Hi All, On behalf of the NumPy team, I'm pleased to announce the release of NumPy 1.23.0rc2. The NumPy 1.23.0 release continues the ongoing work to improve the handling and promotion of dtypes, increase the execution speed, clarify the documentation, and expire old deprecations. The highlights are: - Implementation of ``loadtxt`` in C, greatly improving its performance. - Exposing DLPack at the Python level for easy data exchange. - Changes to the promotion and comparisons of structured dtypes. - Improvements to f2py. The Python versions supported in this release are 3.8-3.10, 3.11 will be supported when it comes out. Note that 32 bit wheels are only provided for Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit OpenBLAS. Wheels can be downloaded from PyPI <https://pypi.org/project/numpy/1.23.0rc2/>; source archives, release notes, and wheel hashes are available on Github <https://github.com/numpy/numpy/releases/tag/v1.23.0rc2>. *Contributors* A total of 150 people contributed to this release. People with a "+" by their names contributed a patch for the first time. - @DWesl - @GalaxySnail + - @code-review-doctor + - @h-vetinari - Aaron Meurer - Alexander Shadchin - Alexandre de Siqueira - Allan Haldane - Amrit Krishnan - Andrei Batomunkuev - Andrew J. Hesford + - Andrew Murray + - Andrey Andreyevich Bienkowski + - André Elimelek de Weber + - Andy Wharton + - Arryan Singh - Arushi Sharma - Bas van Beek - Bharat Raghunathan - Bhavuk Kalra + - Brigitta Sipőcz - Brénainn Woodsend + - Burlen Loring + - Caio Agiani + - Charles Harris - Chiara Marmo - Cornelius Roemer + - Dahyun Kim + - Damien Caliste - David Prosin + - Denis Laxalde - Developer-Ecosystem-Engineering - Devin Shanahan + - Diego Wang + - Dimitri Papadopoulos Orfanos - Ding Liu + - Diwakar Gupta + - Don Kirkby + - Emma Simon + - Eric Wieser - Evan Miller + - Evgeni Burovski - Evgeny Posenitskiy + - Ewout ter Hoeven + - Felix Divo - Francesco Andreuzzi + - Ganesh Kathiresan - Gaëtan de Menten - Geoffrey Gunter + - Hans Meine - Harsh Mishra + - Henry Schreiner - Hood Chatham + - Ilhan Polat - Inessa Pawson - Isuru Fernando - Ivan Gonzalez + - Ivan Meleshko + - Ivan Yashchuk + - Janus Heide + - Jarrod Millman - Jason Thai + - Jeremy Volkman + - Jesús Carrete Montaña + - Jhong-Ken Chen (陳仲肯) + - John Kirkham - John-Mark Gurney + - Jonathan Deng + - Joseph Fox-Rabinovitz - Jouke Witteveen + - Junyan Ou + - Jérôme Richard + - Kassian Sun + - Kazuki Sakamoto + - Kenichi Maehashi - Kevin Sheppard - Kilian Lieret + - Kushal Beniwal + - Leo Singer - Logan Thomas + - Lorenzo Mammana + - Margret Pax - Mariusz Felisiak + - Markus Mohrhard + - Mars Lee - Marten van Kerkwijk - Masamichi Hosoda + - Matthew Barber - Matthew Brett - Matthias Bussonnier - Matthieu Darbois - Matti Picus - Melissa Weber Mendonça - Michael Burkhart + - Morteza Mirzai + - Motahhar Mokf + - Muataz Attaia + - Muhammad Motawe + - Mukulika Pahari - Márton Gunyhó + - Namami Shanker + - Nihaal Sangha + - Niyas Sait - Omid Rajaei + - Oscar Gustafsson + - Ovee Jawdekar + - P. L. Lim + - Pamphile Roy + - Pantelis Antonoudiou + - Pearu Peterson - Peter Andreas Entschev - Peter Hawkins - Pierre de Buyl - Pieter Eendebak + - Pradipta Ghosh + - Rafael Cardoso Fernandes Sousa + - Raghuveer Devulapalli - Ralf Gommers - Raphael Kruse - Raúl Montón Pinillos - Robert Kern - Rohit Goswami - Ross Barnowski - Ruben Garcia + - Sadie Louise Bartholomew + - Saswat Das + - Sayed Adel - Sebastian Berg - Serge Guelton - Simon Surland Andersen + - Siyabend Ürün + - Somasree Majumder + - Soumya + - Stefan van der Walt - Stefano Miccoli + - Stephan Hoyer - Stephen Worsley + - Tania Allard - Thomas Duvernay + - Thomas Green + - Thomas J. Fan - Thomas Li + - Tim Hoffmann - Ting Sun + - Tirth Patel - Toshiki Kataoka - Tyler Reddy - Warren Weckesser - Yang Hau - Yoon, Jee Seok + Cheers, Charles Harris Reply allReplyForward <https://drive.google.com/u/0/settings/storage?hl=en&utm_medium=web&utm_sour…> <https://www.google.com/intl/en/policies/terms/> <https://www.google.com/intl/en/policies/privacy/> <https://www.google.com/gmail/about/policy/>

1 0

numpy.sort: Ability to reverse-sort a subset of columns?
by James Anderson 27 May '22

27 May '22

Near as I can tell the numpy.sort function doesn’t support requesting a descending sort on specific columns. (Docs: https://numpy.org/doc/stable/reference/generated/numpy.sort.html) I wanted to check I’m not missing how this is properly done or if should I file this as a feature request. There are many reasons one would need to choose sorting in ascending/descending order on a column by column basis. My first thought is another parameter to sort which is an array of bools to indicate sort order. An alternative is to prepend ‘-‘ to column names that should be sorted in descending order. However the second idea may conflict with people's names for columns. Thanks, James Anderson Example: In my case I have an sweep line algorithm that needs to walk an ndarray with the following dtype row-by-row: active_dtype = np.dtype([('Value', 'f4'), ('ID', 'u8'), ('Active', 'u1')]) I need to sort one column (Value) in ascending order, but another column (Active) in descending order. (To handle the degenerate case where a rectangle has a zero width/height). For full context the code below takes a rectangle array (rect_array) and returns sorted arrays indicating where rectangle boundaries start and stop on each axis. In this example one can invert the meaning of ‘Active’ to work around the lack of API support. The intent is just to illustrate how I was hoping I had expected to use numpy.sort. x_sweep_array = np.empty(len(rect_array) * 2, dtype=cls.active_dtype) y_sweep_array = np.empty(len(rect_array) * 2, dtype=cls.active_dtype) for i, rect in enumerate(rect_array): x_sweep_array[i * 2] = (rect['MinX'], rect['ID'], True) x_sweep_array[(i * 2) + 1] = (rect['MaxX'], rect['ID'], False) y_sweep_array[i * 2] = (rect['MinY'], rect['ID'], True) y_sweep_array[(i * 2) + 1] = (rect['MaxY'], rect['ID'], False) x_sweep_array = np.sort(x_sweep_array, order=('Value', 'Active')) # <- Cannot reverse sort Active y_sweep_array = np.sort(y_sweep_array, order=('Value', 'Active')) return (x_sweep_array, y_sweep_array)

1 0

NEP 29 and the faster Python release cadence
by Aaron Meurer 27 May '22

27 May '22

Was the faster CPython release cadence (PEP 602 https://peps.python.org/pep-0602/) ever discussed in relation to NEP 29 (https://numpy.org/neps/nep-0029-deprecation_policy.html)? NEP 29 currently says: "The current Python release cadence is 18 months so a 42 month window ensures that there will always be at least two minor versions of Python in the window." However, as of PEP 602, the release cadence of CPython is now 12 months, not 18 months. This schedule now means supporting 3-4 versions of Python at once, rather than 2-3 versions (for those who don't want to do the math, 48/18 is 2.3 and 48/12 is 3.5). PEP 602 started with Python 3.8, which was released in October, 2019. Python 3.8 is currently the oldest version supported in NEP 29, so we are about to enter the full cycle of the new cadence. In particular, when Python 3.11 is released in October of this year, the NEP 29 prescription will imply supporting 4 Python versions: 3.8, 3.9, 3.10, and 3.11, up until April, 2023 when 3.8 support would be dropped (and continuing like this: 3 versions between April and October; 4 versions between October and April). Furthermore, the NEP seems to imply that the exact CPython release schedule is not known: "The window is extended 6 months beyond the anticipated two-release interval for Python to provide resilience against small fluctuations / delays in its release schedule." I'm not sure how true this was before, but PEP 602 pretty clearly prescribes a Python release in October of every year. Delays presumably might still be possible, but I don't see why it's necessary to build that possibility into the default period in the NEP given that it should be an unusual situation. Personally I would prefer not ever supporting 4 simultaneous Python versions (I rather wish Python hadn't increased their cadence, but that ship has sailed). I don't know if it's possible to update the NEP to adjust for the new cadence. Even if it isn't, the text is clearly out of date and should be updated. I wasn't on this list back in 2019, so my apologies if this has already been discussed. Aaron Meurer

7 14

GSoC Student Announcement 2022
by Rohit Goswami 27 May '22

27 May '22

NumPy as a sub-organization under the Python Software Foundation organization will be mentoring a summer student this year funded by the Google Summer of Code program. Ralf facilitated the application process and I will be the primary mentor this year (with Gagandeep for support). Please look forward to work done by our mentee, Namami Shankar who will be working on F2PY and in particular our front-end. His proposal entitled "Restructuring the F2PY frontend and replacing the build system" can be found here: https://summerofcode.withgoogle.com/media/user/a7128bebaf4c/proposal/bMbhEN…

3 2

NumPy 1.23.0rc1 released
by Charles R Harris 27 May '22

27 May '22

Hi All, On behalf of the NumPy team, I'm pleased to announce the release of NumPy 1.23.0rc1. The NumPy 1.23.0 release continues the ongoing work to improve the handling and promotion of dtypes, increase the execution speed, clarify the documentation, and expire old deprecations. The highlights are: - Implementation of ``loadtxt`` in C, greatly improving its performance. - Exposing DLPack at the Python level for easy data exchange. - Changes to the promotion and comparisons of structured dtypes. - Improvements to f2py. The Python versions supported in this release are 3.8-3.10, 3.11 will be supported when it comes out. Note that 32 bit wheels are only provided for Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit OpenBLAS. Wheels can be downloaded from PyPI <https://pypi.org/project/numpy/1.23.0rc1/>; source archives, release notes, and wheel hashes are available on Github <https://github.com/numpy/numpy/releases/tag/v1.23.0rc1>. *Contributors* A total of 150 people contributed to this release. People with a "+" by their names contributed a patch for the first time. - @DWesl - @GalaxySnail + - @code-review-doctor + - @h-vetinari - Aaron Meurer - Alexander Shadchin - Alexandre de Siqueira - Allan Haldane - Amrit Krishnan - Andrei Batomunkuev - Andrew J. Hesford + - Andrew Murray + - Andrey Andreyevich Bienkowski + - André Elimelek de Weber + - Andy Wharton + - Arryan Singh - Arushi Sharma - Bas van Beek - Bharat Raghunathan - Bhavuk Kalra + - Brigitta Sipőcz - Brénainn Woodsend + - Burlen Loring + - Caio Agiani + - Charles Harris - Chiara Marmo - Cornelius Roemer + - Dahyun Kim + - Damien Caliste - David Prosin + - Denis Laxalde - Developer-Ecosystem-Engineering - Devin Shanahan + - Diego Wang + - Dimitri Papadopoulos Orfanos - Ding Liu + - Diwakar Gupta + - Don Kirkby + - Emma Simon + - Eric Wieser - Evan Miller + - Evgeni Burovski - Evgeny Posenitskiy + - Ewout ter Hoeven + - Felix Divo - Francesco Andreuzzi + - Ganesh Kathiresan - Gaëtan de Menten - Geoffrey Gunter + - Hans Meine - Harsh Mishra + - Henry Schreiner - Hood Chatham + - Ilhan Polat - Inessa Pawson - Isuru Fernando - Ivan Gonzalez + - Ivan Meleshko + - Ivan Yashchuk + - Janus Heide + - Jarrod Millman - Jason Thai + - Jeremy Volkman + - Jesús Carrete Montaña + - Jhong-Ken Chen (陳仲肯) + - John Kirkham - John-Mark Gurney + - Jonathan Deng + - Joseph Fox-Rabinovitz - Jouke Witteveen + - Junyan Ou + - Jérôme Richard + - Kassian Sun + - Kazuki Sakamoto + - Kenichi Maehashi - Kevin Sheppard - Kilian Lieret + - Kushal Beniwal + - Leo Singer - Logan Thomas + - Lorenzo Mammana + - Margret Pax - Mariusz Felisiak + - Markus Mohrhard + - Mars Lee - Marten van Kerkwijk - Masamichi Hosoda + - Matthew Barber - Matthew Brett - Matthias Bussonnier - Matthieu Darbois - Matti Picus - Melissa Weber Mendonça - Michael Burkhart + - Morteza Mirzai + - Motahhar Mokf + - Muataz Attaia + - Muhammad Motawe + - Mukulika Pahari - Márton Gunyhó + - Namami Shanker + - Nihaal Sangha + - Niyas Sait - Omid Rajaei + - Oscar Gustafsson + - Ovee Jawdekar + - P. L. Lim + - Pamphile Roy + - Pantelis Antonoudiou + - Pearu Peterson - Peter Andreas Entschev - Peter Hawkins - Pierre de Buyl - Pieter Eendebak + - Pradipta Ghosh + - Rafael Cardoso Fernandes Sousa + - Raghuveer Devulapalli - Ralf Gommers - Raphael Kruse - Raúl Montón Pinillos - Robert Kern - Rohit Goswami - Ross Barnowski - Ruben Garcia + - Sadie Louise Bartholomew + - Saswat Das + - Sayed Adel - Sebastian Berg - Serge Guelton - Simon Surland Andersen + - Siyabend Ürün + - Somasree Majumder + - Soumya + - Stefan van der Walt - Stefano Miccoli + - Stephan Hoyer - Stephen Worsley + - Tania Allard - Thomas Duvernay + - Thomas Green + - Thomas J. Fan - Thomas Li + - Tim Hoffmann - Ting Sun + - Tirth Patel - Toshiki Kataoka - Tyler Reddy - Warren Weckesser - Yang Hau - Yoon, Jee Seok + Cheers, Charles Harris

1 0

Enhancement for AArch64 SVE instruction set
by kawakami.k＠fujitsu.com 26 May '22

26 May '22

Hello, I am working on speeding up NumPy with the AArch64 SVE instruction set. I could not find a numpy implementation for SVE. Is there already a test implementation or discussion about SVE support?

3 5

Dropping the pdf documentation.
by Charles R Harris 26 May '22

26 May '22

Hi All, This is a proposal to drop the generation of pdf documentation and only generate the html version. This is a one way change due to the difficulty maintaining/fixing the pdf versions. See minimal discussion here <https://github.com/numpy/numpy/issues/21557#issuecomment-1133920412>. Chuck

11 21

normalization of Polynomial
by Pieter Eendebak 24 May '22

24 May '22

Hi, The new Polynomial objects includes a domain and window. How can I normalize such an object to a standard domain and window? There is the .convert method, but it returns a strange format. A minimal example: import numpy as np p =np.polynomial.Polynomial([1, 1]) x=np.linspace(0, 3, 15); y=p(x) y[1]+=.1 q=np.polynomial.Polynomial.fit(x, y, 1) print(f'p: {repr(p)}') print(f'fitted q: {repr(q)}') print(f'fitted q: {str(q)}') qc=q.convert() print(f'q.convert(): str {qc}') print(f'q.convert(): {repr(qc)}') The output of repr(qc) is: Polynomial([Polynomial([1.02166667, 0.99 ], domain=[-1., 1.], window=[-1., 1.])], dtype=object, domain=[-1., 1.], window=[-1., 1.]) but I am looking for: Polynomial([1.02166667, 0.99 ], domain=[-1., 1.], window=[-1., 1.]) With kind regards, Pieter Eendebak

1 0