Mailman 3 April 2023 - NumPy-Discussion

next NumPy triage meeting - now starting at 5 pm UTC
by Inessa Pawson 17 Mar '24

17 Mar '24

The next NumPy triage meeting will be held this Wednesday, January 11th at 5 pm UTC. (Please note the new time!) This is a meeting where we synchronously triage prioritized PRs and issues. Join us via Zoom: https://numfocus-org.zoom.us/j/82096749952?pwd=MW9oUmtKQ1c3a2gydGk1RTdYUUVX… Everyone is welcome to attend and contribute to a conversation. Please notify us of issues or PRs that you’d like to have reviewed by adding a GitHub link to them in the meeting agenda: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg. -- Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy https://numpy.org/ GitHub: inessapawson

1 30

next NumPy Newcomers' Hour - 12 pm UTC
by Inessa Pawson 31 Oct '23

31 Oct '23

Our next Newcomers' Hour will be held tomorrow, December 1st, at 12 pm UTC. Stop by to ask questions or just to say hi. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/3f3otyyuTte3FU9y3QzsLg?both Join the meeting via Zoom: https://us06web.zoom.us/j/82563808729?pwd=ZFU3Z2dMcXBGb05YemRsaGE1OW5nQT09 -- Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy https://numpy.org/ GitHub: inessapawson

2 5

next NumPy community meeting
by Inessa Pawson 22 Oct '23

22 Oct '23

The next NumPy community meeting will be held this Wednesday, August 17th at 18:00 (6 pm) UTC. Join us via Zoom: https://us06web.zoom.us/j/83278611437?pwd=ekhoLzlHRjdWc0NOY2FQM0NPemdkZz09 Everyone is welcome and encouraged to attend. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both For the notes from the previous meeting, visit: https://github.com/numpy/archive/blob/main/community_meetings/community-202… Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy https://numpy.org/ Twitter: @inessapawson

1 30

reminder: put an upper bound on setuptools if you use numpy.distutils!
by Ralf Gommers 31 Aug '23

31 Aug '23

Hi all, After setuptools 65.0 was released a few days ago, all users of numpy.distutils had their builds broken. This is already fixed in setuptools 65.0.2 because the breakage was particularly bad. However, the next breakage may not be fixed anymore (and more breakages *are* expected). So this is a good time to remind you all that you should put an upper bound on the setuptools version you allow in the releases of your package - to the last version that is known to work with your package. Our official stance here is that setuptools versions >=60 are not supported - see the "deprecating numpy.distutils" thread: https://mail.python.org/archives/list/numpy-discussion@python.org/message/P…. Newer versions may work for you, depending on what features you use. They don't for NumPy and for SciPy; both projects pin to 59.X to avoid problems. For the recent issue with setuptools 65.0.0, see https://github.com/numpy/numpy/issues/22135. We have also made the warnings about this topic in our docs more explicit, see https://github.com/numpy/numpy/pull/22154. Cheers, Ralf

2 2

Adding bit_count ufunc
by Doug Turnbull 13 Aug '23

13 Aug '23

👋Long time numpy user, and big fan of all your work TL; DR - there's a `bit_count` method on numpy scalars, and Python ints, I'm advocating for a `ufunc` `bit_count` as has been implemented in this PR: https://github.com/numpy/numpy/pull/21429 A number of people have requested this as a numpy feature and there's a lot of great discussion at this issue https://github.com/numpy/numpy/issues/16325 A bit count comes up in certain similarity situations (like hamming distance). This involves an xor with numpy arrays and a bit count, where the bit count is currently the bottleneck by a few orders of magnitude in my local benchmarking. Currently there's a number of not particularly fast, workarounds for doing this with numpy arrays, like the bit-twiddling solutions here https://stackoverflow.com/a/68943135/8123 https://stackoverflow.com/a/109025/8123 So I'd love to be able to continue the work of bit_count -> numpy array ufunc to get performance gains at the array level 🙏

5 7

mixed mode arithmetic
by Neal Becker 11 Jul '23

11 Jul '23

I've been browsing the numpy source. I'm wondering about mixed-mode arithmetic on arrays. I believe the way numpy handles this is that it never does mixed arithmetic, but instead converts arrays to a common type. Arguably, that might be efficient for a mix of say, double and float. Maybe not. But for a mix of complex and a scalar type (say, CDouble * Double), it's clearly suboptimal in efficiency. So, do I understand this correctly? If so, is that something we should improve?

4 6

Representation of NumPy scalars
by Sebastian Berg 09 Jun '23

09 Jun '23

TL;DR: NumPy scalars representation is e.g. `34.3` instead of `float32(34.3)`. So the representation is missing the type information. What are your thoughts on changing that? Hi all, I am thinking about the next steps for NEP 50 (The NEP wants to fix the NumPy promotion rules, especially with respect to scalars): https://numpy.org/neps/nep-0050-scalar-promotion.html In relation to that, there was one point that Stéfan brought up previously. The NumPy scalars (representation) currently print as numbers: >>> np.float32(34.3) 34.3 >>> np.uint8(5) 5 That can already be confusing now. However, it gets more problematic if NEP 50 is introduced since the behavior between a Python `34.3` and `np.float32(34.3)` would differ more than it does now (please refer to the NEP). The change would be that we should print as: float64(34.3) (or similar?) This Email is mainly to ask for any feedback or concern on such a change. I suspect we may have to write a very brief NEP about it. If there is little concern, maybe we could move forward such a change promptly. Otherwise it could be moved forward together with NEP 50 and take effect in a "major" release [1]. Cheers, Sebastian [1] Note that for me, even a major release would hopefully not affect the majority of users or be very disruptive.

11 21

next NumPy Newcomers' Hour - 8 pm UTC
by Inessa Pawson 28 May '23

28 May '23

Our next Newcomers' Hour will be held this Thursday, December 15th at 8 pm UTC. Stop by to ask questions or just to say hi. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/3f3otyyuTte3FU9y3QzsLg?both Join the meeting via Zoom: https://us06web.zoom.us/j/82563808729?pwd=ZFU3Z2dMcXBGb05YemRsaGE1OW5nQT09 -- Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy https://numpy.org/ GitHub: inessapawson

1 9

Giving deprecation of e.g. `float(np.array([1]))` a shot (not 0-d)
by Sebastian Berg 16 May '23

16 May '23

Hi all, Unlike conversions of 0-d arrays via: float(np.array([1])) conversions of 1-D or higher dimensional arrays with a single element are a bit strange: float(np.array([1])) And deprecating it has come up often enough with many in favor, but also many worried about the possible annoyance to users. I decided to give the PR a shot, I may have misread the room on it though: https://github.com/numpy/numpy/pull/10615 So if this turns out noisy (or you may simply disagree), I am happy to revert! There was always the worry that it might be painful for downstream. SciPy, pandas, matplotlib should all be fine (were fixed in the past years). And the fact that SciPy required much more changes than the other gives me some hope that many libraries won't mind. For end-users, I would lean towards taking it slow, but if you see issues there we can also revert of course. Cheers, Sebastian

5 8

Consider linking to Accelerate in Apple Silicon ARM64 wheels
by Clemens Brunner 11 May '23

11 May '23

Hello! I recently got a new MacBook Pro with an M2 Pro CPU (ARM64). When I ran some numerical computations (ICA to be precise), I was surprised how slow it was - way slower than e.g. my almost 10 year old Intel Mac. It turns out that the default OpenBLAS, which is what you get when installing the binary wheel with pip (i.e. "pip install numpy"), is the reason why computations are so slow. When installing NumPy from source (by using "pip install --no-binary :all: --no-use-pep517 numpy"), it uses the Apple-provided Accelerate framework, which includes an optimized BLAS library. The difference is mind-boggling, I'd even say that NumPy is pretty much unusable with the default OpenBLAS backend (at least for the applications I tested). In my test with four different ICA algorithms, I got these runtimes with the default OpenBLAS: - FastICA: 6.3s - Picard: 26.3s - Infomax: 0.8s - Extended Infomax: 1.4s Especially the second algorithm is way slower than on my 10 year old Intel Mac using OpenBLAS. Here are the times with Accelerate: - FastICA: 0.4s - Picard: 0.6s - Infomax: 1.0s - Extended Infomax: 1.3s Given this huge performance difference, my question is if you would consider distributing a binary wheel for ARM64-based Macs which links to Accelerate. Or are there any caveats why you do not want to do that? I know that NumPy moved away from Accelerate years ago on Intel Macs, but maybe now is the time to reconsider this decision. Thanks! Clemens

3 6