Mailman 3 September 2025 - NumPy-Discussion

next NumPy Optimization Team meeting - Monday, January 20th, 2025 at 5 pm UTC
by Inessa Pawson June 5, 2026

June 5, 2026

Hi, all! The next NumPy Optimization Team meeting will be held on Monday, January 20th at 17:00 UTC. Join us via Zoom: https://numfocus-org.zoom.us/j/81261288210?pwd=iwV99tGSjR61RTGEERKM4QKxe46g… Everyone is welcome and encouraged to attend. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/dVdSlQ0TThWkOk0OkmGsmw?both For the notes from the previous meetings, visit: https://github.com/numpy/archive/tree/main/optim_team_meetings -- Cheers, Inessa Inessa Pawson GitHub: inessapawson

1 23

next NumPy community meeting - Wednesday, April 9th, 2025 at 5 pm UTC - now meeting ONE HOUR EARLIER
by Inessa Pawson May 16, 2026

May 16, 2026

The next NumPy community meeting will be held this Wednesday, April 9th at 17:00 (5 pm) UTC. Please note the TIME CHANGE! Join us via Zoom: https://numfocus-org.zoom.us/j/83278611437?pwd=ekhoLzlHRjdWc0NOY2FQM0NPemdk… Everyone is welcome and encouraged to attend. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both For the notes from the previous meetings, visit: https://github.com/numpy/archive/tree/main/community_meetings -- Cheers, Inessa Inessa Pawson GitHub: inessapawson

1 29

Fixing definition of reduceat for Numpy 2.0?
by Martin Ling Feb. 27, 2026

Feb. 27, 2026

Hi folks, I don't follow numpy development in much detail these days but I see that there is a 2.0 release planned soon. Would this be an opportunity to change the behaviour of 'reduceat'? This issue has been open in some form since 2006! https://github.com/numpy/numpy/issues/834 The current behaviour was originally inherited from Numeric, and makes reduceat often unusable in practice, even where it should be the perfect, concise, efficient solution. But it has been impossible to change it without breaking compatibіlity with existing code. As a result, horrible hacks are needed instead, e.g. my answer here: https://stackoverflow.com/questions/57694003 Is this something that could finally be fixed in 2.0? Martin

6 12

next NumPy triage meeting - Wednesday, May 28th, 2025 at 5 pm UTC
by Inessa Pawson Jan. 4, 2026

Jan. 4, 2026

The next NumPy triage meeting will be held this Wednesday, May 28th at 17:00 (5 pm) UTC. This is a meeting where we synchronously triage prioritized PRs and issues. Everyone is welcome to attend and contribute to a conversation. Join us via Zoom: https://numfocus-org.zoom.us/j/82096749952?pwd=MW9oUmtKQ1c3a2gydGk1RTdYUUVX… Please notify us of issues or PRs that you’d like to have reviewed by adding a GitHub link to them in the meeting agenda: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg -- Cheers, Inessa Inessa Pawson GitHub: inessapawson

1 14

ENH: Add a non-circular shift function
by Carlos Martin Nov. 23, 2025

Nov. 23, 2025

Feature request: Add a non-circular shift function that is similar to [`numpy.roll`](https://numpy.org/doc/stable/reference/generated/numpy.roll.html) but, instead of wrapping elements around, replaces unoccupied entries with a specified fill value. Examples: ``` >>> import numpy as np >>> a = np.arange(1, 6) >>> a array([1, 2, 3, 4, 5]) >>> np.shifted(a, 2) array([0, 0, 1, 2, 3]) >>> np.shifted(a, -2) array([3, 4, 5, 0, 0]) >>> a = np.arange(1, 10).reshape((3, 3)) >>> a array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]) >>> np.shifted(a, (1, -2), (0, 1)) array([[0, 0, 0], [3, 0, 0], [6, 0, 0]]) ``` Prior discussion: https://github.com/numpy/numpy/issues/26220 Pull request: https://github.com/numpy/numpy/pull/29389

1 1

dropping Intel x86-64 wheels for the next release?
by Ralf Gommers Oct. 23, 2025

Oct. 23, 2025

Hi all, GitHub Actions has deprecated the macos-13 image [1], which was the last image to support Intel x86-64 runners. As a result, we won't be able to use our current wheel build setup. A quick sampling of what others are doing with their support policies: CPython dropped it to Tier 2 [2], Anaconda dropped support [3], conda-forge hasn't decided anything yet [4], PyTorch dropped support already 1.5 years ago [5], Numba is dropping wheels [6]. We will of course continue to accept issues and PR and keep things working for macOS x86-64 either way, just like for many other platforms that either don't have good CI support available or are more niche and for which we don't supply wheels to begin with. However, for wheels we do ship on PyPI, it seems like there should be high enough demand and a good case. It would probably be possible to cross-compile and test x86-64 wheels on arm64 runners with Rosetta. There are a number of downsides to that though, from extra maintenance load, slower CI (the x86-64 builds are already the slowest ones except for PyPy which we'll drop soon as well), and some things potentially not being testable/tested (e.g., the translation to arm code under the hood won't actually exercise SIMD code I'd think). Shipping official wheels that aren't normally testable feels a bit questionable. It's easy enough nowadays to pick older wheels by the user, and `pip install numpy --no-binary` will continue to work just fine. And given that support across the ecosystem will very likely quickly fall away, it feels like effort that could be better spent elsewhere. Hence I'd like to propose to drop macOS x86-64 wheels for the 2.4.0 release series. Cheers, Ralf [1] https://github.blog/changelog/2025-07-11-upcoming-changes-to-macos-hosted-r… [2] https://discuss.python.org/t/dropping-intel-mac-to-tier-2/102100/1 [3] https://www.anaconda.com/blog/intel-mac-package-support-deprecation [4] https://github.com/conda-forge/conda-forge.github.io/issues/2467 [5] https://dev-discuss.pytorch.org/t/pytorch-macos-x86-builds-deprecation-star… [6] https://github.com/numba/numba/issues/10187

9 13

Improving the Thread Safety of NumPy's Test Suite
by Britney Whittington Oct. 17, 2025

Oct. 17, 2025

Hello! My name is Britney Whittington, and I'm an intern at Quansight working with Nathan Goldbaum. This is my first time working with an open-source project like NumPy, and I'm excited to be here! For my project, I am working on making the NumPy test suite more thread safe, to improve its support of Python's free-threaded build of Python 3.14. To do this, I have been using `pytest-run-parallel` [1]. You can see the README of the plugin for more details, but briefly it runs each test in a test suite many times in a thread pool. This exercise results in failures in the test suite, often due to thread-safety issues in pytest itself, or thread-safety issues with NumPy due to use of global state in the NumPy implementation. As we work on making the test suite more thread safe, we have exposed thread safety issues in NumPy. Party of the difficulty of using `pytest-run-parallel` is that Pytest itself and the constructs provided by it are not thread safe [2], so it takes some effort to make a test suite as big as NumPy's run under `pytest-run-parallel` without any failures. We've already merged some work along these lines. You may have seen some PRs I've submitted so far related to this project [3]. We are getting close to turning on `pytest-run-parallel` in CI. Before we do that, we want to make sure there aren't any objections. So far, I've made two major changes to the test suite: ---------------------------------------------------- 1. Refactoring xunit Setup and Teardown Methods ---------------------------------------------------- #### Problem - NumPy makes heavy use of pytest's xunit setup and teardown methods [4], which are mentioned in the testing guidelines [5]. - When using `pytest-run-parallel`, pytest will try to run the teardown before all threads complete running a test. #### Solutions - Replace setup/teardown with fixtures. While fixtures play more nicely with threads, they are still shared between threads. If one thread mutates a fixture, this mutation will carry over to all the other threads. Additionally, dependency injection via a fixture is fundamentally *hard to debug and understand at a glance*. - Use **explicit setup**. Instead of pytest calling setup methods, we modify the setup methods to be manually called in each test. This fixes the teardown issue, and allows us to declare variables locally and not worry about mutations between threads. Currently we are favoring the usage of explicit setup. Of course, this may not work for every xunit setup. For more complex cases, fixtures may be more useful, or context managers. ------------------------------------------- 2. Refactoring Global np.random Calls ------------------------------------------- #### Problem - Calls to `np.random` use the same global instance. This results in errors with tests that heavily rely on seeded results, due to threads sharing the same global RNG state. While this may not necessarily cause failures, we also feel that it's a fundamentally bad practice for tests to rely on global state like this. #### Solution - Instead of `np.random`, each test should use a local instance of `np.random.RandomState`, so that threads can increment through their own local RNG stream. I have made this change to the tests that fail under `pytest-run-parallel` [7], and am working on making all test calls to `np.random` local. Note that `RandomState` uses the same MT RNG that the global RNG uses, so the RNG streams are the same as before. =============================== ----------------------------------------- > What Does This Mean Going Forward? ----------------------------------------- By refactoring the test suite to be more thread safe, if we'd like to add `pytest-run-parallel` CI, contributors may need to write tests in a somewhat different style. It is possible for `pytest-run-parallel` to fix some of these issues on its side, such as making it so xunit setup runs properly. However, with the current state of pytest and the plugin, this will require a lot of work, time, and maintenance. It also may make it more difficult to improve the thread safety of pytest itself in the future. See this issue [8] for further discussion. We think that a refactor of the NumPy test suite is more straightforward for now, and can always be reverted once `pytest-run-parallel` develops a way to handle thread-unsafe setup fixtures. -------------------- > Testing Guidelines -------------------- In addition to make the tests thread safe, I'd like to update the testing guidelines [9]. Some things can be clarified (such as NumPy's opinion on fixture usage) and it would be good to update it with current best practices, and, if folks are open to it, guidelines on writing thread-safe tests. [1] https://github.com/Quansight-Labs/pytest-run-parallel [2] https://docs.pytest.org/en/stable/explanation/flaky.html#thread-safety [3] https://github.com/numpy/numpy/issues/29552 (mentioned by all PRs related to this project) [4] https://docs.pytest.org/en/stable/how-to/xunit_setup.html [5] https://numpy.org/doc/stable/reference/testing.html#easier-setup-and-teardo… [6] https://docs.pytest.org/en/stable/explanation/fixtures.html [7] https://github.com/numpy/numpy/pull/29729 [8] https://github.com/Quansight-Labs/pytest-run-parallel/issues/14 [9] https://numpy.org/doc/stable/reference/testing.html

2 4

Regarding nparray reshape() and refcount
by 腾刘 Sept. 30, 2025

Sept. 30, 2025

Hi all, In one of numexpr's Issue: https://github.com/pydata/numexpr/issues/521 a user points out an unexpected behavior regarding refcount of a ndarray. After some experiments I found that it can be replicated in a small example (https://dpaste.com/337WPLFCM): ``` import numpy as np import sys A = np.arange(100) B = A out = B print("real number of ref of A before del B: ", sys.getrefcount(A) - 1) del B print("real number of ref of A after del B: ", sys.getrefcount(A) - 1) del out print("real number of ref of A after del out: ", sys.getrefcount(A) - 1) print('===========') A1 = np.arange(100) B1 = A1.reshape((10, 10)) print("real number of ref of A1 before del B1: ", sys.getrefcount(A1) - 1) del B1 print("real number of ref of A1 before del B1: ", sys.getrefcount(A1) - 1) print('===========') A2 = np.arange(100) B2 = A2.reshape((10, 10)) out2 = B2 print("real number of ref of A2 before del B2: ", sys.getrefcount(A2) - 1) print("real number of ref of B2 before del B2: ", sys.getrefcount(B2) - 1) print("real number of ref of out2 before del B2: ", sys.getrefcount(out2) - 1) del B2 print("real number of ref of A2 after del B2: ", sys.getrefcount(A2) - 1) print("real number of ref of out2 after del B2: ", sys.getrefcount(out2) - 1) del out2 print("real number of ref of A2 after del out2: ", sys.getrefcount(A2) - 1) print('===========') A3 = np.arange(100) B3 = A3.reshape((10, 10)) out3 = B3 print("real number of ref of A3 before del out3: ", sys.getrefcount(A3) - 1) del out3 print("real number of ref of A3 after del out3: ", sys.getrefcount(A3) - 1) del B3 print("real number of ref of A3 after further del B3: ", sys.getrefcount(A3) - 1) ``` The output is: ``` ❯ python test_reshape.py real number of ref of A before del B: 3 real number of ref of A after del B: 2 real number of ref of A after del out: 1 =========== real number of ref of A1 before del B1: 2 real number of ref of A1 before del B1: 1 =========== real number of ref of A2 before del B2: 2 real number of ref of B2 before del B2: 2 real number of ref of out2 before del B2: 2 real number of ref of A2 after del B2: 2 real number of ref of out2 after del B2: 1 real number of ref of A2 after del out2: 1 =========== real number of ref of A3 before del out3: 2 real number of ref of A3 after del out3: 2 real number of ref of A3 after del B3: 1 ``` It seems that B2's ref of A2 is stolen by this assignment out = B2. And In A3 , B3 case, we deleteout first and it seems that the ref in out is returned back to B3. I'm not sure if this is the intended behavior and I cannot find any material or documentation about this. Any thoughts or suggestions would be helpful! Teng Liu

1 0

ANN: numexpr 2.13.0 released
by Francesc Alted Sept. 24, 2025

Sept. 24, 2025

========================= Announcing NumExpr 2.13.0 ========================= Hi everyone, NumExpr 2.13.0 introduced a bunch of new features including new bitwise operators (&, |, ^, ~), floor division (//). It also adds many new functions (like hypot, log2, maximum, minimum, nextafter...). Thanks to Luke Shaw for these contributions. Project documentation is available at: https://numexpr.readthedocs.io/ Changes from 2.12.1 to 2.13.0 ----------------------------- * New functionality has been added: * Bitwise operators (and, or, not, xor): `&, |, ~, ^` * New binary arithmetic operator for floor division: `//` * New functions: `signbit`, `hypot`, `copysign`, `nextafter`, `maximum`, `minimum`, `log2`, `trunc`, `round` and `sign`. * Also enables integer outputs for integer inputs for `abs`, `fmod`, `copy`, `ones_like`, `sign` and `round`. Thanks to Luke Shaw for the contributions. * New wheels for Python 3.14 and 3.14t are provided. What's Numexpr? --------------- Numexpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like "3*a+4*b") are accelerated and use less memory than doing the same calculation in Python. It has multi-threaded capabilities, as well as support for Intel's MKL (Math Kernel Library), which allows an extremely fast evaluation of transcendental functions (sin, cos, tan, exp, log...) while squeezing the last drop of performance out of your multi-core processors. Look here for a some benchmarks of numexpr using MKL: https://github.com/pydata/numexpr/wiki/NumexprMKL Its only dependency is NumPy (MKL is optional), so it works well as an easy-to-deploy, easy-to-use, computational engine for projects that don't want to adopt other solutions requiring more heavy dependencies. Where I can find Numexpr? ------------------------- The project is hosted at GitHub in: https://github.com/pydata/numexpr You can get the packages from PyPI as well (but not for RC releases): http://pypi.python.org/pypi/numexpr Documentation is hosted at: http://numexpr.readthedocs.io/en/latest/ Share your experience --------------------- Let us know of any bugs, suggestions, gripes, kudos, etc. you may have. Enjoy data!

1 0

Have any PRs in the last week or so touched objects that are followed by ResourceTracker
by Andrew Nelson Sept. 22, 2025

Sept. 22, 2025

Hi all, in the last few weeks have there been any PRs that have touched objects that are followed by the Python ResourceTracker. I'm talking about things like Locks, semaphores, mutexes, etc? I'm seeing some weird behaviour in scipy CI where a test ( https://github.com/scipy/scipy/blob/main/scipy/optimize/tests/test_optimize…) that uses an object that wraps `multiprocessing.Pool` has started giving test fails. A typical log ( https://github.com/scipy/scipy/actions/runs/17925659616/job/50971069339#ste…) has text like: "multiprocessing.resource_tracker.ReentrantCallError: Reentrant call into the multiprocessing resource tracker" and "UserWarning: ResourceTracker called reentrantly for resource cleanup, which is unsupported. The semaphore object '/mp-380ciby1' might leak." In my investigations (https://github.com/andyfaff/scipy/pull/83) it only seems to happen for cp312 and the numpy nightly wheel. It doesn't seem to trigger for cp312+np2.3.3 or for cp311. Given that it only seems to happen with the numpy nightly wheel I'm wondering if it's therefore a numpy issue. As far as I can tell it's started happening in the last five days or so. -- _____________________________________ Dr. Andrew Nelson _____________________________________

2 1