Mailman 3 July 2023 - NumPy-Discussion

next NumPy triage meeting - now starting at 5 pm UTC
by Inessa Pawson 26 Nov '23

26 Nov '23

The next NumPy triage meeting will be held this Wednesday, January 11th at 5 pm UTC. (Please note the new time!) This is a meeting where we synchronously triage prioritized PRs and issues. Join us via Zoom: https://numfocus-org.zoom.us/j/82096749952?pwd=MW9oUmtKQ1c3a2gydGk1RTdYUUVX… Everyone is welcome to attend and contribute to a conversation. Please notify us of issues or PRs that you’d like to have reviewed by adding a GitHub link to them in the meeting agenda: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg. -- Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy https://numpy.org/ GitHub: inessapawson

1 23

next NumPy Newcomers' Hour - 12 pm UTC
by Inessa Pawson 31 Oct '23

31 Oct '23

Our next Newcomers' Hour will be held tomorrow, December 1st, at 12 pm UTC. Stop by to ask questions or just to say hi. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/3f3otyyuTte3FU9y3QzsLg?both Join the meeting via Zoom: https://us06web.zoom.us/j/82563808729?pwd=ZFU3Z2dMcXBGb05YemRsaGE1OW5nQT09 -- Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy https://numpy.org/ GitHub: inessapawson

2 5

next NumPy community meeting
by Inessa Pawson 22 Oct '23

22 Oct '23

The next NumPy community meeting will be held this Wednesday, August 17th at 18:00 (6 pm) UTC. Join us via Zoom: https://us06web.zoom.us/j/83278611437?pwd=ekhoLzlHRjdWc0NOY2FQM0NPemdkZz09 Everyone is welcome and encouraged to attend. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both For the notes from the previous meeting, visit: https://github.com/numpy/archive/blob/main/community_meetings/community-202… Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy https://numpy.org/ Twitter: @inessapawson

1 30

Next Documentation team meeting
by Mukulika Pahari 10 Sep '23

10 Sep '23

Hi all! Our next Documentation Team meeting will happen on *Monday, June 5* at ***4PM UTC***. We now alternate the meeting times to be a bit more inclusive. This means that we'll have a meeting at 12pm UTC every 28 days, and a meeting at 4pm UTC every 28 days. All are welcome - you don't need to already be a contributor to join. If you have questions or are curious about what we're doing, we'll be happy to meet you! If you wish to join on Zoom, use this (updated) link: https://numfocus-org.zoom.us/j/85016474448?pwd=TWEvaWJ1SklyVEpwNXUrcHV1YmFJQ... Here's the permanent hackmd document with the meeting notes (still being updated): https://hackmd.io/oB_boakvRqKR-_2jRV-Qjg Hope to see you around! * You can also visit https://scientific-python.org/calendars to add the NumPy community calendar as an .ics file to your preferred calendar manager. * Best wishes, Mukulika

1 5

reminder: put an upper bound on setuptools if you use numpy.distutils!
by Ralf Gommers 31 Aug '23

31 Aug '23

Hi all, After setuptools 65.0 was released a few days ago, all users of numpy.distutils had their builds broken. This is already fixed in setuptools 65.0.2 because the breakage was particularly bad. However, the next breakage may not be fixed anymore (and more breakages *are* expected). So this is a good time to remind you all that you should put an upper bound on the setuptools version you allow in the releases of your package - to the last version that is known to work with your package. Our official stance here is that setuptools versions >=60 are not supported - see the "deprecating numpy.distutils" thread: https://mail.python.org/archives/list/numpy-discussion@python.org/message/P…. Newer versions may work for you, depending on what features you use. They don't for NumPy and for SciPy; both projects pin to 59.X to avoid problems. For the recent issue with setuptools 65.0.0, see https://github.com/numpy/numpy/issues/22135. We have also made the warnings about this topic in our docs more explicit, see https://github.com/numpy/numpy/pull/22154. Cheers, Ralf

2 2

Adding bit_count ufunc
by Doug Turnbull 13 Aug '23

13 Aug '23

👋Long time numpy user, and big fan of all your work TL; DR - there's a `bit_count` method on numpy scalars, and Python ints, I'm advocating for a `ufunc` `bit_count` as has been implemented in this PR: https://github.com/numpy/numpy/pull/21429 A number of people have requested this as a numpy feature and there's a lot of great discussion at this issue https://github.com/numpy/numpy/issues/16325 A bit count comes up in certain similarity situations (like hamming distance). This involves an xor with numpy arrays and a bit count, where the bit count is currently the bottleneck by a few orders of magnitude in my local benchmarking. Currently there's a number of not particularly fast, workarounds for doing this with numpy arrays, like the bit-twiddling solutions here https://stackoverflow.com/a/68943135/8123 https://stackoverflow.com/a/109025/8123 So I'd love to be able to continue the work of bit_count -> numpy array ufunc to get performance gains at the array level 🙏

5 7

The 1.26.x maintenance branch has been created.
by Charles R Harris 31 Jul '23

31 Jul '23

Hi All, The 1.26.x maintenance branch has been created. The 1.26.x branch is a continuation of the 1.25.x branch and serves to mark the change from our distutils based builds to the meson builds needed to support the upcoming Python 3.12 release. Chuck

2 4

NumPy 1.25.2 released
by Charles R Harris 31 Jul '23

31 Jul '23

Hi All, On behalf of the NumPy team, I'm pleased to announce the release of NumPy 1.25.2. NumPy 1.25.2 is a maintenance release that fixes bugs and regressions discovered after the 1.25.1 release. This is the last planned release in the 1.25.x series, the next final release will be 1.26.0, which will use the meson build system and support Python 3.12. The Python versions supported by this release are 3.9-3.11 Note that 32 bit wheels are only provided for Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. Wheels can be downloaded from PyPI <https://pypi.org/project/numpy/1.25.2/>; source archives, release notes, and wheel hashes are available on Github <https://github.com/numpy/numpy/releases/tag/v1.25.2>. *Contributors* A total of 13 people contributed to this release. People with a "+" by their names contributed a patch for the first time. - Aaron Meurer - Andrew Nelson - Charles Harris - Kevin Sheppard - Matti Picus - Nathan Goldbaum - Peter Hawkins - Ralf Gommers - Randy Eckenrode + - Sam James + - Sebastian Berg - Tyler Reddy - dependabot[bot] *Pull requests merged* A total of 19 pull requests were merged for this release. - #24148: MAINT: prepare 1.25.x for further development - #24174: ENH: Improve clang-cl compliance - #24179: MAINT: Upgrade various build dependencies. - #24182: BLD: use ``-ftrapping-math`` with Clang on macOS - #24183: BUG: properly handle negative indexes in ufunc_at fast path - #24184: BUG: PyObject_IsTrue and PyObject_Not error handling in setflags - #24185: BUG: histogram small range robust - #24186: MAINT: Update meson.build files from main branch - #24234: MAINT: exclude min, max and round from ``np.__all__`` - #24241: MAINT: Dependabot updates - #24242: BUG: Fix the signature for np.array_api.take - #24243: BLD: update OpenBLAS to an intermediate commit - #24244: BUG: Fix reference count leak in str(scalar). - #24245: BUG: fix invalid function pointer conversion error - #24255: BUG: Factor out slow ``getenv`` call used for memory policy warning - #24292: CI: correct URL in cirrus.star [skip cirrus] - #24293: BUG: Fix C types in scalartypes - #24294: BUG: do not modify the input to ufunc_at - #24295: BUG: Further fixes to indexing loop and added tests Cheers, Charles Harris

1 0

preparing downstream code for NumPy 2.0 & API deprecations/removals
by Ralf Gommers 31 Jul '23

31 Jul '23

Hi all, This email is about two 2.0 release related topics: 1. Advice/guidance for downstream library authors and end users 2. Strategy for development work around public API changes that will be breaking backwards compatibility. I also just created https://github.com/numpy/numpy/issues/24300 as a tracking issue that we can post announcements on and anyone can subscribe to. I imagine many folks will want some way to follow along and be notified of important changes around the release, but not subscribe to this mailing list. Some of the content of that issue overlaps with this email. In case of questions/comments about the content of that issue, let's discuss it here. Please keep that tracking issue for announcements, not for technical discussion. **Advice or downstream package authors and end users** 1. If you rely on the NumPy C API (e.g. via direct use in C/C++, or via Cython code that uses NumPy), please add a `numpy<2.0` requirement in your package's dependency metadata. Rationale: the NumPy C ABI will change in 2.0, so any compiled extension modules that rely on NumPy are likely to break, they need to be recompiled. 2. If you rely on a large API surface from NumPy's Python API, also consider adding the same ` numpy<2.0` requirement to your metadata. Rationale: we will do a significant cleanup (see NEP 52), so unless you only use modern/recommended functions and objects, your code is likely to require at least some adjustments. 3. Consider cleaning up your code. E.g. remove `from numpy import *`, or importing any private modules like `numpy.core`. See https://github.com/numpy/numpy/blob/main/numpy/tests/test_public_api.py#L11… for what we consider public/private. If it's not in the NumPy docs or in the list of public modules there, don't use it! 4. Plan to do a release of your own packages which depend on `numpy` shortly after the first NumPy 2.0 release candidate is released (probably in Dec 2023). Rationale: at that point, you can release packages that will work with both 2.0 and 1.X, and hence your own end users will not be seeing much/any disruption (you want `pip install mypacackage` to continue working on the day NumPy 2.0 is released). 5. Consider testing against NumPy nightlies in your own CI. We publish those at https://anaconda.org/scientific-python-nightly-wheels/numpy, and have documented that as a stable location at https://numpy.org/devdocs/dev/depending_on_numpy.html. Rationale: this will detect potential issues in your code so you can fix them well ahead of the NumPy 2.0 release. **Strategy for public API changes for 2.0** Based on experience over the past weeks with adding deprecations and making breaking changes, I think it'd be good to articulate a strategy for Python/C API changes. We are not yet collectively used to the change of pace that the run-up to a major release is. I think we want to use and balance these two principles: 1. Make the API and behavior changes that we want to see for 2.0, in a way that doesn't incur unreasonable amounts of effort or get completely blocked by backwards compatibility constraints which we'd apply for a regular minor release. 2. Mitigate the inevitable disturbances for downstream projects and end users as best as we can. To start with (2), issuing good guidance (like in the section above and the tracking issue gh-24300) is one way to help. Another one is to, when you are in the process of making a change, use code search tools and either code that will break downstream proactively or at least notify downstream project authors. The former can be done by sending PRs to at least the largest projects (SciPy, Pandas, scikit-learn, scikit-image, Matplotlib) when there are easy changes to make. And otherwise by filing issues on the issue tracker of other projects. Yet another way, in case of mechanical changes like removal of aliases, would be to provide a sed script that others can run to automatically update their code as much as possible. For (1), I think it's important to understand that 2.0 is a one-off change of pace where our regular backwards compatibility policy does not apply. If particular functions are desirable to touch but widely used, it may be wise to leave them in or deprecate them of course - this is a case by case decision. However, it doesn't have to be done like that. Every single object in the NumPy API is used somewhere, and removing it is going to affect some users/packages. This is inevitable, and we can't achieve our 2.0 goals if every little niche API change is going to require following our regular backwards compatibility strategy. The only thing that would do is to make 2.2 the breaking release. Many downstream libraries have CI setups that turn deprecation warnings into hard errors, hence often it doesn't matter whether we deprecate something in `main`, or remove it straight away. Apply good judgement here I'd say (how widely is something used, is the replacement trivial or does it require some thought, etc.). Here are four recent examples where we had downstream breakage and either did extra work to revert a change or discussed doing so: - Adding np.min, np.max and np.round to the `__all__` dict: this already happened in 1.25.0, and we reverted it after discussion in https://github.com/numpy/numpy/pull/24234 (but left it in the `main` branch for 2.0). - Removing `np.cast` in `main`: this broke SciPy and as a result also JAX/MNE-Python/AstroPy CI. We left it in, but discussed in https://github.com/numpy/numpy/pull/24144 whether or not to revert the removal. - Moving PyArray_MIN/PyArray_MAX to a different header file: this broke SciPy's CI. We left it as is and fixed up the issue in SciPy in https://github.com/numpy/numpy/pull/24234 - Removal of `np.byte_bounds`: we removed it from the main namespace in https://github.com/numpy/numpy/pull/23830, and after discussion on that issue are bringing it back for now (first in its old location in https://github.com/numpy/numpy/pull/24154, and then moving it to a new namespace under `np.lib` once we sort that out). This is going to happen a lot more I'm sure. We have to be careful about what we do, but also we need downstream library authors to be proactive, audit their code and use the nightlies we publish. For very niche APIs like `np.cast` we should expect to not have to justify removal of them in detail or entertain reverting changes made in preparation of 2.0. Final thought: we should get a 1.26.0b1 beta release out shortly, but will likely have a couple of weeks before 1.26.0rc1. So any deprecations we want to put into 1.26.0 can go in until the RC1 release. Cheers, Ralf

1 0

How to get numpy.distutils.fcompiler to find my absoft compiler
by Samuel H Dupree Jr 30 Jul '23

30 Jul '23

Hello everyone, I'm trying to wrap some Fortran77 subroutines from the Math77 library using f2py3. f2py3 responds with a message: File "/Users/user/opt/anaconda3/lib/python3.9/site-packages/numpy/distutils/fcompiler/__init__.py", line 426, in get_version raise CompilerNotFound() numpy.distutils.fcompiler.CompilerNotFound I'm running Python ver. 3.9.17 under the Anaconda distribution (anaconda Command line client (version 1.12.0)) on a Mac Pro (2019) under Mac OS X Ventura ver. 13.4.1. The Fortran compiler is Absoft Pro Fortran ver. 22.0. The set of subroutines to be wrapped were concatenated into a single file named divapy.f. The command used to wrap the routines was --fcompiler=absoft --f77exec=/Applications/Absoft22.0/bin/f77 -c -m diva divapy.f I've attached a file containing the complete set of messages output by f2py3 using the command above. Any suggestions as to how to fix this problem? Sam Dupree.

2 2