
Hi all, Now that 1.22.0 is out the door, I think it's time to deprecate `numpy.distutils` completely. A brief summary for context: - `distutils` was deprecated in Python 3.10, and will be removed in Python 3.12 (Oct 2023 release date) - `setuptools` has re-enabled its vendored copy of `distutils` in its 60.0 release. It's incompatible with `numpy.distutils` and many more breaking changes are to be expected. - Rebasing `numpy.distutils` on top of `setuptools` will be a moving target. More importantly, `numpy.distutils` still has serious and hard to fix bugs and design issues. For example, we cannot build SciPy in parallel due to race conditions in the Fortran support code, and there's a very elusive bug where building an extension module which links in a static Fortran-only library is underlinked and breaks at runtime in dev environments. My initial plan was to contribute Fortran support and other `numpy.distutils` features to `setuptools`, but I've personally changed my mind on that and don't plan to do that work. I think `setuptools` is moving too slow for this to be a productive exercise, and the design of distutils is fundamentally unfixable. So it's a much better investment of time to help projects migrate away from `numpy.distutils` (and hence from `setuptools`). The tracking issue for this is https://github.com/numpy/numpy/issues/18588. Please see the comments and linked issue/thread for more details. My proposal is: 1. deprecate `numpy.distutils` now in main. 2. keep it around for another 2 years after the 1.23.0 release. On >=3.12 it will raise an informative error when someone tries to import it. 3. Only setuptools < 60.0 is supported. This is the status as of today - unless there's an issue with Python 3.11 support, I don't see a point in spending time on keeping up with new setuptools releases, a pin is just fine. 4. Write a docs page with migration info on how to deal with the deprecation, and update it before the 1.23.0 release. Note that the setuptools maintainers have said that they are happy to consider new features if people want to integrate them. The most popular/useful features for `numpy.distutils` are (1) Fortran support, and (2) nested setup.py support. The former is probably not a good idea to integrate, the latter may be. There aren't many packages who need BLAS/LAPACK support anymore, scikit-learn/statsmodels/etc. switched to the Cython/C API exposed by SciPy, rather than linking to BLAS/LAPACK directly. For NumPy itself, we should move to Meson after the SciPy 1.9.0 release has gone out and ironed out all the remaining issues there. We can revisit that in 6 months or so. Thoughts? Cheers, Ralf