Mailman 3 October 2023 - SciPy-Dev

Next SciPy newcomers' meeting - TODAY October 13th at 8pm UTC
by Melissa Mendonça 29 Feb '24

29 Feb '24

Hello, folks! Our next newcomers meeting happens TODAY, October 13th, at 8pm UTC. Feel free to join to ask questions about contributing, our community, follow ups on existing issues or pull requests, or to just chat. If you are a maintainer or a regular contributor, you are also welcome to join as well to connect with other members of our community :) Here's the hackmd we use to take notes, which includes the meeting details: https://hackmd.io/@melissawm/scipy-newcomers And here's the link if you wish to join us on Zoom: https://us06web.zoom.us/j/89085469494?pwd=dVFwK3dVczRyRUowZWZkckFtTmloUT09 Hope to see you around! Cheers, Melissa

1 2

RFC: decouple doctesting from refguide-check
by Evgeni Burovski 23 Feb '24

23 Feb '24

Hi, Code examples in the SciPy and NumPy documentation are doctested, with a modified doctesting machinery which understands floating point, numpy formatting and some details of our documentation and API. Our modified doctesting machinery is buried in the refguide-check utility, which also does several unrelated things, all of which are tightly coupled to each other and to the libraries themselves. It has been sometimes described as hard to understand or work with or extend due to this tight coupling and a lack of a dedicated test suite. The numpy and scipy versions of the utility are both vendored, and the numpy version has diverged from the scipy version somewhat. Following a discussion in https://github.com/numpy/numpy/issues/21070 I did a small experiment to decouple the doctesting into a separate package, so that it's easier to consolidate the two versions. Plus, a separate repo is generally easier to maintain, configure, and possibly extend or adopt to other projects. The work-in-progress result is here: https://github.com/ev-br/scpdt It currently can run the full doctesting of the scipy API documentation (docstrings of objects) --- see https://github.com/ev-br/scpdt/pull/33 (the log of a test run with warnings turned to errors is in the GH actions: https://github.com/ev-br/scpdt/runs/6743881766?check_suite_focus=true). Note that it shows, among other things, a couple of deprecation warnings our docs have accumulated :-). The API of the tool closely follows that of the standard library doctest module and provides (nearly) drop-in replacements for doctest checking, parsing, finding and running. Various configuration options for our modifications are collected into a single bag object which is internally passed around. This way, it's user-configurable all the way from a plain standard doctest module behavior to what refguide-check does now. There are a couple of wrinkles to iron out; overall it does what refguide-check does already. One missing bit is doctesting rst or other text files, but it's coming soon. The current plan is to: - verify that the standalone version does not miss things checked by the refguide-check - plumb it through the SciPy dev interface and rip out the refguide-check bundled doctesting utilities. - Sync changes that NumPy version of refguide-check accumulated over time - Make sure it correctly tests the NumPy docs, too. - Better document the internals, there is currently only a readme file. If someone's interested to join me working on these, great, the more the merrier :-). I think it could make sense to move the tool's repository to the scipy github org (or maybe even numpy org?). I'm offering to maintain it regardless of the location. Thoughts? Cheers, Evgeni

5 8

Deprecate scipy.misc and Add scipy.datasets submodule
by Anirudh Dagar 07 Jan '24

07 Jan '24

Hi all, I just filed https://github.com/scipy/scipy/pull/15607 after an offline discussion with Ralf earlier this month. The summary of that PR is: This is a good time to get rid of the `scipy.misc` submodule since it houses only a handful of functions. As of today, most of the methods have already been deprecated from the `misc` module. They have generally moved under some other submodule. In this case, it is best to create a `scipy.datasets` submodule that can be home to the dataset functions (ascent, face, ecg). Adding a datasets submodule is not a new idea, and was proposed a couple of years ago by Warren (see https://github.com/scipy/scipy/pull/8707), but unfortunately, it never went ahead. Adding this submodule is not only sensible from an organizational/structural point of view but will also help reduce the wheel size of SciPy package. This is possible if we decouple the dataset files from the SciPy repository making use of Pooch to download and cache datasets kept in separate repos. See the PR for more details. I also filed https://github.com/scipy/scipy/issues/15608 Out of the 5 methods in `scipy.misc`, three dataset functions can be moved to this datasets submodule and will slowly be deprecated from misc. The question about the other two methods i.e (derivative, central_diff_weights) is up for discussion and I'd like to understand what will be the best way to deprecate them. Should we move them to a submodule like integrate (keeping derivative makes sense to me)? On the other hand, considering these two methods are not used extensively so one possibility is to deprecate both of them completely and remove them from SciPy. Please share your opinion/concerns, and comment if you see a potential issue with this. Cheers, Anirudh

4 10

SciPy Community Meeting: Wednesday, October 18th at 2pm UTC
by Melissa Mendonça 14 Nov '23

14 Nov '23

Hi all, The next SciPy Community Meeting will happen on* Wednesday October 18th at 2pm UTC*. Here's the hackmd we are using to take notes: https://hackmd.io/pyhudrC5TgSwdJtpPldHgQ If you wish to join on Zoom, use this link: https://us06web.zoom.us/j/6345425936?pwd=aDVFQzVmbk9SVU5jU0Jwc0s3YWUrdz09 Our community calendar is also available here: https://scientific-python.org/calendars/ Cheers, Melissa

1 3

Deprecating positional arguments for functions whose signatures are being already
by h.vetinari＠gmx.com 02 Nov '23

02 Nov '23

Dear all, In the context of maintaining our APIs, previous discussions recognized the advantages of using keyword-only arguments where appropriate, but it was decided that a "big bang" change of all our APIs is not desired, and so the current policy [1] states that this should only be done where APIs are already being changed _anyway_, for example because a certain argument in the signature is being removed (which would make positional-only use of all following arguments invalid). In the context of the ongoing deprecation work, we (meaning mainly myself and @j-bowhay) have taken this to mean "introduce the keyword-only cut-off at the _natural_ point in the signature". Obviously there is some degree of subjectivity in that, but I consider it strictly worse than doing nothing to introduce the cut-off between positional-or-keyword and keyword-only at exactly the point where a given deprecated argument gets removed. To me it is a binary choice "whether to make a function keyword-only at its natural point or not". To illustrate this distinction, let's imagine a function def func(x, y, z, option_A=..., option_B=..., option_C=...): ... Let's assume that option_B has become obsolete, or gets renamed, then - strictly speaking - only option_C would need to become keyword-only, and not option_A. However, I would argue that the only reasonable point to make this function keyword only is as follows: def func(x, y, z, *, option_A=..., option_C=...): ... This results in a "net additional deprecation" of positional usage of option_A vis-à-vis the most minimal deprecation to keep things working as before that would look as follows: def func(x, y, z, option_A=..., *, option_C=...): ... This is worse IMO because it creates an inconsistency in how conceptually equivalent arguments are treated in the API (one can be used positionally, one cannot), and risks having to _again_ deprecate something in the function if anything about option_A changes in the future. Obviously, not all examples are as clear-cut, and more than that, sometimes a given deprecation only affects a subset of functions that belong to an interrelated family, where the API should be consistent overall. More context and discussion about this can be found in [2]. For 1.12, we have taken all the functions being touched by deprecations and handled them accordingly (in accordance with their respective owners, in case the question about where to place the cut-off wasn't self-evident), which happened early in 1.12 development. Currently, the affected functions are (see also release notes): * linalg.{eigh, eigvalsh, pinv} * integrate.simpson * signal.{firls, firwin, firwin2, remez} * sparse.linalg.{bicg, bicgstab, cg, cgs, gcrotmk, gmres, lgmres, minres, qmr, tfqmr} * special.comb * stats.kendalltau It's worth noting that in some cases we delayed signature changes already announced for 1.12 or 1.13 to 1.14, in order to give people time to _also_ react to the fact that they now need to change away from positional usage of some keyword arguments in those functions. There was some discussion in [2] about these additional deprecations (e.g. the option_A case in the example above), resp. whether we should revisit the choice about doing a wholesale change of our APIs like e.g. scikit-learn did with SLEP009 [3]. I agreed to bring up this topic on the mailing list - please let us know your thoughts. The three broad choices (as far as I can tell) are: 1. wholesale change à la SLEP009 (this will need more discussion and a concrete plan of action IMO) 2. keep doing incremental changes as currently present in main (but changing over individual functions wholesale and not piecemeal w.r.t. their signature) 3. revert the change to keyword-only arguments for the above-mentioned functions Option 2. represents the status quo of both the current documented policy and the state of the repo on main. Best regards H. [1] https://docs.scipy.org/doc/scipy/dev/missing-bits.html#required-keyword-nam… [2] https://github.com/scipy/scipy/issues/18703 [3] https://github.com/scikit-learn/scikit-learn/issues/15005

4 3

Let's make it official then... recommend max line length 88
by Warren Weckesser 01 Nov '23

01 Nov '23

SciPy is documented to follow the PEP 8 style guidelines; see https://docs.scipy.org/doc/scipy/dev/contributor/pep8.html#pep8-scipy PEP 8 recommends a maximum line length of 79 characters. The degree of care with which we follow this guideline has varied quite a bit over the years, and there is currently a lot code where the guideline is violated. Lately, in particular, the enforcement of this guideline has been relaxed, and folks seem to be settling on 88 characters as a reasonable maximum. This is partly a side effect of the replacement of `pycodestyle` in CI with `ruff`; the default maximum line length in `ruff` is 88. This means new lines of code with lengths between 80 and 88 are not being flagged in CI. (I don't recall if CI currently fails with lines longer than 88 characters.) If everyone is satisfied with a maximum of 88 characters, let's make it official and update our developer documents to state this explicitly. What do you say, yea or nay? Warren

8 9

Deprecation of scipy.special.sinc
by Daniel Schmitz 21 Oct '23

21 Oct '23

Hi all, scipy.special.sinc is only an alias for numpy.sinc. In this PR <https://github.com/scipy/scipy/pull/19416>, I opened a PR to deprecate it (removal in SciPy 1.14). Please let me know your thoughts on this deprecation. I imagine that this function is used commonly in the wild but the only change needed by users would to switch an import statement. Best, Daniel

3 2

Implement mixture distribution
by Ruggero Turra 20 Oct '23

20 Oct '23

Hello, I have created a draft pull request about the possibility of having mixture distribution in scipy.stats. The implementation is a bit naive, details are in the pull request description. I would be happy to discuss it. https://github.com/scipy/scipy/pull/19413 Ruggero

1 0

Proposal to rewrite special function kernels written in Cython in C++
by Albert Steppi 18 Oct '23

18 Oct '23

Hi team, I've submitted an RFC issue https://github.com/scipy/scipy/issues/19404, asking for thoughts about rewriting special functions written in Cython into C++. The primary reason for doing so would be to allow other array libraries such as CuPy, PyTorch, or Jax to make use of such functions without having to reimplement them. Irwin Zaid has pointed out that if we provide C++ header files with these functions, then it would be straightforward to include them in other libraries. The impetus for this comes from the desire for an array API extension for special functions. Irwin has generously offered to review PRs for this, and I would be willing to review any PRs of his, so reviewer bandwidth shouldn't be an issue. Feel free to chime in at the issue linked above. Best, Albert

1 0

NumPy 1.26.1 released
by Charles R Harris 14 Oct '23

14 Oct '23

Hi All, On behalf of the NumPy team, I'm pleased to announce the release of NumPy 1.26.1. NumPy 1.26.1 is a maintenance release that fixes bugs and regressions discovered after the 1.26.0 release. In addition, it adds new functionality for detecting BLAS and LAPACK when building from source. Highlights are: - Improved detection of BLAS and LAPACK libraries for meson builds - Pickle compatibility with the upcoming NumPy 2.0. The Python versions supported by this release are 3.9-3.12. Wheels can be downloaded from PyPI <https://pypi.org/project/numpy/1.26.1/>; source archives, release notes, and wheel hashes are available on Github <https://github.com/numpy/numpy/releases/tag/v1.26.1>. The release notes have documentation of the new meson functionality. *Contributors* A total of 13 people contributed to this release. People with a "+" by their names contributed a patch for the first time. - Andrew Nelson - Anton Prosekin + - Charles Harris - Chongyun Lee + - Ivan A. Melnikov + - Jake Lishman + - Mahder Gebremedhin + - Mateusz Sokół - Matti Picus - Munira Alduraibi + - Ralf Gommers - Rohit Goswami - Sayed Adel *Pull requests merged* A total of 20 pull requests were merged for this release. - #24742: MAINT: Update cibuildwheel version - #24748: MAINT: fix version string in wheels built with setup.py - #24771: BLD, BUG: Fix build failure for host flags e.g. ``-march=native``... - #24773: DOC: Updated the f2py docs to remove a note on -fimplicit-none - #24776: BUG: Fix SIMD f32 trunc test on s390x when baseline is none - #24785: BLD: add libquadmath to licences and other tweaks (#24753) - #24786: MAINT: Activate ``use-compute-credits`` for Cirrus. - #24803: BLD: updated vendored-meson/meson for mips64 fix - #24804: MAINT: fix licence path win - #24813: BUG: Fix order of Windows OS detection macros. - #24831: BUG, SIMD: use scalar cmul on bad Apple clang x86_64 (#24828) - #24840: BUG: Fix DATA statements for f2py - #24870: API: Add ``NumpyUnpickler`` for backporting - #24872: MAINT: Xfail test failing on PyPy. - #24879: BLD: fix math func feature checks, fix FreeBSD build, add CI... - #24899: ENH: meson: implement BLAS/LAPACK auto-detection and many CI... - #24902: DOC: add a 1.26.1 release notes section for BLAS/LAPACK build... - #24906: MAINT: Backport ``numpy._core`` stubs. Remove ``NumpyUnpickler`` - #24911: MAINT: Bump pypa/cibuildwheel from 2.16.1 to 2.16.2 - #24912: BUG: loongarch doesn't use REAL(10) Cheers, Charles Harris

1 0