Mailman 3 June 2023 - SciPy-Dev

RFC: decouple doctesting from refguide-check
by Evgeni Burovski 23 Feb '24

23 Feb '24

Hi, Code examples in the SciPy and NumPy documentation are doctested, with a modified doctesting machinery which understands floating point, numpy formatting and some details of our documentation and API. Our modified doctesting machinery is buried in the refguide-check utility, which also does several unrelated things, all of which are tightly coupled to each other and to the libraries themselves. It has been sometimes described as hard to understand or work with or extend due to this tight coupling and a lack of a dedicated test suite. The numpy and scipy versions of the utility are both vendored, and the numpy version has diverged from the scipy version somewhat. Following a discussion in https://github.com/numpy/numpy/issues/21070 I did a small experiment to decouple the doctesting into a separate package, so that it's easier to consolidate the two versions. Plus, a separate repo is generally easier to maintain, configure, and possibly extend or adopt to other projects. The work-in-progress result is here: https://github.com/ev-br/scpdt It currently can run the full doctesting of the scipy API documentation (docstrings of objects) --- see https://github.com/ev-br/scpdt/pull/33 (the log of a test run with warnings turned to errors is in the GH actions: https://github.com/ev-br/scpdt/runs/6743881766?check_suite_focus=true). Note that it shows, among other things, a couple of deprecation warnings our docs have accumulated :-). The API of the tool closely follows that of the standard library doctest module and provides (nearly) drop-in replacements for doctest checking, parsing, finding and running. Various configuration options for our modifications are collected into a single bag object which is internally passed around. This way, it's user-configurable all the way from a plain standard doctest module behavior to what refguide-check does now. There are a couple of wrinkles to iron out; overall it does what refguide-check does already. One missing bit is doctesting rst or other text files, but it's coming soon. The current plan is to: - verify that the standalone version does not miss things checked by the refguide-check - plumb it through the SciPy dev interface and rip out the refguide-check bundled doctesting utilities. - Sync changes that NumPy version of refguide-check accumulated over time - Make sure it correctly tests the NumPy docs, too. - Better document the internals, there is currently only a readme file. If someone's interested to join me working on these, great, the more the merrier :-). I think it could make sense to move the tool's repository to the scipy github org (or maybe even numpy org?). I'm offering to maintain it regardless of the location. Thoughts? Cheers, Evgeni

5 8

Deprecate scipy.misc and Add scipy.datasets submodule
by Anirudh Dagar 07 Jan '24

07 Jan '24

Hi all, I just filed https://github.com/scipy/scipy/pull/15607 after an offline discussion with Ralf earlier this month. The summary of that PR is: This is a good time to get rid of the `scipy.misc` submodule since it houses only a handful of functions. As of today, most of the methods have already been deprecated from the `misc` module. They have generally moved under some other submodule. In this case, it is best to create a `scipy.datasets` submodule that can be home to the dataset functions (ascent, face, ecg). Adding a datasets submodule is not a new idea, and was proposed a couple of years ago by Warren (see https://github.com/scipy/scipy/pull/8707), but unfortunately, it never went ahead. Adding this submodule is not only sensible from an organizational/structural point of view but will also help reduce the wheel size of SciPy package. This is possible if we decouple the dataset files from the SciPy repository making use of Pooch to download and cache datasets kept in separate repos. See the PR for more details. I also filed https://github.com/scipy/scipy/issues/15608 Out of the 5 methods in `scipy.misc`, three dataset functions can be moved to this datasets submodule and will slowly be deprecated from misc. The question about the other two methods i.e (derivative, central_diff_weights) is up for discussion and I'd like to understand what will be the best way to deprecate them. Should we move them to a submodule like integrate (keeping derivative makes sense to me)? On the other hand, considering these two methods are not used extensively so one possibility is to deprecate both of them completely and remove them from SciPy. Please share your opinion/concerns, and comment if you see a potential issue with this. Cheers, Anirudh

4 10

Implementation of Landau distribution into scipy
by Matias Senger 28 Aug '23

28 Aug '23

Hi, I am new here, though have been a SciPy user for several years. I am working on solid state radiation detectors and often need to use the [Landau distribution](https://es.wikipedia.org/wiki/Distribuci%C3%B3n_de_Landau) to model my data. Since I could not find it in Python I implemented it myself (see here https://github.com/SengerM/landaupy). It is based on the Root implementation, still it is a "pure Python" version only using scipy and numpy. Now I would like to add it into SciPy so it is there for other people as well. I would like to do this myself mainly for two reasons, one is to give back a grain of salt to SciPy and the other is to actually contribute to a large open source project, which I've never done before but always wanted to. I am following the steps listed in the [contributor quickstart guide](https://scipy.github.io/devdocs/dev/dev_quickstart.html). I run into [an issue](https://github.com/scipy/scipy/issues/18390) which will hopefully be solved but while asking for help about that I was suggested to propose this feature here before sitting to work on it. Please let me know your thoughts. Kind regards, Matias.

2 2

SciPy Community Meeting on Wed, April 5 at 12pm UTC
by Pamphile Roy 22 Aug '23

22 Aug '23

Hi everyone :) The next SciPy Community Meeting will happen tomorrow, on Wednesday April 5 at 12pm (noon) UTC. Here's the hackmd we are using to take notes: https://hackmd.io/pyhudrC5TgSwdJtpPldHgQ If you wish to join on Zoom, use this link: https://us06web.zoom.us/j/6345425936?pwd=aDVFQzVmbk9SVU5jU0Jwc0s3YWUrdz09 Our community calendar is also available here: https://scientific-python.org/calendars/ Cheers, Pamphile

2 12

Next newcomers' meeting: Friday, July 22 at 8pm UTC
by Melissa Mendonça 17 Aug '23

17 Aug '23

Hello, folks! This friday, *July 22 at 8pm UTC* we are having another newcomers' meeting. All are welcome to join and ask questions about contributing, setting up your development environment, PRs or issues you may want to work on, or just chat. Join us on Zoom <https://us06web.zoom.us/j/6345425936?pwd=aDVFQzVmbk9SVU5jU0Jwc0s3YWUrdz09>, and if you need more information check out some notes from our previous meeting here: https://hackmd.io/@melissawm/scipy-newcomers Hope to see you around! - Melissa

1 14

Fast versions of scipy.io.mmread/write
by alugowski＠gmail.com 13 Jul '23

13 Jul '23

Hi, I'm the author of fast_matrix_market, a package that reads and writes Matrix Market files: https://github.com/alugowski/fast_matrix_market/tree/main/python The Python binding is API compatible with scipy.io.mmio, but about 25x faster. A few other handy improvements include 64-bit indices (only if necessary), direct csc/csr writes with no coo intermediary, longdouble. It passes the SciPy mmio test suite. Would there be any interest in somehow integrating this library into SciPy? The speed difference really does make a big difference for large files. The package is written in C++17 and pybind11. Threading is with a simple thread pool based on c++11 threads, but can be changed. A significant part of the speed increase is parsing with std::from_chars instead of strto*, which alone contributes a massive improvement.

4 13

Array API machinery
by Pamphile Roy 30 Jun '23

30 Jun '23

1 0

Interest in an implementation for a weighted geometric standard deviation and suitable reviewers
by Max Frei 30 Jun '23

30 Jun '23

Hello everyone! I'd like to bring your attention to the following small feature request: https://github.com/scipy/scipy/issues/13424 Matt Haberland referred me to this mailing list, so that I can 1. Check if people are interested in this feature. 2. Find suitable reviewers, since the interpretation of the weighted geometric standard deviation is not trivial. Kind regards.

1 0

ANN: SciPy 1.11.1
by Tyler Reddy 29 Jun '23

29 Jun '23

Hi all, On behalf of the SciPy development team, I'm pleased to announce the release of SciPy 1.11.1, which addresses a few bugs and in particular a licensing issue discovered shortly after the release of 1.11.0, which has now been yanked from PyPI. Sources and binary wheels can be found at: https://pypi.org/project/scipy/ and at: https://github.com/scipy/scipy/releases/tag/v1.11.1 One of a few ways to install this release with pip: pip install scipy==1.11.1 ====================== SciPy 1.11.1 Release Notes ====================== SciPy 1.11.1 is a bug-fix release with no new features compared to 1.11.0. In particular, a licensing issue discovered after the release of 1.11.0 has been addressed. Authors ======= * Name (commits) * h-vetinari (1) * Robert Kern (1) * Ilhan Polat (4) * Tyler Reddy (8) A total of 4 people contributed to this release. People with a "+" by their names contributed a patch for the first time. This list of names is automatically generated, and may not be fully complete. Issues closed for 1.11.1 -------------------------------- * `#18739 <https://github.com/scipy/scipy/issues/18739>`__: BUG: run method of scipy.odr.ODR class fails when delta0 parameter... * `#18751 <https://github.com/scipy/scipy/issues/18751>`__: BUG: segfault in \`scipy.linalg.lu\` on x86_64 windows and macos... * `#18753 <https://github.com/scipy/scipy/issues/18753>`__: BUG: factorial return type inconsistent for 0-dim arrays * `#18759 <https://github.com/scipy/scipy/issues/18759>`__: determinant of a 1x1 matrix returns an array, not a scalar * `#18765 <https://github.com/scipy/scipy/issues/18765>`__: Licensing concern Pull requests for 1.11.1 -------------------------------- * `#18741 <https://github.com/scipy/scipy/pull/18741>`__: BUG: Fix work array construction for various weight shapes. * `#18747 <https://github.com/scipy/scipy/pull/18747>`__: REL, MAINT: prep for 1.11.1 * `#18754 <https://github.com/scipy/scipy/pull/18754>`__: BUG: fix handling for \`factorial(..., exact=False)\` for 0-dim... * `#18762 <https://github.com/scipy/scipy/pull/18762>`__: FIX:linalg.lu:Guard against permute_l out of bound behavior * `#18763 <https://github.com/scipy/scipy/pull/18763>`__: MAINT:linalg.det:Return scalars for singleton inputs * `#18778 <https://github.com/scipy/scipy/pull/18778>`__: MAINT: fix unuran licensing Checksums ========= MD5 ~~~ 0d8ef8d4b3a484a1edb5b1027c635175 Changelog 9c279f2e8c4bfc18c08dee388f0e3937 README.txt aca7e7047ecb15a0a8fbdc6704ba88ab scipy-1.11.1-cp310-cp310-macosx_10_9_x86_64.whl 9619336484eb1f39799a771456882cb0 scipy-1.11.1-cp310-cp310-macosx_12_0_arm64.whl e3943627bc04c4b2cd5bbc8c46928199 scipy-1.11.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl e48d61b72e9af960387b5249a7c3fd37 scipy-1.11.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 508a6c71a1d8c040646fdc91d7b2067f scipy-1.11.1-cp310-cp310-musllinux_1_1_x86_64.whl 8fb76fe2d1b7e45ec16f39a3d50748bb scipy-1.11.1-cp310-cp310-win_amd64.whl e82b0d4dd2f7f71f0a4ed6360c77954e scipy-1.11.1-cp311-cp311-macosx_10_9_x86_64.whl 044ce4958fe3640a9415f0203b1cc0be scipy-1.11.1-cp311-cp311-macosx_12_0_arm64.whl a8d56936cd49c8e903c798133f5e065f scipy-1.11.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 4cc3bd0a1be2457471d017eeed54c7aa scipy-1.11.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 9cc5e6100de5f3d5cef21ccbba6693e1 scipy-1.11.1-cp311-cp311-musllinux_1_1_x86_64.whl b73a6e1a5cf1bc2ad5aa8a2dc99ad194 scipy-1.11.1-cp311-cp311-win_amd64.whl db9b049b05206fb30b10ec247d959d94 scipy-1.11.1-cp39-cp39-macosx_10_9_x86_64.whl fcc109d0a300aaf982854b13208381fe scipy-1.11.1-cp39-cp39-macosx_12_0_arm64.whl 7eff7b1508f1ebbcc9f99aba48531677 scipy-1.11.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 9c90f85da178c4d9dc6d795f1ebd1151 scipy-1.11.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 1b652ed5b032737fcb56ef58f2158f6b scipy-1.11.1-cp39-cp39-musllinux_1_1_x86_64.whl 7e0c3c4d0b37a346b28cfd9f66e22b9f scipy-1.11.1-cp39-cp39-win_amd64.whl SHA256 ~~~~~~ 7835dab6f69b7551fb7b76534ecc8ee559bcdfc4d255a1a28dc7b1bcdda5050c Changelog c6cb0f39b1bb2eb6347f12d21027e11546cbfa7a9c2e1f8d3e823a2ecf918bb6 README.txt aec8c62fbe52914f9cf28d846cf0401dd80ab80788bbab909434eb336ed07c04 scipy-1.11.1-cp310-cp310-macosx_10_9_x86_64.whl 3b9963798df1d8a52db41a6fc0e6fa65b1c60e85d73da27ae8bb754de4792481 scipy-1.11.1-cp310-cp310-macosx_12_0_arm64.whl 3e8eb42db36526b130dfbc417609498a6192381abc1975b91e3eb238e0b41c1a scipy-1.11.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 366a6a937110d80dca4f63b3f5b00cc89d36f678b2d124a01067b154e692bab1 scipy-1.11.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 08d957ca82d3535b3b9ba6c8ff355d78fe975271874e2af267cb5add5bd78625 scipy-1.11.1-cp310-cp310-musllinux_1_1_x86_64.whl e866514bc2d660608447b6ba95c8900d591f2865c07cca0aa4f7ff3c4ca70f30 scipy-1.11.1-cp310-cp310-win_amd64.whl ba94eeef3c9caa4cea7b402a35bb02a5714ee1ee77eb98aca1eed4543beb0f4c scipy-1.11.1-cp311-cp311-macosx_10_9_x86_64.whl 512fdc18c65f76dadaca139348e525646d440220d8d05f6d21965b8d4466bccd scipy-1.11.1-cp311-cp311-macosx_12_0_arm64.whl cce154372f0ebe88556ed06d7b196e9c2e0c13080ecb58d0f35062dc7cc28b47 scipy-1.11.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl b4bb943010203465ac81efa392e4645265077b4d9e99b66cf3ed33ae12254173 scipy-1.11.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 249cfa465c379c9bb2c20123001e151ff5e29b351cbb7f9c91587260602c58d0 scipy-1.11.1-cp311-cp311-musllinux_1_1_x86_64.whl ffb28e3fa31b9c376d0fb1f74c1f13911c8c154a760312fbee87a21eb21efe31 scipy-1.11.1-cp311-cp311-win_amd64.whl 39154437654260a52871dfde852adf1b93b1d1bc5dc0ffa70068f16ec0be2624 scipy-1.11.1-cp39-cp39-macosx_10_9_x86_64.whl b588311875c58d1acd4ef17c983b9f1ab5391755a47c3d70b6bd503a45bfaf71 scipy-1.11.1-cp39-cp39-macosx_12_0_arm64.whl d51565560565a0307ed06fa0ec4c6f21ff094947d4844d6068ed04400c72d0c3 scipy-1.11.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl b41a0f322b4eb51b078cb3441e950ad661ede490c3aca66edef66f4b37ab1877 scipy-1.11.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 396fae3f8c12ad14c5f3eb40499fd06a6fef8393a6baa352a652ecd51e74e029 scipy-1.11.1-cp39-cp39-musllinux_1_1_x86_64.whl be8c962a821957fdde8c4044efdab7a140c13294997a407eaee777acf63cbf0c scipy-1.11.1-cp39-cp39-win_amd64.whl

3 2

RFC: ufunc return type for 0d array input
by Albert Steppi 27 Jun '23

27 Jun '23

Hi team, I recently became aware of gh-18754 <https://github.com/scipy/scipy/pull/18754>, a PR to make the factorial function in special return a 0d array when given 0d input. This brings it in line with the behavior of factorial2, and factorialk, which was settled down in gh-15600 <https://github.com/scipy/scipy/issues/15600>. The longstanding behavior for ufuncs in NumPy and SciPy has been to return a scalar when the output should otherwise be expected to be a 0d-array. All ufuncs generated in scipy.special through the standard infrastructure will have the expected behavior. The factorial functions are however not implemented with the standard infrastructure in order to allow exact calculation with arbitrary-precision integers. I would prefer to have consistency between the factorial functions and the other ufuncs in special. There however isn't a consensus on what the correct behavior should be. I would appreciate any input anyone has to offer, especially from those familiar with the reasons behind the current default convention to convert 0d-arrays to scalar before returning an array from the NumPy C API into Python. Feel free to join the discussion at gh-18768 <https://github.com/scipy/scipy/issues/18768>. Thanks everyone! Albert

1 0