Mailman 3 November 2023 - SciPy-Dev

SciPy Community Meeting: Wednesday November 29th at 8pm UTC
by Melissa Mendonça 05 Mar '24

05 Mar '24

Hi all, The next SciPy Community Meeting will happen on* Wednesday November 29th at 8pm UTC*. Here's the hackmd we are using to take notes: https://hackmd.io/pyhudrC5TgSwdJtpPldHgQ If you wish to join on Zoom, use this link: https://us06web.zoom.us/j/6345425936?pwd=aDVFQzVmbk9SVU5jU0Jwc0s3YWUrdz09 Our community calendar is also available here: https://scientific-python.org/calendars/ Cheers, Melissa

1 7

Next SciPy newcomers' meeting - TODAY October 13th at 8pm UTC
by Melissa Mendonça 29 Feb '24

29 Feb '24

Hello, folks! Our next newcomers meeting happens TODAY, October 13th, at 8pm UTC. Feel free to join to ask questions about contributing, our community, follow ups on existing issues or pull requests, or to just chat. If you are a maintainer or a regular contributor, you are also welcome to join as well to connect with other members of our community :) Here's the hackmd we use to take notes, which includes the meeting details: https://hackmd.io/@melissawm/scipy-newcomers And here's the link if you wish to join us on Zoom: https://us06web.zoom.us/j/89085469494?pwd=dVFwK3dVczRyRUowZWZkckFtTmloUT09 Hope to see you around! Cheers, Melissa

1 2

RFC: decouple doctesting from refguide-check
by Evgeni Burovski 23 Feb '24

23 Feb '24

Hi, Code examples in the SciPy and NumPy documentation are doctested, with a modified doctesting machinery which understands floating point, numpy formatting and some details of our documentation and API. Our modified doctesting machinery is buried in the refguide-check utility, which also does several unrelated things, all of which are tightly coupled to each other and to the libraries themselves. It has been sometimes described as hard to understand or work with or extend due to this tight coupling and a lack of a dedicated test suite. The numpy and scipy versions of the utility are both vendored, and the numpy version has diverged from the scipy version somewhat. Following a discussion in https://github.com/numpy/numpy/issues/21070 I did a small experiment to decouple the doctesting into a separate package, so that it's easier to consolidate the two versions. Plus, a separate repo is generally easier to maintain, configure, and possibly extend or adopt to other projects. The work-in-progress result is here: https://github.com/ev-br/scpdt It currently can run the full doctesting of the scipy API documentation (docstrings of objects) --- see https://github.com/ev-br/scpdt/pull/33 (the log of a test run with warnings turned to errors is in the GH actions: https://github.com/ev-br/scpdt/runs/6743881766?check_suite_focus=true). Note that it shows, among other things, a couple of deprecation warnings our docs have accumulated :-). The API of the tool closely follows that of the standard library doctest module and provides (nearly) drop-in replacements for doctest checking, parsing, finding and running. Various configuration options for our modifications are collected into a single bag object which is internally passed around. This way, it's user-configurable all the way from a plain standard doctest module behavior to what refguide-check does now. There are a couple of wrinkles to iron out; overall it does what refguide-check does already. One missing bit is doctesting rst or other text files, but it's coming soon. The current plan is to: - verify that the standalone version does not miss things checked by the refguide-check - plumb it through the SciPy dev interface and rip out the refguide-check bundled doctesting utilities. - Sync changes that NumPy version of refguide-check accumulated over time - Make sure it correctly tests the NumPy docs, too. - Better document the internals, there is currently only a readme file. If someone's interested to join me working on these, great, the more the merrier :-). I think it could make sense to move the tool's repository to the scipy github org (or maybe even numpy org?). I'm offering to maintain it regardless of the location. Thoughts? Cheers, Evgeni

5 8

Deprecate scipy.misc and Add scipy.datasets submodule
by Anirudh Dagar 07 Jan '24

07 Jan '24

Hi all, I just filed https://github.com/scipy/scipy/pull/15607 after an offline discussion with Ralf earlier this month. The summary of that PR is: This is a good time to get rid of the `scipy.misc` submodule since it houses only a handful of functions. As of today, most of the methods have already been deprecated from the `misc` module. They have generally moved under some other submodule. In this case, it is best to create a `scipy.datasets` submodule that can be home to the dataset functions (ascent, face, ecg). Adding a datasets submodule is not a new idea, and was proposed a couple of years ago by Warren (see https://github.com/scipy/scipy/pull/8707), but unfortunately, it never went ahead. Adding this submodule is not only sensible from an organizational/structural point of view but will also help reduce the wheel size of SciPy package. This is possible if we decouple the dataset files from the SciPy repository making use of Pooch to download and cache datasets kept in separate repos. See the PR for more details. I also filed https://github.com/scipy/scipy/issues/15608 Out of the 5 methods in `scipy.misc`, three dataset functions can be moved to this datasets submodule and will slowly be deprecated from misc. The question about the other two methods i.e (derivative, central_diff_weights) is up for discussion and I'd like to understand what will be the best way to deprecate them. Should we move them to a submodule like integrate (keeping derivative makes sense to me)? On the other hand, considering these two methods are not used extensively so one possibility is to deprecate both of them completely and remove them from SciPy. Please share your opinion/concerns, and comment if you see a potential issue with this. Cheers, Anirudh

4 10

Next newcomers' meeting - Friday, November 10th at 12pm UTC
by Melissa Mendonça 03 Jan '24

03 Jan '24

Hello all, Our next newcomers meeting happens this friday, November 10th, at 12pm (noon) UTC. Come by if you have questions about contributing, or if you'd like to learn more about the SciPy community. Here's the hackmd we use to take notes, which includes the meeting details: https://hackmd.io/@melissawm/scipy-newcomers And here's the link if you wish to join on Zoom: https://us06web.zoom.us/j/89085469494?pwd=dVFwK3dVczRyRUowZWZkckFtTmloUT09 Hope to see you around! Cheers, Melissa

2 2

Deprecate `scipy.integrate.romberg`/`scipy.integrate.quadrature`?
by Matt Haberland 02 Dec '23

02 Dec '23

Hi Team, `scipy.integrate.romberg` and `scipy.integrate.quadrature` do not reliably hit their target accuracies, and when they do, they typically require more function evaluations and take more wall clock time than `scipy.integrate.quad` (all with default settings). This is documented in https://github.com/scipy/scipy/issues/18574#issuecomment-1703320598, which compares the accuracies, function call counts, and wall clock times on a benchmark set of ~800 integrals with known analytical solutions. gh-19510 (https://github.com/scipy/scipy/pull/19510) proposes to deprecate these functions, suggesting `scipy.integrate.quad` as a drop-in replacement. Please let us know your thoughts either here or in gh-19510! Thanks, Matt

4 7

Issue #19567: New optimization method
by Bruno Peixoto 21 Nov '23

21 Nov '23

This post is related to issue #19567 (https://github.com/scipy/scipy/issues/19567). The here-described optimization method is called the "Barycenter Method". His name originates from calculating the barycenter respective to some Oracle map based on evaluation points. Each point evaluates to a scalar quantity, and the barycenter weight is the exponential of the scaled negative of this value. The higher the value, the lower the weight. It has a gradient behaviour, but it derivative-free. I would be glad to have this method incorporated to scipy, since I worked with on my Master thesis. [1] https://arxiv.org/abs/2102.10467 Currently, I have this python version: https://github.com/asmove/pybary . I would be very grateful for any feedback in this regard. Thanks for reading!

1 0

statistics for complex valued random variables
by josef.pktd＠gmail.com 20 Nov '23

20 Nov '23

This is not scipy specific. How much interest and usefulness is there in supporting statistics for complex random variables in Python? I spent the last 2 weeks trying to figure out regression and statistics for complex random variables. References are mainly in the signal processing literature. But I still have not figured out if there is actually a demand for basic statistics. For example I found very few cases of hypothesis testing with complex random variables in the literature. https://github.com/statsmodels/statsmodels/issues/3528 OLS for complex https://github.com/statsmodels/statsmodels/issues/9064 statistics for complex Example: For means squared error computation, there was a brief discussion whether there should be an `abs` (i.e. conjugate) in the definition. Standard variance uses x.H x as inner product, pseudo-variance uses x.T x. numpy and scipy support the standard variance and covariance but not the pseudo version. If the pseudo-variance is zero, then the random variable is (second order) circular or proper. In that case just looking at conj/hermition products is enough. If the pseudo-variance is not zero, then the statistics for the random variables needs to take the "pseudo" part into account. AFAIK, there is currently no pseudo (co)variance in either numpy or scipy (although they are easy to compute). (Another area that I have not looked at in details is `optimize` for complex valued functions and parameters, even if the objective function is real, like hermitian least squares) The definitions for distributions and statistics for complex random variables that I have seen are all derived from the combined real representation [real, imag]. example circular and non-circular complex normal distribution. The motivation for complex valued statistics is then mainly a convenience when it is easier to work with complex numbers instead of their homomorphic real representation. I looked at this mainly out of curiosity and for fun. I still don't know how useful some "generic" statistics is. There are many applications in the signal processing literature, but most of those seem to be rather specific to signal processing (and not so much in the range of what statsmodels would cover.) Actual implementation could be spread across numpy, scipy.stats and statsmodels similar to other areas of statistics. Josef

1 0

ANN: SciPy 1.11.4
by Tyler Reddy 19 Nov '23

19 Nov '23

Hi all, On behalf of the SciPy development team, I'm pleased to announce the release of SciPy 1.11.4, which is a bug fix release. Sources and binary wheels can be found at: https://pypi.org/project/scipy/ and at: https://github.com/scipy/scipy/releases/tag/v1.11.4 One of a few ways to install this release with pip: pip install scipy==1.11.4 ====================== SciPy 1.11.4 Release Notes ====================== SciPy 1.11.4 is a bug-fix release with no new features compared to 1.11.3. Authors ====== * Name (commits) * Jake Bowhay (2) * Ralf Gommers (4) * Julien Jerphanion (2) * Nikolay Mayorov (2) * Melissa Weber Mendonça (1) * Tirth Patel (1) * Tyler Reddy (22) * Dan Schult (3) * Nicolas Vetsch (1) + A total of 9 people contributed to this release. People with a "+" by their names contributed a patch for the first time. This list of names is automatically generated, and may not be fully complete. Issues closed for 1.11.4 -------------------------------- * `#19189 <https://github.com/scipy/scipy/issues/19189>`__: Contradiction in \`pyproject.toml\` requirements? * `#19228 <https://github.com/scipy/scipy/issues/19228>`__: Doc build fails with Python 3.11 * `#19245 <https://github.com/scipy/scipy/issues/19245>`__: BUG: upcasting of indices dtype from DIA to COO/CSR/BSR arrays * `#19351 <https://github.com/scipy/scipy/issues/19351>`__: BUG: Regression in 1.11.3 can still fail for \`optimize.least_squares\`... * `#19357 <https://github.com/scipy/scipy/issues/19357>`__: BUG: build failure with Xcode 15 linker * `#19359 <https://github.com/scipy/scipy/issues/19359>`__: BUG: DiscreteAliasUrn construction fails with UNURANError for... * `#19387 <https://github.com/scipy/scipy/issues/19387>`__: BUG: problem importing libgfortran.5.dylib on macOS Sonoma * `#19403 <https://github.com/scipy/scipy/issues/19403>`__: BUG: scipy.sparse.lil_matrix division by complex number leads... * `#19437 <https://github.com/scipy/scipy/issues/19437>`__: BUG: can't install scipy on mac m1 with poetry due to incompatible... * `#19500 <https://github.com/scipy/scipy/issues/19500>`__: DOC: doc build failing * `#19513 <https://github.com/scipy/scipy/issues/19513>`__: BUG: Python version constraints in releases causes issues for... Pull requests for 1.11.4 -------------------------------- * `#19230 <https://github.com/scipy/scipy/pull/19230>`__: DOC, MAINT: workaround for py311 docs * `#19307 <https://github.com/scipy/scipy/pull/19307>`__: set idx_dtype in sparse dia_array.tocoo * `#19316 <https://github.com/scipy/scipy/pull/19316>`__: MAINT: Prep 1.11.4 * `#19320 <https://github.com/scipy/scipy/pull/19320>`__: BLD: fix up version parsing issue in cythonize.py for setup.py... * `#19329 <https://github.com/scipy/scipy/pull/19329>`__: DOC: stats.chisquare: result object contains attribute 'statistic' * `#19335 <https://github.com/scipy/scipy/pull/19335>`__: BUG: fix pow method for sparrays with power zero * `#19364 <https://github.com/scipy/scipy/pull/19364>`__: MAINT, BUG: stats: update the UNU.RAN submodule with DAU fix * `#19379 <https://github.com/scipy/scipy/pull/19379>`__: BUG: Restore the original behavior of 'trf' from least_squares... * `#19400 <https://github.com/scipy/scipy/pull/19400>`__: BLD: use classic linker on macOS 14 (Sonoma), the new linker... * `#19408 <https://github.com/scipy/scipy/pull/19408>`__: BUG: Fix typecasting problem in scipy.sparse.lil_matrix truediv * `#19504 <https://github.com/scipy/scipy/pull/19504>`__: DOC, MAINT: Bump CircleCI Python version to 3.11 * `#19517 <https://github.com/scipy/scipy/pull/19517>`__: MAINT, REL: unpin Python 1.11.x branch * `#19550 <https://github.com/scipy/scipy/pull/19550>`__: MAINT, BLD: poetry loongarch shims Checksums ========= MD5 ~~~ 3324298ede271261e0b74df81ca2aa3a Changelog e8b456252ac318dd9f853bf8d8c45029 README.txt b99a5cf10c6a76d2d87ce59d4f87b811 scipy-1.11.4-cp310-cp310-macosx_10_9_x86_64.whl 67d95164f2366ff0efdff3dd25313e8e scipy-1.11.4-cp310-cp310-macosx_12_0_arm64.whl 198cdbc80b37bc4eb262b4478f8bab52 scipy-1.11.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl e61c5838aa069eddaa1e71d8d9d85509 scipy-1.11.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 517f82f0b581aa34630158591377b127 scipy-1.11.4-cp310-cp310-musllinux_1_1_x86_64.whl 9b9fa69812be52fc1da643d4dcab2abf scipy-1.11.4-cp310-cp310-win_amd64.whl 955a224e94e29e03d0c3cfda7147d756 scipy-1.11.4-cp311-cp311-macosx_10_9_x86_64.whl c6d0bfc7b2e57822de9249f468e234e6 scipy-1.11.4-cp311-cp311-macosx_12_0_arm64.whl 617c587f9442fa26fb36876efefdb311 scipy-1.11.4-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 00fe7507f91e11730ed6f48d5881fd74 scipy-1.11.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 340b520348315d183455e7b79fc0c1d3 scipy-1.11.4-cp311-cp311-musllinux_1_1_x86_64.whl fc3aade39fda56dbcd07106bd4b26c84 scipy-1.11.4-cp311-cp311-win_amd64.whl 6b2eeebed81a8aba66d703d785b562e1 scipy-1.11.4-cp312-cp312-macosx_10_9_x86_64.whl f61c07733fcd8747fa3212295bf639e7 scipy-1.11.4-cp312-cp312-macosx_12_0_arm64.whl b8526ce9d78ee7629fa1c00733233a01 scipy-1.11.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 13794337c432999fa37f8cf0280b08f8 scipy-1.11.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 30e84671d615659f0041a47f6aee5105 scipy-1.11.4-cp312-cp312-musllinux_1_1_x86_64.whl eae94466eb0c36b61db723b5cd835fec scipy-1.11.4-cp312-cp312-win_amd64.whl 616e80dff4d6590aba7096ad1dcccb31 scipy-1.11.4-cp39-cp39-macosx_10_9_x86_64.whl 750ff6a0490796885a159a0aa338072b scipy-1.11.4-cp39-cp39-macosx_12_0_arm64.whl 96c002ce95d2cc8e16c81bfe9746198b scipy-1.11.4-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 07695f6f859e93e736008d592cf2080e scipy-1.11.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 700b898a5a373cefdc76954282c66bab scipy-1.11.4-cp39-cp39-musllinux_1_1_x86_64.whl af9246a44e37b2ecfb7370eeb55bd84c scipy-1.11.4-cp39-cp39-win_amd64.whl SHA256 ~~~~~~ cd3df3413447fa60638bd98eb7b3066d93855197ce67210b52f8cf4093b6242b Changelog 753ed3b3ee750a8d52535c860aac29f8bcfb0183c6eafc925f7b41c68921a68b README.txt bc9a714581f561af0848e6b69947fda0614915f072dfd14142ed1bfe1b806710 scipy-1.11.4-cp310-cp310-macosx_10_9_x86_64.whl cf00bd2b1b0211888d4dc75656c0412213a8b25e80d73898083f402b50f47e41 scipy-1.11.4-cp310-cp310-macosx_12_0_arm64.whl b9999c008ccf00e8fbcce1236f85ade5c569d13144f77a1946bef8863e8f6eb4 scipy-1.11.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 933baf588daa8dc9a92c20a0be32f56d43faf3d1a60ab11b3f08c356430f6e56 scipy-1.11.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 8fce70f39076a5aa62e92e69a7f62349f9574d8405c0a5de6ed3ef72de07f446 scipy-1.11.4-cp310-cp310-musllinux_1_1_x86_64.whl 6550466fbeec7453d7465e74d4f4b19f905642c89a7525571ee91dd7adabb5a3 scipy-1.11.4-cp310-cp310-win_amd64.whl f313b39a7e94f296025e3cffc2c567618174c0b1dde173960cf23808f9fae4be scipy-1.11.4-cp311-cp311-macosx_10_9_x86_64.whl 1b7c3dca977f30a739e0409fb001056484661cb2541a01aba0bb0029f7b68db8 scipy-1.11.4-cp311-cp311-macosx_12_0_arm64.whl 00150c5eae7b610c32589dda259eacc7c4f1665aedf25d921907f4d08a951b1c scipy-1.11.4-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 530f9ad26440e85766509dbf78edcfe13ffd0ab7fec2560ee5c36ff74d6269ff scipy-1.11.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 5e347b14fe01003d3b78e196e84bd3f48ffe4c8a7b8a1afbcb8f5505cb710993 scipy-1.11.4-cp311-cp311-musllinux_1_1_x86_64.whl acf8ed278cc03f5aff035e69cb511741e0418681d25fbbb86ca65429c4f4d9cd scipy-1.11.4-cp311-cp311-win_amd64.whl 028eccd22e654b3ea01ee63705681ee79933652b2d8f873e7949898dda6d11b6 scipy-1.11.4-cp312-cp312-macosx_10_9_x86_64.whl 2c6ff6ef9cc27f9b3db93a6f8b38f97387e6e0591600369a297a50a8e96e835d scipy-1.11.4-cp312-cp312-macosx_12_0_arm64.whl b030c6674b9230d37c5c60ab456e2cf12f6784596d15ce8da9365e70896effc4 scipy-1.11.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl ad669df80528aeca5f557712102538f4f37e503f0c5b9541655016dd0932ca79 scipy-1.11.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl ce7fff2e23ab2cc81ff452a9444c215c28e6305f396b2ba88343a567feec9660 scipy-1.11.4-cp312-cp312-musllinux_1_1_x86_64.whl 36750b7733d960d7994888f0d148d31ea3017ac15eef664194b4ef68d36a4a97 scipy-1.11.4-cp312-cp312-win_amd64.whl 6e619aba2df228a9b34718efb023966da781e89dd3d21637b27f2e54db0410d7 scipy-1.11.4-cp39-cp39-macosx_10_9_x86_64.whl f3cd9e7b3c2c1ec26364856f9fbe78695fe631150f94cd1c22228456404cf1ec scipy-1.11.4-cp39-cp39-macosx_12_0_arm64.whl d10e45a6c50211fe256da61a11c34927c68f277e03138777bdebedd933712fea scipy-1.11.4-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 91af76a68eeae0064887a48e25c4e616fa519fa0d38602eda7e0f97d65d57937 scipy-1.11.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 6df1468153a31cf55ed5ed39647279beb9cfb5d3f84369453b49e4b8502394fd scipy-1.11.4-cp39-cp39-musllinux_1_1_x86_64.whl ee410e6de8f88fd5cf6eadd73c135020bfbbbdfcd0f6162c36a7638a1ea8cc65 scipy-1.11.4-cp39-cp39-win_amd64.whl

1 0

Argument to suppress Gimbal Lock Warning in rotations.
by Matthias Bussonnier 16 Nov '23

16 Nov '23

Hello SciPy Dev, I recently sent a PR[1] to add a parameter to suppress the Gimbal Lock Warning message when using SciPy's rotation, and I'm coming to the mailing list for advice. The problem I'm encountering is that the typical way of silencing a specific warning would be to warnings.catch_warnings – unfortunately this hits a bug in CPython[2] (that has been open for 5 years and does not seem close to be fixed). And while I typically do not care about warnings being printed in CLI, this creates issues in GUI applications where the messages can get spammed when dragging a camera around – in my case in napari. I don't see any other way to prevent this than to add a keyword only to rotation's methods to not emit the warning. Your thoughts are appreciated. Thanks, -- Matthias 1: https://github.com/scipy/scipy/pull/19512 2: https://github.com/python/cpython/issues/73858

1 0