Mailman 3 April 2023 - SciPy-Dev

RFC: decouple doctesting from refguide-check
by Evgeni Burovski 23 Feb '24

23 Feb '24

Hi, Code examples in the SciPy and NumPy documentation are doctested, with a modified doctesting machinery which understands floating point, numpy formatting and some details of our documentation and API. Our modified doctesting machinery is buried in the refguide-check utility, which also does several unrelated things, all of which are tightly coupled to each other and to the libraries themselves. It has been sometimes described as hard to understand or work with or extend due to this tight coupling and a lack of a dedicated test suite. The numpy and scipy versions of the utility are both vendored, and the numpy version has diverged from the scipy version somewhat. Following a discussion in https://github.com/numpy/numpy/issues/21070 I did a small experiment to decouple the doctesting into a separate package, so that it's easier to consolidate the two versions. Plus, a separate repo is generally easier to maintain, configure, and possibly extend or adopt to other projects. The work-in-progress result is here: https://github.com/ev-br/scpdt It currently can run the full doctesting of the scipy API documentation (docstrings of objects) --- see https://github.com/ev-br/scpdt/pull/33 (the log of a test run with warnings turned to errors is in the GH actions: https://github.com/ev-br/scpdt/runs/6743881766?check_suite_focus=true). Note that it shows, among other things, a couple of deprecation warnings our docs have accumulated :-). The API of the tool closely follows that of the standard library doctest module and provides (nearly) drop-in replacements for doctest checking, parsing, finding and running. Various configuration options for our modifications are collected into a single bag object which is internally passed around. This way, it's user-configurable all the way from a plain standard doctest module behavior to what refguide-check does now. There are a couple of wrinkles to iron out; overall it does what refguide-check does already. One missing bit is doctesting rst or other text files, but it's coming soon. The current plan is to: - verify that the standalone version does not miss things checked by the refguide-check - plumb it through the SciPy dev interface and rip out the refguide-check bundled doctesting utilities. - Sync changes that NumPy version of refguide-check accumulated over time - Make sure it correctly tests the NumPy docs, too. - Better document the internals, there is currently only a readme file. If someone's interested to join me working on these, great, the more the merrier :-). I think it could make sense to move the tool's repository to the scipy github org (or maybe even numpy org?). I'm offering to maintain it regardless of the location. Thoughts? Cheers, Evgeni

5 8

Deprecate scipy.misc and Add scipy.datasets submodule
by Anirudh Dagar 07 Jan '24

07 Jan '24

Hi all, I just filed https://github.com/scipy/scipy/pull/15607 after an offline discussion with Ralf earlier this month. The summary of that PR is: This is a good time to get rid of the `scipy.misc` submodule since it houses only a handful of functions. As of today, most of the methods have already been deprecated from the `misc` module. They have generally moved under some other submodule. In this case, it is best to create a `scipy.datasets` submodule that can be home to the dataset functions (ascent, face, ecg). Adding a datasets submodule is not a new idea, and was proposed a couple of years ago by Warren (see https://github.com/scipy/scipy/pull/8707), but unfortunately, it never went ahead. Adding this submodule is not only sensible from an organizational/structural point of view but will also help reduce the wheel size of SciPy package. This is possible if we decouple the dataset files from the SciPy repository making use of Pooch to download and cache datasets kept in separate repos. See the PR for more details. I also filed https://github.com/scipy/scipy/issues/15608 Out of the 5 methods in `scipy.misc`, three dataset functions can be moved to this datasets submodule and will slowly be deprecated from misc. The question about the other two methods i.e (derivative, central_diff_weights) is up for discussion and I'd like to understand what will be the best way to deprecate them. Should we move them to a submodule like integrate (keeping derivative makes sense to me)? On the other hand, considering these two methods are not used extensively so one possibility is to deprecate both of them completely and remove them from SciPy. Please share your opinion/concerns, and comment if you see a potential issue with this. Cheers, Anirudh

4 10

Implementation of Landau distribution into scipy
by Matias Senger 28 Aug '23

28 Aug '23

Hi, I am new here, though have been a SciPy user for several years. I am working on solid state radiation detectors and often need to use the [Landau distribution](https://es.wikipedia.org/wiki/Distribuci%C3%B3n_de_Landau) to model my data. Since I could not find it in Python I implemented it myself (see here https://github.com/SengerM/landaupy). It is based on the Root implementation, still it is a "pure Python" version only using scipy and numpy. Now I would like to add it into SciPy so it is there for other people as well. I would like to do this myself mainly for two reasons, one is to give back a grain of salt to SciPy and the other is to actually contribute to a large open source project, which I've never done before but always wanted to. I am following the steps listed in the [contributor quickstart guide](https://scipy.github.io/devdocs/dev/dev_quickstart.html). I run into [an issue](https://github.com/scipy/scipy/issues/18390) which will hopefully be solved but while asking for help about that I was suggested to propose this feature here before sitting to work on it. Please let me know your thoughts. Kind regards, Matias.

2 2

SciPy Community Meeting on Wed, April 5 at 12pm UTC
by Pamphile Roy 22 Aug '23

22 Aug '23

Hi everyone :) The next SciPy Community Meeting will happen tomorrow, on Wednesday April 5 at 12pm (noon) UTC. Here's the hackmd we are using to take notes: https://hackmd.io/pyhudrC5TgSwdJtpPldHgQ If you wish to join on Zoom, use this link: https://us06web.zoom.us/j/6345425936?pwd=aDVFQzVmbk9SVU5jU0Jwc0s3YWUrdz09 Our community calendar is also available here: https://scientific-python.org/calendars/ Cheers, Pamphile

2 12

Next newcomers' meeting: Friday, July 22 at 8pm UTC
by Melissa Mendonça 17 Aug '23

17 Aug '23

Hello, folks! This friday, *July 22 at 8pm UTC* we are having another newcomers' meeting. All are welcome to join and ask questions about contributing, setting up your development environment, PRs or issues you may want to work on, or just chat. Join us on Zoom <https://us06web.zoom.us/j/6345425936?pwd=aDVFQzVmbk9SVU5jU0Jwc0s3YWUrdz09>, and if you need more information check out some notes from our previous meeting here: https://hackmd.io/@melissawm/scipy-newcomers Hope to see you around! - Melissa

1 14

Fast versions of scipy.io.mmread/write
by alugowski＠gmail.com 13 Jul '23

13 Jul '23

Hi, I'm the author of fast_matrix_market, a package that reads and writes Matrix Market files: https://github.com/alugowski/fast_matrix_market/tree/main/python The Python binding is API compatible with scipy.io.mmio, but about 25x faster. A few other handy improvements include 64-bit indices (only if necessary), direct csc/csr writes with no coo intermediary, longdouble. It passes the SciPy mmio test suite. Would there be any interest in somehow integrating this library into SciPy? The speed difference really does make a big difference for large files. The package is written in C++17 and pybind11. Threading is with a simple thread pool based on c++11 threads, but can be changed. A significant part of the speed increase is parsing with std::from_chars instead of strto*, which alone contributes a massive improvement.

4 13

New keyword name for linalg.lu
by Ilhan Polat 22 May '23

22 May '23

Dear all, In [1], I have proposed a new keyword for scipy.linalg.lu function, with prospective name "p_indices" as a bool switch. This will have the effect of having the permutation matrix P in A = P L U decomposition returned as a vector of indices such as [3, 1, 2, 0] meaning first and fourth row is swapped. With this you can still compute A through L[P, :] @ U without allocating a very costly 2D matrix of 1s and 0s. In tall arrays, say 400x10, the resulting 2D array P is 400x400 and in the vector form this reduces to 400x1. Hence, the switch to control this behavior is done by a new keyword and I don't know if "p_indices" is a nice one (doesn't feel like it). I considered, return_indices, perm_as_1d, p_as_1d, p_as_vec and bunch more but either too many underscores or too verbose. Thus, I'd like to ask for either validation or new ideas. We should maybe ping the NumPy audience for this too as this has no counterpart in NumPy. Thanks, ilhan [1] : https://github.com/scipy/scipy/pull/18358

7 15

Sub-category on Scientific Python's Discuss server
by Pamphile Roy 28 Apr '23

28 Apr '23

Hi everyone! Following the retirement of the SciPy-User mailing list, Stéfan had offered a space on Scientific Python's discuss server: https://discuss.scientific-python.org <https://discuss.scientific-python.org/> Since there was no objection, I worked with him to create a sub-category here: https://discuss.scientific-python.org/c/user/scipy/30 Note that you can also post on this server by sending an email to: scipy(a)discuss.scientific-python.org <mailto:scipy@discuss.scientific-python.org> It will only work if you are registered to the server. And you can subscribe to categories. i.e. like a mailing list. I hope you find this useful and pleasant to use. Let us know if you have any questions or suggestions. Cheers, Pamphile

1 0

Retiring old Fortran code for linalg.interpolative and possibly sparse.linalg.iterative
by Ilhan Polat 27 Apr '23

27 Apr '23

Dear all, We have been discussing the fate of the mentioned codebase, in the RFC [1] We are coming to a point where it looks like a semi-consensus that we need to do something about them. Less with the "sparse.linalg.iterative" which only has open issues burden but mainly "linalg.interpolative" which is causing all kinds of compilation issues. Just to mention, I think we don't have any appetite left for discussing the state of Fortran (multiple times already and also see [2] ) moreover Ondřej Čertík already broke some really good news about LFortran project. So let's try to get to a decision about scipy code first then we can digress further. The rather interesting option; can we deprecate "linalg.interpolative" instead of modernize? Input for that would be great. Thanks, ilhan [1] : https://github.com/scipy/scipy/issues/18367 [2] : https://discuss.scientific-python.org/t/releasing-or-not-32-bit-windows-whe…

1 0

Proposal for New Scipy Function: fwind1() for 2D FIR Filter Design Using 1D Window Method
by Sagi Ezri 27 Apr '23

27 Apr '23

Dear Scipy Development Community, I am writing to propose a new function for inclusion in the Scipy library, called `fwind1()`, which implements 2D FIR filter design using the window method. The `fwind1()` function computes the coefficients of a 2D finite impulse response filter that is separable with linear phase. The filter is designed as a product of two 1D filters with dimensions defined by the `hsize` parameter. The `window` parameter specifies the desired window to use for each 1D filter, and the `fc` parameter specifies the cutoff frequency of the filter. The `fwind1()` function is inspired by the `fwind1()` function in Matlab's Image Processing Toolbox. However, it is implemented from scratch and provides a Pythonic interface consistent with other Scipy signal processing functions. Here is an example usage of the `fwind1()` function: ``` import numpy as np from scipy.signal import get_window from scipy.signal import fwind1 # Generate a 5x5 low-pass filter with cutoff frequency 0.1. filter_2d = fwind1((5, 5), (get_window(('kaiser', 5.0)), 'boxcar'), 0.1) print(filter_2d) ``` Output: ``` array([[0.003, 0.023, 0.052, 0.023, 0.003], [0.023, 0.173, 0.391, 0.173, 0.023], [0.052, 0.391, 0.882, 0.391, 0.052], [0.023, 0.173, 0.391, 0.173, 0.023], [0.003, 0.023, 0.052, 0.023, 0.003]]) ``` As this is a new function, I would appreciate feedback from the Scipy development community on the proposed implementation and whether there is a consensus on adding it to Scipy. I believe this function could be a useful addition to the Scipy library, especially for researchers and practitioners in the field of image processing. Thank you for your consideration and I look forward to your feedback. Sincerely, Sagi Ezri

1 0