Mailman 3 August 2023 - SciPy-Dev

Next newcomers' meeting: Friday, July 22 at 8pm UTC
by Melissa Mendonça Aug. 17, 2023

Aug. 17, 2023

Hello, folks! This friday, *July 22 at 8pm UTC* we are having another newcomers' meeting. All are welcome to join and ask questions about contributing, setting up your development environment, PRs or issues you may want to work on, or just chat. Join us on Zoom <https://us06web.zoom.us/j/6345425936?pwd=aDVFQzVmbk9SVU5jU0Jwc0s3YWUrdz09>, and if you need more information check out some notes from our previous meeting here: https://hackmd.io/@melissawm/scipy-newcomers Hope to see you around! - Melissa

1 14

SciPy Community calls - time and dates [POLL]
by Melissa Mendonça Aug. 15, 2023

Aug. 15, 2023

Hello, folks! In order to make our meetings more convenient for the community and understand who would like to attend but maybe isn't able to, I am proposing a meeting poll to answer two questions: - What day of the week/time would be best for these meetings? - Should we move on to 30 min meetings instead of 1h every two weeks, if that's better for folks? Here is the link to the poll: http://whenisgood.net/kfd94pw <https://whenisgood.net/kfd94pw> Feel free to respond especially if you haven't been attending the meetings (but would like to). Ideally we can also keep the alternating times to include 2 blocks of timezones. (The specific dates listed in the poll are just examples, please only consider the days of the week in your responses). Let me know if you have other thoughts. Cheers! - Melissa

1 1

DifferentialEvolution: custom mutation and recombination functions?
by Karen Róbertsdóttir Aug. 15, 2023

Aug. 15, 2023

Hi - newbie here, my apologies for any ignorance here! It is my understanding that scipy has no ability for the user to specify custom mutation or recombination functions, or any sort of manual alteration of new population members at all - correct? If so: are there any plans to introduce such a thing? If not: would there be interest in it? It's among the most trivial of all possible feature additions - just adding in ", mutation_fn=<current_scipy_mutation_fn>, recombination_fn=<current_scipy_recombination_fn>" to scipy.optimize.differential_evolution, updating the docs, and writing test functions that just return a string of 1s, then printing the input provided to the minimize function, which should eventually turn into all 1s. Use case: the user knows far more about what their data "means" than scipy will ever know. Including when a change in one place implies changes in another, how various factors ahould impact new values, etc. A particularly useful case is where the data represents a string of "genes", where duplication is not inherently deleterious. One of natural evolution's most powerful features is gene duplication followed by genetic drift of the copies, with their functionality diverging: the original function is retained while a new function is created. But scipy's stock mutation and recombination functions can't duplicate, shift around, or match up genes. A user however could readily provide such a function. I'm currently dealing with such a case (geometry optimization - each cluster of floats represents how a new element attaches to a mesh) If there are no plans to implement it, I might (or might not, depending) be able to find the time to do so. Though I know 98% of the time required would not be coding / testing, but rather figuring out how to setup and develop for the scipy test environment and figuring out how to contribute the changes - so it would be easier if an existing dev could just be bothered to add ", mutation_fn=<current_scipy_mutation_fn>, recombination_fn=<current_scipy_recombination_fn>" :) Anyway, sorry to bother you all! - kv, Karen

2 1

NumPy 1.26.0b1 released
by Charles R Harris Aug. 14, 2023

Aug. 14, 2023

Hi All, On behalf of the NumPy team, I'm pleased to announce the release of NumPy 1.26.0b1. The NumPy 1.26.0 release is a continuation of the 1.25.x release cycle with the addition of Python 3.12.0 support. Python 3.12 dropped distutils, consequently supporting it required finding a replacement for the setup.py/distutils based build system NumPy was using. We have chosen to use the Meson build system instead, and this is the first NumPy release supporting it. This is also the first release that supports Cython 3.0 in addition to retaining 0.29.X compatibility. Supporting those two upgrades was a large project, over 100 files have been touched in this release. The changelog doesn't capture the full extent of the work, special thanks to Ralf Gommers, Sayed Adel, Stéfan van der Walt, and Matti Picus who did much of the work in the main development branch. The highlights of this release are: - Python 3.12.0 support. - Cython 3.0.0 compatibility. - Use of the Meson build system - Updated SIMD support The Python versions supported by this release are 3.9-3.12. Note that no 32 bit wheels are provided. Wheels can be downloaded from PyPI <https://pypi.org/project/numpy/1.26.0b1/>; source archives, release notes, and wheel hashes are available on Github <https://github.com/numpy/numpy/releases/tag/v1.26.0b1>. *Contributors* A total of 11 people contributed to this release. People with a "+" by their names contributed a patch for the first time. - Bas van Beek - Charles Harris - Matti Picus - Melissa Weber Mendonça - Ralf Gommers - Sayed Adel - Sebastian Berg - Stefan van der Walt - Tyler Reddy - Warren Weckesser *Pull requests merged* A total of 18 pull requests were merged for this release. - #24305: MAINT: Prepare 1.26.x branch for development - #24308: MAINT: Massive update of files from main for numpy 1.26 - #24322: CI: fix wheel builds on the 1.26.x branch - #24326: BLD: update openblas to newer version - #24327: TYP: Trim down the ``_NestedSequence.__getitem__`` signature - #24328: BUG: fix choose refcount leak - #24337: TST: fix running the test suite in builds without BLAS/LAPACK - #24338: BUG: random: Fix generation of nan by dirichlet. - #24340: MAINT: Dependabot updates from main - #24342: MAINT: Add back NPY_RUN_MYPY_IN_TESTSUITE=1 - #24353: MAINT: Update ``extbuild.py`` from main. - #24356: TST: fix distutils tests for deprecations in recent setuptools... - #24375: MAINT: Update cibuildwheel to version 2.15.0 - #24381: MAINT: Fix codespaces setup.sh script - #24403: ENH: Vendor meson for multi-target build support - #24404: BLD: vendor meson-python to make the Windows builds with SIMD... - #24405: BLD, SIMD: The meson CPU dispatcher implementation - #24406: MAINT: Remove versioneer *Build system changes* In this release, NumPy has switched to Meson as the build system and meson-python as the build backend. Installing NumPy or building a wheel can be done with standard tools like ``pip`` and ``pypa/build``. The following are supported: - Regular installs: ``pip install numpy`` or (in a cloned repo) ``pip install .`` - Building a wheel: ``python -m build`` (preferred), or ``pip wheel .`` - Editable installs: ``pip install -e . --no-build-isolation`` - Development builds through the custom CLI implemented with spin <https://github.com/scientific-python/spin>: ``spin build``. All the regular ``pip`` and ``pypa/build`` flags (e.g., ``--no-build-isolation``) should work as expected. *NumPy-specific build customization* Many of the NumPy-specific ways of customizing builds have changed. The ``NPY_*`` environment variables which control BLAS/LAPACK, SIMD, threading, and other such options are no longer supported, nor is a ``site.cfg`` file to select BLAS and LAPACK. Instead, there are command-line flags that can be passed to the build via ``pip``/``build``'s config-settings interface. These flags are all listed in the ``meson_options.txt`` file in the root of the repo. Detailed documented will be available before the final 1.26.0 release; for now please see `the SciPy building from source <http://scipy.github.io/devdocs/building/index.html> docs since most build customization works in an almost identical way in SciPy as it does in NumPy. *Build dependencies* While the runtime dependencies of NumPy have not changed, the build dependencies have. Because we temporarily vendor Meson and meson-python, there are several new dependencies - please see the ``[build-system]`` section of ``pyproject.toml`` for details. *Troubleshooting* This build system change is quite large. In case of unexpected issues, it is still possible to use a ``setup.py``-based build as a temporary workaround (on Python 3.9-3.11, not 3.12), by copying ``pyproject.toml.setuppy`` to ``pyproject.toml``. However, please open an issue with details on the NumPy issue tracker. We aim to phase out ``setup.py`` builds as soon as possible, and therefore would like to see all potential blockers surfaced early on in the 1.26.0 release cycle. Cheers, Charles Harris

1 0

port minpack2.dcsrch to Python, future removal of minpack2?
by Andrew Nelson Aug. 13, 2023

Aug. 13, 2023

Currently `scipy.optimize.minpack2` is used in linesearch operations in `scipy.optimize._linesearch.line_search_wolfe1`. minpack2 is a Fortran module that exports two functions, dcsrch and dcstep. I have ported the contents of those functions to Python, see a PR on my fork at https://github.com/andyfaff/scipy/pull/57. I have examined the relative timings of the Fortran module with the Python version using the example at https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.line_se…, and there is no difference (albeit for a simple example). I propose that we switch to using the Python port, as this will be easier to maintain than the Fortran version. The Python version is a straightforward translation of the Fortran. At the moment minpack2 can be imported from scipy.optimize. Its namespace is deprecated for removal in scipy 2. I'm not sure how much use it actually gets outside our internal use of it. When we get to that point we could remove the module. I'd appreciate thoughts on this proposal.

2 1

Sparse Arrays working group
by CJ Carey Aug. 12, 2023

Aug. 12, 2023

Are you interested in helping to move the scipy.sparse API away from matrix-like semantics? Are you a fan of scipy.sparse in general? If so, you may be interested in attending our biweekly Sparse Array meetings <https://scientific-python.org/calendars/#:~:text=Sparse%20Arrays%20Group,-S…>. You can also follow along asynchronously via the meeting notes: https://hackmd.io/9UCjKdRUTEeoS8KdQNll-A?view We just had our second meeting after the 2023 Developer Summit <https://hackmd.io/1Q2832LDR_2Uv_-cV-wnYg>, and there's plenty of work to be done yet. The next meeting will be two weeks from today, on July 21st. -CJ, on behalf of the Sparse Arrays working group

3 3

Discuss Addition of Critically Sampled Polyphase Channelizer in Scipy.Signal
by Adam Thompson Aug. 9, 2023

Aug. 9, 2023

Hi SciPy Devs, I'm the creator of [cuSignal](https://github.com/rapidsai/cusignal), a GPU-port of the SciPy `signal` module to GPU. As part of this library, we have implemented much of the scipy.signal API into cuSignal and have selectively expanded features, particularly for phased array signal processing. Further, as part of a CZI grant, NVIDIA has agreed to port our work with cuSignal to CuPy and archive cuSignal as a project. One of the functions that is currently *not* in SciPy but *is* in cuSignal is the critically sampled polyphase channelizer (`cusignal.channelize_poly`). Polyphase channelizers are commonly used in the signal processing community to channelize complex valued samples into N channels of equal bandwidth where each channel is a downsampled version of the input signal. In cusignal, we received a feature request to implement the standard, critically sampled polyphase channelizer based on fred harris' paper (https://studylib.net/doc/25433453/digital-receivers-and-transmitters-using-…...). We ported the provided MATLAB code, following that provided in Appendix-I (10-stage channelizer) to a CUDA kernel and packaged it with cusignal. In addition to the polyphase resampler, the polyphase channelizer is one of the more commonly used functions in the cusignal library, and I strongly believe the Scipy community would benefit from implementing it in their library. Porting the MATLAB script to vanilla NumPy shouldn't be too challenging, and given our implementation with CUDA, successful porting from MATLAB -> NumPy for this algorithm should be relatively straightforward. Thanks! Adam Thompson (@awthomp) - NVIDIA Principal Technical Product Manager for Sensor Processing and creator of cuSignal.

3 4

move __init__.py's import of importlib into __getattr__?
by Nathan Jensen Aug. 9, 2023

Aug. 9, 2023

Hi, On a project I work on we encountered a complicated issue when upgrading from scipy 1.8 to scipy 1.10, and we traced it to being introduced in scipy 1.9 with the changes to how scipy submodules are imported. The issue is described here: https://github.com/ninia/jep/issues/487 Some background: The application uses Jep which enables Java processes to run CPython inside the process. Jep supports two types of interpreters, SubInterpreters which use CPython's sub-interpreter API, and SharedInterpreters which do not. The application is using SubInterpreters because it predates Jep's introduction of SharedInterpreters by many years and there are pros and cons of each type of interpreter and it's not trivial to switch. Because most CPython extension modules are not coded or tested with sub-interpreters in mind, Jep introduced a concept called "shared modules" that shares a module across sub-interpreters and ensures it is only ever imported once per process and is never disposed of or garbage collected in that process. scipy is imported as a shared module so it will work in sub-interpreters but then the usage of importlib is messing up the subpackage import. It's complicated but explained in that issue I linked above. We found a fix for the issue is to move the import of importlib in scipy's __init__.py into the __getattr__ method. https://github.com/scipy/scipy/blob/main/scipy/__init__.py#L185 So what I'm wondering is, if I submitted a pull request that moved the import of importlib into __getattr__, either before the if name in modules statement or inside the if name in modules statement, is that a change that the scipy team would consider accepting for a future release?

2 1

ENH: fft: support array API standard (#19005)
by Lucas Colley Aug. 8, 2023

Aug. 8, 2023

Hi all, I'm writing to raise visibility for my PR which adds support for the Array API standard to scipy.fft. I am working on this as part of my internship at Quansight Labs. ENH: fft: support array API standard by lucascolley · Pull Request #19005 · scipy/scipy (github.com) <https://github.com/scipy/scipy/pull/19005> The machinery used by this PR and coverage of scipy.cluster were merged recently <https://github.com/scipy/scipy/pull/18668>. More context on the standard and the status of SciPy support for it can be found at the top of the PR. fft is an array API standard extension module <https://data-apis.org/array-api/latest/extensions/fourier_transform_functio…>, which means that these changes allow automatic dispatching - for the functions which are included in the standard - to any array library which complies with the standard. NumPy, CuPy, and PyTorch have plans to fully comply with the standard, and array-api-compat <https://github.com/data-apis/array-api-compat> means that this PR already dispatches to them, as well as numpy.array_api <https://numpy.org/doc/stable/reference/array_api.html>. JAX and Dask also have implementations in progress. This PR includes a substantive refactor of the fft submodule and a full coverage of changes to make it compatible with the standard. This is accompanied by modifications to the unit tests for (optional) testing with other array libraries, and some new tests of the added functionality. Full details of the changes can be found in the PR description. Any comments or suggestions are more than welcome and very much appreciated! Cheers, Lucas

1 0

ENH: Geometric quality metrics for random sampling (#18893)
by Artem Glebov Aug. 3, 2023

Aug. 3, 2023

Hi, I would like to add a new function to the scipy.stats.qmc module to calculate simple quality metrics for random samples based on the geometric properties of the distribution of points in a sample: https://github.com/scipy/scipy/pull/18893 This change implements the following request: https://github.com/scipy/scipy/issues/14037 Please let me know if you have any objections or comments. Thanks

1 0