Mailman 3 August 2023 - NumPy-Discussion

[NEP 52] Introduction of `array_utils` public namespace in `np.lib` submodule
by Mateusz Sokol Aug. 28, 2023

Aug. 28, 2023

Hi all! The two most important goals of NEP 52 are cleanups of the top `np` namespace and `np.lib` namespace. A set of functions from `np` and `np.lib.*` submodules has been identified that could establish a new public namespace within the `np.lib` submodule. The issue https://github.com/numpy/numpy/issues/24166 proposes a `np.lib.array_utils` name for the new namespace that could host functions such as `byte_bounds` (originally in `np`), `normalize_axis_tuple` (originally in `np.lib.stride_tricks`) or `normalize_axis_index` (originally in `np.lib.shape_base`). If you have any thoughts on naming/content of the new namespace, please share your feedback! Best regards, Mateusz

1 0

No numpy documentation meeting 08/28
by Ross Barnowski Aug. 28, 2023

Aug. 28, 2023

Hi all, There will be no documentation meeting this tomorrow, Monday 08/28 as the regular attendees have conflicts. We'll pick back up in 2 weeks! Ross

1 0

Inquiry about the difference between numpy.trunc() and numpy.fix()
by Bao Guo Aug. 25, 2023

Aug. 25, 2023

I am writing to seek clarification on the specific differences between the numpy.trunc() and numpy.fix() functions. I have come across opinions stating that these two functions exist for backward compatibility and that they serve different functional requirements. However, I am puzzled as to why these functions could not be merged or one of them upgraded to encompass both functionalities. Despite my efforts in researching and contemplating on this matter, I have not come across a convincing explanation. Could anyone kindly shed some light on this matter? I am seeking a deeper understanding of the design choices made for these functions and the rationale behind their coexistence. Thank you in advance for your time and assistance.

2 2

welcome Andrew Nelson to the NumPy maintainers team
by Ralf Gommers Aug. 22, 2023

Aug. 22, 2023

Hi all, On behalf of the steering council, I am very happy to announce that Andrew is joining the Maintainers team. Andrew has been contributing to our CI setup in particular for the past year, and has contributed for example the Cirrus CI setup and the musllinux builds: https://github.com/numpy/numpy/pulls/andyfaff. Welcome Andrew, I'm looking forward to working with you more! Cheers, Ralf

7 6

How does Numpy model non-float arrays?
by Dylon Edwards Aug. 22, 2023

Aug. 22, 2023

It is my understanding that Numpy accelerates array operations with BLAS where possible, but BLAS does not support all the dtypes that Numpy does. How does Numpy model non-float arrays like arrays of dtype=bool or dtype=object?

2 1

Generalized UFunc without output dimension specified as argument
by Doug Turnbull Aug. 20, 2023

Aug. 20, 2023

First of all, I really love the docs of the C API :) It's way above what I would expect! I was reviewing the signature possibilities for generalized UFuncs, and had a question https://numpy.org/doc/stable/reference/c-api/generalized-ufuncs.html I am playing with a UFunc that scores and returns some top N, where N could be specified the user. IE the user might do get_most_similar(X, y, n=10) You can imagine situations where this could happen in similarity functions, where we want to get some Top N rows of X most similar to y. But sometimes users will want 10, or 100, or need to page through results etc. For performance reasons, I wouldn't want to maintain an index of every row of X, I'd prefer to only have to care about the top 10 or so. I wonder what the best way to do this? One thought I had was always set the output dimension to 10 for now, and handle paging on the python side by perhaps also having an offset parameter for my function, to window into the similar results. The second thought I had was to just get 100 instead of 10, as that probably is enough for most use cases. And users can slice out what they need. It's a little annoying in terms of perf cost, but probably not a big deal. But it would be convenient to just let the user specify the N they want. Thanks for any insights! -Doug

2 1

Arbitrarily large random integers
by Dan Schult Aug. 19, 2023

Aug. 19, 2023

How can we use numpy's random `integers` function to get uniformly selected integers from an arbitrarily large `high` limit? This is important when dealing with exact probabilities in combinatorially large solution spaces. I propose that we add the capability for `integers` to construct arrays of type object_ by having it construct python int's as the objects in the returned array. This would allow arbitrarily large integers. The Python random library's `randrange` constructs values for arbitrary upper limits -- and they are exact when using subclasses of `random.Random` with a `getrandbits` methods (which includes the default rng for most operating systems). Numpy's random `integers` function rightfully raises on `integers(20**20, dtype=int64)` because the upper limit is above what can be held in an `int64`. But Python `int` objects store arbitrarily large integers. So I would expect `integers(20**20, dtype=object)` to create random integers on the desired range. Instead a TypeError is raised `Unsupported dtype dtype('O') for integers`. It seems we could provide support for dtype('O') by constructing Python `int` values and this would allow arbitrarily large ranges of integers. The core of this functionality would be close to the seven lines used in [the code of random.Random._randbelow](https://github.com/python/cpython/blob/eb953d6e44… which 1) finds the number of bits needed to describe the `high` argument. 2) generates that number of random bits. 3) converts them to a python int and checks if it is larger than the input `high`. If so, repeat from step 2. I realize that people can just use `random.randrange` to obtain this functionality, but that doesn't return an array, and uses a different RNG possibly requiring tracking two RNG states. This text was also used to create [Issue #24458](https://github.com/numpy/numpy/issues/24458)

3 2

ANN: SciPy 1.11.2
by Tyler Reddy Aug. 18, 2023

Aug. 18, 2023

Hi all, On behalf of the SciPy development team, I'm pleased to announce the release of SciPy 1.11.2, which includes new wheels for Python 3.12 and musllinux. Sources and binary wheels can be found at: https://pypi.org/project/scipy/ and at: https://github.com/scipy/scipy/releases/tag/v1.11.2 One of a few ways to install this release with pip: pip install scipy==1.11.2 ====================== SciPy 1.11.2 Release Notes ====================== SciPy 1.11.2 is a bug-fix release with no new features compared to 1.11.1. Python 3.12 and musllinux wheels are provided with this release. Authors ======= * Name (commits) * Evgeni Burovski (2) * CJ Carey (3) * Dieter Werthmüller (1) * elbarso (1) + * Ralf Gommers (2) * Matt Haberland (1) * jokasimr (1) + * Thilo Leitzbach (1) + * LemonBoy (1) + * Ellie Litwack (2) + * Sturla Molden (1) * Andrew Nelson (5) * Tyler Reddy (39) * Daniel Schmitz (6) * Dan Schult (2) * Albert Steppi (1) * Matus Valo (1) * Stefan van der Walt (1) A total of 18 people contributed to this release. People with a "+" by their names contributed a patch for the first time. This list of names is automatically generated, and may not be fully complete. Issues closed for 1.11.2 -------------------------------- * `#4690 <https://github.com/scipy/scipy/issues/4690>`__: special.jn_zeros(281, 6) hangs * `#12247 <https://github.com/scipy/scipy/issues/12247>`__: Complex matrix square root of positive semi-definite matrix * `#18398 <https://github.com/scipy/scipy/issues/18398>`__: BUG: \`loadmat\` fails to load matlab structures with anonymous... * `#18603 <https://github.com/scipy/scipy/issues/18603>`__: BUG: Floating point CSC with int64 indices doesn't work with... * `#18730 <https://github.com/scipy/scipy/issues/18730>`__: BUG: \`scipy.optimize.minimize\` fails when \`dtype=float32\`... * `#18788 <https://github.com/scipy/scipy/issues/18788>`__: DOC: Broken link to installation instructions in README.rst * `#18792 <https://github.com/scipy/scipy/issues/18792>`__: BUG: Build failure with Cython 3.0.0b3 if scipy is already installed * `#18793 <https://github.com/scipy/scipy/issues/18793>`__: BUG: optimize.least_squares with method='trf' yields wrong result... * `#18800 <https://github.com/scipy/scipy/issues/18800>`__: BUG: cKDtree.query no longer accepts DataFrame as input * `#19002 <https://github.com/scipy/scipy/issues/19002>`__: Spalde error with scipy 1.10: 0-th dimension must be fixed * `#19022 <https://github.com/scipy/scipy/issues/19022>`__: BUG: <Compilation of scipy 1.11 falls with python3.12> * `#19026 <https://github.com/scipy/scipy/issues/19026>`__: BUG: Compilation of scipy 1.10.1 and 1.11.1 fails with Python... Pull requests for 1.11.2 ------------------------------- * `#17918 <https://github.com/scipy/scipy/pull/17918>`__: BUG: Fix error in linalg/_matfuncs_sqrtm.py * `#18644 <https://github.com/scipy/scipy/pull/18644>`__: BUG: sparse.linalg: Cast index arrays to intc before calling... * `#18784 <https://github.com/scipy/scipy/pull/18784>`__: Allow johnsonsu parameters to be floats * `#18785 <https://github.com/scipy/scipy/pull/18785>`__: MAINT: stats: fix NumPy DeprecationWarnings * `#18787 <https://github.com/scipy/scipy/pull/18787>`__: REL, MAINT: prep for 1.11.2 * `#18790 <https://github.com/scipy/scipy/pull/18790>`__: DOC: Fix broken link to installation guide in README * `#18804 <https://github.com/scipy/scipy/pull/18804>`__: BUG: Ensure cKDtree.query does not pass Pandas DataFrame to np.isfinite * `#18809 <https://github.com/scipy/scipy/pull/18809>`__: CI, MAINT: 32-bit Pillow pin * `#18810 <https://github.com/scipy/scipy/pull/18810>`__: BLD: copy \`cython_optimize.pxd\` to build dir * `#18825 <https://github.com/scipy/scipy/pull/18825>`__: BUG: make \`L-BFGS-B\` optimizer work with single precision gradient * `#18831 <https://github.com/scipy/scipy/pull/18831>`__: BUG: io/matlab: Fix loading of mat files containing fn handles... * `#18859 <https://github.com/scipy/scipy/pull/18859>`__: BUG: make Bessel-roots function not hang and not skip roots * `#18894 <https://github.com/scipy/scipy/pull/18894>`__: DOC: linking interp1d docstring to tutorial * `#18896 <https://github.com/scipy/scipy/pull/18896>`__: BUG: lsq trf gives x=1e-10 if x0 is near a bound * `#18937 <https://github.com/scipy/scipy/pull/18937>`__: CI/BLD: create cp312 wheels * `#18961 <https://github.com/scipy/scipy/pull/18961>`__: DOC: Fix installation instructions using venv/pip * `#18985 <https://github.com/scipy/scipy/pull/18985>`__: CI: move the musllinux Cirrus job to GHA, optimize other jobs * `#18999 <https://github.com/scipy/scipy/pull/18999>`__: CI: reduce Cirrus CI usage during wheel builds * `#19004 <https://github.com/scipy/scipy/pull/19004>`__: BUG: interpolate: fix spalde with len(c) < len(t) * `#19025 <https://github.com/scipy/scipy/pull/19025>`__: BUG: pass unused xrtol in fmin_bfgs to _minimize_bfgs * `#19027 <https://github.com/scipy/scipy/pull/19027>`__: BLD: rename \`setup.py\` to \`_setup.py\` to signal it should... * `#19034 <https://github.com/scipy/scipy/pull/19034>`__: MAINT: NumPy 1.25.x deprecations * `#19054 <https://github.com/scipy/scipy/pull/19054>`__: MAINT: ensure cobyla objective returns scalar Checksums ========= MD5 ~~~ ff799002607fbaed2d81a1d25cce38a3 Changelog a1ef4ec8ff10e50f2ac1a8c0ee253ffd README.txt 7038aeb6c007e7114afa2539135bd142 scipy-1.11.2-cp310-cp310-macosx_10_9_x86_64.whl 3a375f4efced07df10e2990d1d73c811 scipy-1.11.2-cp310-cp310-macosx_12_0_arm64.whl 6be4527b35d09313ce2802f99f71527a scipy-1.11.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl cc681011c84e9eb051a43ea895cf4073 scipy-1.11.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl de9afa37cd0c3654a5a4abe2a29548e4 scipy-1.11.2-cp310-cp310-musllinux_1_1_x86_64.whl 85c785288036b94826c2c564116b5e6b scipy-1.11.2-cp310-cp310-win_amd64.whl e08a50bf4c113a1e310bece42410a01f scipy-1.11.2-cp311-cp311-macosx_10_9_x86_64.whl fcf37aaee7f567daf43e8f613bd80ef1 scipy-1.11.2-cp311-cp311-macosx_12_0_arm64.whl a8a8b98a8162f5684597f4852a494f9a scipy-1.11.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 5cccac1aaca9a7a9701212b523771486 scipy-1.11.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl d59f8140d6eb9052c1958a21d9fa97a9 scipy-1.11.2-cp311-cp311-musllinux_1_1_x86_64.whl 5ea7ab2643e34a106b0b07eddf50f96d scipy-1.11.2-cp311-cp311-win_amd64.whl 2cedd23dd78ffb8eb8ef09ce24f68b21 scipy-1.11.2-cp312-cp312-macosx_10_9_x86_64.whl 5109008341246dd63e170b550628f6b6 scipy-1.11.2-cp312-cp312-macosx_12_0_arm64.whl ea58c5dc26dd439a19bf0b34eca42713 scipy-1.11.2-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl c6eb39fbca24c96ea91f45ed4694e16f scipy-1.11.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 170c392f56185e9c8336f3501e38a675 scipy-1.11.2-cp312-cp312-musllinux_1_1_x86_64.whl b1b2f01e007b05aad3461f1e454e2c57 scipy-1.11.2-cp312-cp312-win_amd64.whl 6d7de323abd9b33f53ea77b0915c0e5e scipy-1.11.2-cp39-cp39-macosx_10_9_x86_64.whl 900b0591520bd3864b34fd619b0a500a scipy-1.11.2-cp39-cp39-macosx_12_0_arm64.whl f6621b3355fec0fe58a37e463a08fa0e scipy-1.11.2-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 4f3ea3d87dd0bfd2816583080b7e3934 scipy-1.11.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 268340a51245f23b81a3f40dae5819a1 scipy-1.11.2-cp39-cp39-musllinux_1_1_x86_64.whl 2e2b1fcf6ea9c311576700daf18ccc19 scipy-1.11.2-cp39-cp39-win_amd64.whl SHA256 ~~~~~~ ba310500c0ebddc3df18f083c30fec060ba5b04b5d38493f7939ee3cafc00302 Changelog 82afc655164a2a7609c7c59e1c186d7ea0f6ffee2fcfb6c83c26658f6656f906 README.txt 2b997a5369e2d30c97995dcb29d638701f8000d04df01b8e947f206e5d0ac788 scipy-1.11.2-cp310-cp310-macosx_10_9_x86_64.whl 95763fbda1206bec41157582bea482f50eb3702c85fffcf6d24394b071c0e87a scipy-1.11.2-cp310-cp310-macosx_12_0_arm64.whl e367904a0fec76433bf3fbf3e85bf60dae8e9e585ffd21898ab1085a29a04d16 scipy-1.11.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl d690e1ca993c8f7ede6d22e5637541217fc6a4d3f78b3672a6fe454dbb7eb9a7 scipy-1.11.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl d2b813bfbe8dec6a75164523de650bad41f4405d35b0fa24c2c28ae07fcefb20 scipy-1.11.2-cp310-cp310-musllinux_1_1_x86_64.whl afdb0d983f6135d50770dd979df50bf1c7f58b5b33e0eb8cf5c73c70600eae1d scipy-1.11.2-cp310-cp310-win_amd64.whl 8d9886f44ef8c9e776cb7527fb01455bf4f4a46c455c4682edc2c2cc8cd78562 scipy-1.11.2-cp311-cp311-macosx_10_9_x86_64.whl 1342ca385c673208f32472830c10110a9dcd053cf0c4b7d4cd7026d0335a6c1d scipy-1.11.2-cp311-cp311-macosx_12_0_arm64.whl b133f237bd8ba73bad51bc12eb4f2d84cbec999753bf25ba58235e9fc2096d80 scipy-1.11.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 3aeb87661de987f8ec56fa6950863994cd427209158255a389fc5aea51fa7055 scipy-1.11.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 90d3b1364e751d8214e325c371f0ee0dd38419268bf4888b2ae1040a6b266b2a scipy-1.11.2-cp311-cp311-musllinux_1_1_x86_64.whl f73102f769ee06041a3aa26b5841359b1a93cc364ce45609657751795e8f4a4a scipy-1.11.2-cp311-cp311-win_amd64.whl fa4909c6c20c3d91480533cddbc0e7c6d849e7d9ded692918c76ce5964997898 scipy-1.11.2-cp312-cp312-macosx_10_9_x86_64.whl ac74b1512d38718fb6a491c439aa7b3605b96b1ed3be6599c17d49d6c60fca18 scipy-1.11.2-cp312-cp312-macosx_12_0_arm64.whl b8425fa963a32936c9773ee3ce44a765d8ff67eed5f4ac81dc1e4a819a238ee9 scipy-1.11.2-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 542a757e2a6ec409e71df3d8fd20127afbbacb1c07990cb23c5870c13953d899 scipy-1.11.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl ea932570b1c2a30edafca922345854ff2cd20d43cd9123b6dacfdecebfc1a80b scipy-1.11.2-cp312-cp312-musllinux_1_1_x86_64.whl 4447ad057d7597476f9862ecbd9285bbf13ba9d73ce25acfa4e4b11c6801b4c9 scipy-1.11.2-cp312-cp312-win_amd64.whl b0620240ef445b5ddde52460e6bc3483b7c9c750275369379e5f609a1050911c scipy-1.11.2-cp39-cp39-macosx_10_9_x86_64.whl f28f1f6cfeb48339c192efc6275749b2a25a7e49c4d8369a28b6591da02fbc9a scipy-1.11.2-cp39-cp39-macosx_12_0_arm64.whl 214cdf04bbae7a54784f8431f976704ed607c4bc69ba0d5d5d6a9df84374df76 scipy-1.11.2-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 10eb6af2f751aa3424762948e5352f707b0dece77288206f227864ddf675aca0 scipy-1.11.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 0f3261f14b767b316d7137c66cc4f33a80ea05841b9c87ad83a726205b901423 scipy-1.11.2-cp39-cp39-musllinux_1_1_x86_64.whl 2c91cf049ffb5575917f2a01da1da082fd24ed48120d08a6e7297dfcac771dcd scipy-1.11.2-cp39-cp39-win_amd64.whl

1 0

Cirrus testing
by Charles R Harris Aug. 16, 2023

Aug. 16, 2023

Hi All, This is a heads up that we have already exceeded our allotment of free time on Cirrus CI. They are giving us a pass this month, but next month they will start enforcing the limits. That will impact both our testing and our releases. We have taken steps to reduce our use of Cirrus, but it could still be a problem, we should have a contingency plan in place for at least the next two months. Thoughts? Chuck.

4 6

Test failures m1 OSX with apple clang
by Doug Turnbull Aug. 14, 2023

Aug. 14, 2023

Hey all First time trying to build / test numpy main branch, so I'm probably doing something wrong. I brought down main and everything built on the first try. However I'm getting test failures related to meson. Specifically the tests in "python -m pytest numpy/core/tests/test_mem_policy.py" all give errors. The errors seem related to meson and visual studio: *> Visual Studio environment is needed to run Ninja. It is recommended to use Meson wrapper:/Users/douglas.turnbull/ws/numpy/venv/bin/meson compile -C .> ERROR: Could not detect Ninja v1.8.2 or newer* This is an m1 mac with Python 3.11.4. Scrolling up in meson_log.txt I see a linker error: *> ld: symbol(s) not found for architecture arm64* Included system information Project version: undefined C compiler for the host machine: cc (clang 14.0.3 "Apple clang version 14.0.3 (clang-1403.0.22.14.1)") C linker for the host machine: cc ld64 857.1 Host machine cpu family: aarch64 Host machine cpu: aarch64 Snippet from meson log *Detecting Apple linker via: `cc -Wl,-v` -> 1stderr:@(#)PROGRAM:ld PROJECT:ld64-857.1BUILD 23:13:29 May 7 2023configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7emLibrary search paths: . /opt/homebrew/lib /usr/local/lib /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/libFramework search paths: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Undefined symbols for architecture arm64: "_main", referenced from: implicit entry/start for main executableld: symbol(s) not found for architecture arm64clang: error: linker command failed with exit code 1 (use -v to see invocation)* Seeing a note recommending gcc in the docs, I'm going to try using gcc rather than clang as next steps. But I wanted to share this issue if it is indeed one. -Doug

2 2