Mailman 3 September 2023 - NumPy-Discussion

ANN: NumExpr 2.8.6 Released
by Robert McLeod 13 Sep '23

13 Sep '23

Hi everyone, NumExpr 2.8.6 is a release to deal with issues related to downstream `pandas` where the sanitization blacklist was hitting private variables used in their evaluate. In addition the sanitization was hitting on scientific notation. For those who do not wish to have sanitization on by default, it can be changed by setting an environment variable, `NUMEXPR_SANITIZE=0`. If you use `pandas` in your packages it is advisable you pin `numexpr >= 2.8.6` in your requirements. Project documentation is available at: http://numexpr.readthedocs.io/ Changes from 2.8.5 to 2.8.6 --------------------------- * The sanitization can be turned off by default by setting an environment variable, `set NUMEXPR_SANITIZE=0` * Improved behavior of the blacklist to avoid triggering on private variables and scientific notation numbers. What's Numexpr? --------------- Numexpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like "3*a+4*b") are accelerated and use less memory than doing the same calculation in Python. It has multi-threaded capabilities, as well as support for Intel's MKL (Math Kernel Library), which allows an extremely fast evaluation of transcendental functions (sin, cos, tan, exp, log...) while squeezing the last drop of performance out of your multi-core processors. Look here for a some benchmarks of numexpr using MKL: https://github.com/pydata/numexpr/wiki/NumexprMKL Its only dependency is NumPy (MKL is optional), so it works well as an easy-to-deploy, easy-to-use, computational engine for projects that don't want to adopt other solutions requiring more heavy dependencies. Where I can find Numexpr? ------------------------- The project is hosted at GitHub in: https://github.com/pydata/numexpr You can get the packages from PyPI as well (but not for RC releases): http://pypi.python.org/pypi/numexpr Documentation is hosted at: http://numexpr.readthedocs.io/en/latest/ Share your experience --------------------- Let us know of any bugs, suggestions, gripes, kudos, etc. you may have. Enjoy data! -- Robert McLeod robbmcleod(a)gmail.com robert.mcleod(a)hitachi-hightech.com

2 1

Skew symmetric/matrix cross product function proposal
by Mark Stephenson 12 Sep '23

12 Sep '23

As proposed in https://github.com/numpy/numpy/issues/24687 Dynamicists frequently use the matrix representation of the cross product (https://en.m.wikipedia.org/wiki/Cross_product#Conversion_to_matrix_multipli…), and the lack of a skew symmetric operator in NumPy is a frequent source of annoyance. I propose either adding a skew function to linalg that returns the skew symmetric matrix of a vector `np.cross(a, np.identity(a.shape[0]) * -1)` (from https://stackoverflow.com/questions/66707295/numpy-cross-product-matrix-fun…), or more radically, making the second argument `b` of np.cross optional and have `np.cross(a)` return the matrix representation of [a x]. For the latter, would need to consider the interaction with other kwargs and different input shapes.

1 0

Next Documentation team meeting
by Mukulika Pahari 11 Sep '23

11 Sep '23

Hi all! Our next Documentation Team meeting will happen on *Monday, June 5* at ***4PM UTC***. We now alternate the meeting times to be a bit more inclusive. This means that we'll have a meeting at 12pm UTC every 28 days, and a meeting at 4pm UTC every 28 days. All are welcome - you don't need to already be a contributor to join. If you have questions or are curious about what we're doing, we'll be happy to meet you! If you wish to join on Zoom, use this (updated) link: https://numfocus-org.zoom.us/j/85016474448?pwd=TWEvaWJ1SklyVEpwNXUrcHV1YmFJQ... Here's the permanent hackmd document with the meeting notes (still being updated): https://hackmd.io/oB_boakvRqKR-_2jRV-Qjg Hope to see you around! * You can also visit https://scientific-python.org/calendars to add the NumPy community calendar as an .ics file to your preferred calendar manager. * Best wishes, Mukulika

1 5

Suggestions for begginer
by Michael Daniel 10 Sep '23

10 Sep '23

Hello Everyone! I am new to Numpy community.I would like to start to contribute in project although i dont know how to start. I have 1 year expierence in python/numpy programming.

2 1

NumPy 1.26.0rc1 released
by Charles R Harris 07 Sep '23

07 Sep '23

Hi All, On behalf of the NumPy team, I'm pleased to announce the release of NumPy 1.26.0rc1. The NumPy 1.26.0 release is a continuation of the 1.25.x release cycle with the addition of Python 3.12.0 support. Python 3.12 dropped distutils, consequently supporting it required finding a replacement for the setup.py/distutils based build system NumPy was using. We have chosen to use the Meson build system instead, and this is the first NumPy release supporting it. This is also the first release that supports Cython 3.0 in addition to retaining 0.29.X compatibility. Supporting those two upgrades was a large project, over 100 files have been touched in this release. The changelog doesn't capture the full extent of the work, special thanks to Ralf Gommers, Sayed Adel, Stéfan van der Walt, and Matti Picus who did much of the work in the main development branch. The highlights of this release are: - Python 3.12.0 support. - Cython 3.0.0 compatibility. - Use of the Meson build system - Updated SIMD support - f2py fixes, meson and bind(x) support - Support for the updated Accelerate BLAS/LAPACK library The Python versions supported by this release are 3.9-3.12. Note that no 32 bit wheels are provided. Wheels can be downloaded from PyPI <https://pypi.org/project/numpy/1.26.0rc1/>; source archives, release notes, and wheel hashes are available on Github <https://github.com/numpy/numpy/releases/tag/v1.26.0rc1> *Contributors* A total of 18 people contributed to this release. People with a \"+\" by their names contributed a patch for the first time. - @DWesl - Albert Steppi + - Bas van Beek - Charles Harris - Developer-Ecosystem-Engineering - Jake Vanderplas - Marten van Kerkwijk - Matti Picus - Melissa Weber Mendonça - Namami Shanker - Nathan Goldbaum - Ralf Gommers - Rohit Goswami - Sayed Adel - Sebastian Berg - Stefan van der Walt - Tyler Reddy - Warren Weckesser *Build system changes* In this release, NumPy has switched to Meson as the build system and meson-python as the build backend. Installing NumPy or building a wheel can be done with standard tools like ``pip`` and ``pypa/build``. The following are supported: - Regular installs: ``pip install numpy`` or (in a cloned repo) ``pip install .`` - Building a wheel: ``python -m build`` (preferred), or ``pip wheel .`` - Editable installs: ``pip install -e . --no-build-isolation`` - Development builds through the custom CLI implemented with spin <https://github.com/scientific-python/spin>: ``spin build``. All the regular ``pip`` and ``pypa/build`` flags (e.g., ``--no-build-isolation``) should work as expected. *NumPy-specific build customization* Many of the NumPy-specific ways of customizing builds have changed. The ``NPY_*`` environment variables which control BLAS/LAPACK, SIMD, threading, and other such options are no longer supported, nor is a ``site.cfg`` file to select BLAS and LAPACK. Instead, there are command-line flags that can be passed to the build via ``pip``/``build``'s config-settings interface. These flags are all listed in the ``meson_options.txt`` file in the root of the repo. Detailed documented will be available before the final 1.26.0 release; for now, please see `the SciPy building from source <http://scipy.github.io/devdocs/building/index.html> docs since most build customization works in an almost identical way in SciPy as it does in NumPy. *Build dependencies* While the runtime dependencies of NumPy have not changed, the build dependencies have. Because we temporarily vendor Meson and meson-python, there are several new dependencies - please see the ``[build-system]`` section of ``pyproject.toml`` for details. *Troubleshooting* This build system change is quite large. In case of unexpected issues, it is still possible to use a ``setup.py``-based build as a temporary workaround (on Python 3.9-3.11, not 3.12), by copying ``pyproject.toml.setuppy`` to ``pyproject.toml``. However, please open an issue with details on the NumPy issue tracker. We aim to phase out ``setup.py`` builds as soon as possible, and therefore would like to see all potential blockers surfaced early on in the 1.26.0 release cycle. Cheers, Charles Harris

2 2

Dealing with static local variables in Numpy
by Nicolas Holzschuch 01 Sep '23

01 Sep '23

Hello, This is my first post to this group; I'd like to start by expressing my appreciation for the amazing work in developing and maintaining Numpy. I have a question. Numpy has quite a lot of static local variables (variables defined as static inside a function, like this (core/src/multiarraymodule.c, line 4483): if (raise_exceptions) { static PyObject *too_hard_cls = NULL; /* ... */ } I understand that these variables provide local caching and are important for efficiency. They do however cause some issues when dealing with multiple subinterpreters, where the static local variable might have been initialized by one of the subinterpreters, and is not reset when accessed by another subinterpreter. More globally, they cannot be reset when the Numpy module is released, and thus will likely cause an issue if it is reloaded after being released. I have seen the issue mentionned in at least one pull request: https://github.com/numpy/numpy/pull/15169 and in several issues. If I understand correctly, the issue is not considered as important because subinterpreters are not yet prominent in CPython, static local variables provide an important service in caching data locally (instead of exposing these variables globally). So the benefits outweigh the costs and risks (that would be a huge change to the code base). I happen to maintain, compile and run a version of Python on iOS (https://github.com/holzschu/a-shell/ or https://apps.apple.com/us/app/a-shell/id1473805438), where I have to remove all these static local variables, because of the specificity of the platform (in order to run Python multiple times, I have to release and reset all modules). Right now, I'm maintaining the changes to the code base in a separate branch (https://github.com/holzschu/numpy/) and not necessarily in a very clean way. With the recent renewed interest in subinterpreters, I was wondering if there was a way I could contribute these changes back to the main numpy branch. I would have to clean up the code, obviously, and probably get guidance on how to do it cleanly, but the first question is: would there be an interest, or is that something I should keep in my separate branch? From a technical point of view, about 80% of these static local variables are just before a call to npy_cache_import(), and the most efficient way to do it (in terms of lines of code) is just to remove the part where npy_cache_import uses the static local variable. You pay a price in performance, but gain in usability. Best regards, Nicolas Holzschuch

3 2