Mailman 3 December 2021 - NumPy-Discussion

Announcing NumExpr 2.8.1
by Robert McLeod 10 Dec '21

10 Dec '21

Hi everyone, This is another maintenance release to further modernize our install script for distributions that do not include `setuptools` by default. Thanks to Antonio Valentino for the changes. Project documentation is available at: http://numexpr.readthedocs.io/ Changes from 2.8.0 to 2.8.1 --------------------------- * Fixed dependency list. * Added ``pyproject.toml`` and modernize the ``setup.py`` script. Thanks to Antonio Valentino for the PR. What's Numexpr? --------------- Numexpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like "3*a+4*b") are accelerated and use less memory than doing the same calculation in Python. It has multi-threaded capabilities, as well as support for Intel's MKL (Math Kernel Library), which allows an extremely fast evaluation of transcendental functions (sin, cos, tan, exp, log...) while squeezing the last drop of performance out of your multi-core processors. Look here for a some benchmarks of numexpr using MKL: https://github.com/pydata/numexpr/wiki/NumexprMKL Its only dependency is NumPy (MKL is optional), so it works well as an easy-to-deploy, easy-to-use, computational engine for projects that don't want to adopt other solutions requiring more heavy dependencies. Where I can find Numexpr? ------------------------- The project is hosted at GitHub in: https://github.com/pydata/numexpr You can get the packages from PyPI as well (but not for RC releases): http://pypi.python.org/pypi/numexpr Documentation is hosted at: http://numexpr.readthedocs.io/en/latest/ Share your experience --------------------- Let us know of any bugs, suggestions, gripes, kudos, etc. you may have. Enjoy data! -- Robert McLeod robbmcleod(a)gmail.com robert.mcleod(a)hitachi-hightech.com

1 0

Speeding up __array_function__ by moving to C (interest in pushing that forward?)
by Sebastian Berg 08 Dec '21

08 Dec '21

Hi all, is anyone interested in digging into speeding up __array_function__? I distracted myself briefly looking into moving (more) of it to C and got up with this: https://github.com/numpy/numpy/compare/main...seberg:faster-array-function The effect seems to be an up to 40% speed improvement for very lightweight functions. (There may be more room for improvement but I am not sure how much and it should not be as easy.) I had never really tried this before, because on older Python versions it was just slower/harder to jump between C and Python. That was due to the lack of "vectorcall", now it is all there, although for PyPy it might still need fixups. The main point is moving the object that we replace our functions with (the entry point) to C [1]. This works, but it was a one-evening quick try, so it needs a lot of polishing: * Fixing `like=` for Python defined functions: They are broken, I have one possible approach, but it is just an "idea stage". (should not be hard, but needs a some thought) * Make sure the C entry-point looks and feel right, such as printing (I am not sure what is left to do here, maybe we can find prior art?) * Enable pickling (should be very easy) * Ensure PyPy compatibility if necessary * Change all warning `stacklevel=` to one lower (in wrapped functions). (simple, but a lot of churn) * General code cleanup Thought I would put it out there. It might be a cool and worthwhile project for someone interested in decreasing overheads and familiar with the Python C-API. Cheers, Sebastian [1] This also avoids the need for packing `args` and `kwargs` into a tuple/dict, which is faster, even if we have to pack it again for `__array_function__` (we can avoid it for the NumPy version though). I had wondered if we could evolve `__array_function__` to accept `*args, **kwargs`. I expect this is only useful if `__array_function__` was defined in C though.

2 1

NumPy Community Meeting Wednesday -- Later time!
by Sebastian Berg 07 Dec '21

07 Dec '21

Hi all, There will be a NumPy Community meeting Wednesday December 8th at 19:00 UTC. Everyone is invited and encouraged to join in and edit the work-in-progress meeting topics and notes at: https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both Best wishes Sebastian PS: 19:00 UTC is 11am Pacific time

1 0

setuptools re-enabling vendored distutils soon, may break numpy.distutils
by Ralf Gommers 07 Dec '21

07 Dec '21

Hi all, Setuptools is going to re-enable a setting which makes it prefer its vendored copy of distutils over the distutils code in the stdlib in two weeks. If you only use setuptools and not numpy.distutils directly, then you're probably fine. If you do use numpy.distutils, then your build may break. It's recommended you test this now - see https://twitter.com/jaraco/status/1467660250954317827 for how. Note that it's not possible to avoid/fix this kind of breakage in a sensible fashion before it happens. Setuptools is going to be making significant backwards-incompatible changes to distutils, so if you are using numpy.distutils then you should be prepared for regular breakage. If you release your package regularly, then it's recommended you always pin setuptools in your pyproject.toml to the last version that's known to work. Setuptools typically just works and the pin will only affect your own from-source installs. The reason not to pin is if you release very irregularly - your package may then need a newer setuptools for Python 3.11 support in Oct'22 for example. It's a trade-off between avoiding breakage and not getting support for new Python versions and bug fixes. Also note the tracking issues for this build system interaction are at: https://github.com/numpy/numpy/issues/18588 https://github.com/pypa/setuptools/issues/2372 Cheers, Ralf

1 0

Fwd: ndarray should offer __format__ that can adjust precision
by Ivan Gonzalez 03 Dec '21

03 Dec '21

It would be nice to be able to use the Python syntax we already use to format the precision of floating numbers in numpy: >>> a = np.array([-np.pi, np.pi]) >>> print(f"{a:+.2f}") [-3.14 +3.14] This is particularly useful when you have large arrangements. The problem is that if you want to do it today, it is not implemented: >>> print(f"{a:+.2f}") Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported format string passed to numpy.ndarray.__format__ In this PR (https://github.com/numpy/numpy/pull/19550) I propose a very basic formatting implementation for numeric numbers that uses `array2string` just like it currently does `str` At first, since we are only considering formatting the numeric type, floating numbers specifically, we are only interested in being able to change the precision, the sign, and possibly the rounding or truncation. Since the `array2string` function already does everything we need, we only need to implement the` __format__` function of the `ndarray` class which parses a predefined format (similar to the one already used by Python for built-in data types) to indicate the parameters before said. I propose a mini format specification inspired in the [Format Specification Mini-Language](https://docs.python.org/3/library/string.html#formatspec). ``` format_spec ::= [sign][.precision][type] sign ::= "+" | "-" | " " precision ::= [0-9]+ type ::= "f" | "e" ``` We are going to consider only 3 arguments of the `array2string` function:` precision`, `suppress_small`,` sign`. In particular, the `type` token sets the` suppress_small` argument to True when the type is `f` and False when it is `e`. This is in order to mimic Python's behavior in truncating decimals when using the fixed-point notation. As @brandon-rhodes said in gh-5543, the behavior when you try to format an array containing Python objects, the behavior should be the same as Python has implemented by default in the `object` class: ` format (a, "") ` should be equivalent to `str (a)` and `format(a, "not empty")` should raise an exception. What remains to be defined is the behavior when trying to format an array with a non-numeric data type (`np.numeric`) other than `np.object_`. Should we raise an exception? In my opinion yes, since in the future formatting is extended -- for example, for dates -- people are aware that before that was not implemented. I'm open to suggestions. - Ivan

3 2

Announcing NumExpr 2.8.0
by Robert McLeod 02 Dec '21

02 Dec '21

Hi everyone, NumExpr 2.8.0 is released. This is mostly a version bump. We now support Python 3.10 and support for 2.7 and 3.5 has been discontinued. Project documentation is available at: http://numexpr.readthedocs.io/ Changes from 2.7.3 to 2.8.0 --------------------------- * Wheels for Python 3.10 are now provided. * Support for Python 2.7 and 3.5 has been discontinued. * All residual support for Python 2.X syntax has been removed, and therefore the setup build no longer makes calls to the `2to3` script. The `setup.py` has been refactored to be more modern. * The examples on how to link into Intel VML/MKL/oneAPI now use the dynamic library. What's Numexpr? --------------- Numexpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like "3*a+4*b") are accelerated and use less memory than doing the same calculation in Python. It has multi-threaded capabilities, as well as support for Intel's MKL (Math Kernel Library), which allows an extremely fast evaluation of transcendental functions (sin, cos, tan, exp, log...) while squeezing the last drop of performance out of your multi-core processors. Look here for a some benchmarks of numexpr using MKL: https://github.com/pydata/numexpr/wiki/NumexprMKL Its only dependency is NumPy (MKL is optional), so it works well as an easy-to-deploy, easy-to-use, computational engine for projects that don't want to adopt other solutions requiring more heavy dependencies. Where I can find Numexpr? ------------------------- The project is hosted at GitHub in: https://github.com/pydata/numexpr You can get the packages from PyPI as well (but not for RC releases): http://pypi.python.org/pypi/numexpr Documentation is hosted at: http://numexpr.readthedocs.io/en/latest/ Share your experience --------------------- Let us know of any bugs, suggestions, gripes, kudos, etc. you may have. Enjoy data! -- Robert McLeod robbmcleod(a)gmail.com robert.mcleod(a)hitachi-hightech.com

1 0

Newcomer's meeting - later today (4pm UTC)!
by Melissa Mendonça 01 Dec '21

01 Dec '21

Hi all! Sorry for the late notice - our next Newcomer's Meeting is today, * July 15, at 4pm UTC.* This is an informal meeting with no agenda to ask questions, get to know other people and (hopefully) figure out ways to contribute to NumPy. Feel free to join if you are lurking around but found it hard to start contributing - we'll do our best to support you. If you wish to join on Zoom, use this link: https://zoom.us/j/6345425936 Hope to see you around! ** You can click this link to get the correct time at your timezone: https://www.timeanddate.com/worldclock/fixedtime.html?msg=NumPy+Newcomer%27… <https://www.timeanddate.com/worldclock/fixedtime.html?msg=NumPy+Newcomer%27…> *** You can add the NumPy community calendar to your google calendar by clicking this link: https://calendar.google.com/calendar /r?cid=YmVya2VsZXkuZWR1X2lla2dwaWdtMjMyamJobGRzZmIyYzJqODFjQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20 - Melissa

1 7

NumPy Development Meeting Wednesday - Triage Focus
by Sebastian Berg 01 Dec '21

01 Dec '21

Hi all, Our bi-weekly triage-focused NumPy development meeting is Wednesday, December 1st at 16:30 UTC (8:30am Pacific Time). Everyone is invited to join in and edit the work-in-progress meeting topics and notes: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg I encourage everyone to notify us of issues or PRs that you feel should be prioritized, discussed, or reviewed. Best regards Sebastian

1 0