Mailman 3 May 2022 - NumPy-Discussion

NumPy community meeting – Wednesday, March 30th, 2022 at 18:00 (6 pm) UTC
by Inessa Pawson May 10, 2022

May 10, 2022

Our next NumPy community meeting is scheduled for Wednesday, March 30th at 18:00 (6 pm) UTC. Join us via Zoom: https://berkeley.zoom.us/j/762261535 As always, everyone is welcome and encouraged to attend. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy

1 2

Automatic formatters for only changed lines
by Trevor Gross May 8, 2022

May 8, 2022

Hey all, Matti pointed out to me that discussion of an autoformatter has come up before, with the decision that an entire codebase rewrite would be undesirable (ref: https://mail.python.org/pipermail/scipy-dev/2020-January/023927.html). As a result, tools/linter.py was added to only validate the changed lines. There is better support for autoformatters that only work on changed lines now, and I'd like to propose adding one. I already have a pre-commit demo implemented here, with an example run: https://github.com/numpy/numpy/pull/21449. It applies Black to changed python file lines plus flake8 to validate them (changed lines only), and clang-format with the pep7 config to any changed c/c++ lines. Pre-commit has the easy interface to run on staged files (default) all files, specific files, or files changed between --from-ref and --to-ref. I don't necessarily know that this should be enforced on CI off the bat (though in the future it could be), but at least having the tool available and easily usable would help to incrementally improve codebase style, and remove the need for formatting by hand. Any thoughts? - Trevor

10 14

ndarray repr - put shape & type first
by Ilya Kamenshchikov May 7, 2022

May 7, 2022

Hello, I wanted to discuss a pain point I've experienced while debugging numpy code. When dealing with e.g. transformed image arrays or other non-trivial ndarrays in debugger, I'm swamped by a bunch of numbers in their repr that don't help me at all. What I really care about is *shape and dtype*, as they are essentially what distinguishes complete type of numpy arrays. Example of not helpful output: ``` array([[[0.49113064, 0.42102904, 0.62108876], [0.25435884, 0.18665439, 0.53790145], ``` By counting the [ brackets, I can at least get the number of dimensions, but usefulness stops here. Could repr of an array start with something like: `array(*shape=[32,32,3], dtype=float, data=*([[[0.49113064, ...)` I know one invariant that repr likes to keep is that you can post repr and it should be the initialisation of the represented object, but given that we replace long number sequence with ..., this is already not the case. Short term: Perhaps a plugin / change to IDE could do what I ask, please let me know if something like this already exists :) Long term: I think that behavior will be more useful than what is currently there. It could also be conditionally there once we anyhow need to use ... for too long arrays. Best Regards, -- Ilya Kamen

2 1

Numpy-related jobs, grants, etc
by Renato Fabbri May 4, 2022

May 4, 2022

This is not only for me, but a question for the community in general. Do we have an open knowledge base of budget (interested companies or individuals, public funding) for developing numpy-related packages? Explaining a bit with a specific case... In the last few years, I have many times thought about the possibility of working with Numpy: finding payment for developing libraries using Numpy or even Numpy internals. Especially, I have written a music python library (called 'music' on PyPI). It holds many useful things, equal-loudness contours, musical notes, localization, HRTF, LUT, singing, glissandi, tremolos & vibratos with transitions, etc etc. I believe it is something well done, which involved scientific research and over a decade of experience with the algorithms, but it still lacks a better organization of the functionalities, better documentation, and better maintenance. To reach that point, one would probably have to be paid to take the necessary time. Beyond that, I know it is useful for making art/music, teaching, learning music, learning music-related signal processing and programming, etc. Maybe the most important point: this is probably the case with other libraries/projects. ((( On the other side, we as a company (here at Modena/Italy) may look into (public, private, own) funding to develop Numpy (internals, libraries). We would also enjoy developing Numpy-related libraries (internals?) if someone needs or wants them. ))) Should this be posted in another list/forum? Any thoughts? R. -- Renato Fabbri --

2 1

Discussion: History & Possible Deprecation/Removal of numpy.linalg.{eig(), eigvals()}?
by Leo Fang May 4, 2022

May 4, 2022

Hi, I am catching up with an assigned task that I slipped. I am wondering a few things about the numpy.linalg.eig() API (eigvals() is also included in this discussion, but for simplicity let's focus on eig()). AFAIU the purpose of eig() is for handling general eigen-systems which have left and right eigenvectors (that could differ beyond transpose/conjugate). For this, I have two questions: 1. What's the history of NumPy adding this eig() routine? My guess would be NumPy added it because Matlab had it, but I couldn't tell from a quick commit search. 2. Would it be possible to deprecate/remove this API? From past discussions with users and developers, I found: - numpy.linalg.eig() only returns the right eigenvectors, so it's not very useful: * When the left eigenvectors are not needed --- which is the majority of the use cases --- eigh() is the right routine, but most users (especially those transitioning from Matlab) don't know it's there * When the left eigenvectors are needed, users have to use scipy.linalg.eig() anyway - A general support for eig() could be challenging (numerical issues), and doing SVDs instead is usually the better way out (in terms of both numerical stability and mathematical sense). Thanks, Leo

3 2

Types in pure Python sources
by Sergei Lebedev May 3, 2022

May 3, 2022

Hi everyone, I've noticed that NumPy currently users separate type stub files for specifying types for both pure Python and native modules. For example the (untyped) implementation of np.core._asarray is in [1], but the types are in [2]. This works fine for type checkers, and if anything, improves type checking performance, because type stubs are much smaller than their pure Python counterparts. It does, however, create problems for IDEs which now have to either work with untyped APIs or manually merge them with type stubs to provide better coding assistance. Is there a reason for type stubs being separate (other than historical [3])? If not, how should I go about starting a conversation upstream about this? Sergei [1]: https://github.com/numpy/numpy/blob/main/numpy/core/_asarray.py [2]: https://github.com/numpy/numpy/blob/main/numpy/core/_asarray.pyi [3]: https://github.com/numpy/numpy-stubs

3 4

NumPy Development Meeting Wednesday - Triage Focus
by Sebastian Berg May 3, 2022

May 3, 2022

Hi all, Our bi-weekly triage-focused NumPy development meeting is Wednesday, May 4th at 16:00 UTC (9:00am Pacific Time). Everyone is invited to join in and edit the work-in-progress meeting topics and notes: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg I encourage everyone to notify us of issues or PRs that you feel should be prioritized, discussed, or reviewed. A calendar to can be found at: https://scientific-python.org/calendars/ please make sure to not import the static `ics` but the url. Best regards Sebastian

2 1