Mailman 3 May 2022 - NumPy-Discussion

ENH: Support Random Unit Vector
by saroad2＠gmail.com May 17, 2022

May 17, 2022

### Proposed new feature or change: Description ======== Often you want to randomize a "direction" that doesn't have "size". This can be useful when: * You want to randomize the direction of an NPC walk developing in a 2D or 3D games, where its speed is a predefined constant. * You're developing particles simulation with all particles moving in random directions * etc. This random direction should be an `n`th dimensional unit vector which is randomize **uniformly** from the unit sphere. That means that sections of the unit sphere with equal areas should have the same chance of getting a vector from. Implementation =========== If one wants to randomize `n`th dimensional direction uniformly, one must do the following: 1. Generate `n` components of the vector using the standard normal distribution 2. Find the norm the generated vector 3. Divide the vector by its norm In simple Python code: ```python import numpy as np def sphere_uniform(n: int): vec = np.random.normal(size=n) norm = np.sqrt(np.sum(vec ** 2)) return vec / norm ``` This implementation suggestion is based on [this blog post](https://towardsdatascience.com/the-best-way-to-pick-a-unit-vector-7bd…. I'm not sure if there's a faster way of implementing it using C++ and Cython, but I'm confident that someone here will know. Why Implement It in Numpy? =================== I believe that random unit vectors are common enough to be a part of Numpy. Scientists, gamers and engineers use random directions all the time, and I believe there is no reason why Numpy won't provide this ability.

2 1

NumPy Development Meeting Wednesday - Triage Focus
by Sebastian Berg May 17, 2022

May 17, 2022

Hi all, Our bi-weekly triage-focused NumPy development meeting is Wednesday, May 18th at 16:00 UTC (9:00am Pacific Time). Everyone is invited to join in and edit the work-in-progress meeting topics and notes: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg I encourage everyone to notify us of issues or PRs that you feel should be prioritized, discussed, or reviewed. A calendar to can be found at: https://scientific-python.org/calendars/ please make sure to not import the static `ics` but the url. Best regards Sebastian

1 0

Interesting security project
by Charles R Harris May 14, 2022

May 14, 2022

Hi All, There is a project underway to secure open source software <https://venturebeat.com/2022/05/13/how-much-will-it-cost-to-secure-open-sou…>. This doesn't affect us (yet), but is of interest as it shows the growing importance of open source. Chuck

1 0

Fixing/changing structured dtype casting safety, promotion, and comparisons
by Sebastian Berg May 11, 2022

May 11, 2022

Hi all, This was probably announced before, but I wanted to mention it again, with https://github.com/numpy/numpy/pull/19226 as the last step the casting safety, promotion, and comparisons of structured (void) dtypes will be updated to fix some bugs and fit better with changes that happened a few years ago already. The most interesting change is probably to comparisons: It is now possible to compare sturctured dtypes when included dtypes mismatch: arr1 = np.array([1, 2, 3], dtype="i,i") arr2 = np.array([1, 2, 1], dtype="i,f8") # note the f8 arr1 == arr2 # Will now return [True, True, False] while previously this returned `False` with a `FutureWarning` (although that warning was not quite correct. Comparisons will now always generally succeed if the two dtypes can be promoted. The other change to `==` and `!=` is that in cases where promotion is not possible, a `TypeError` will now be given for structured dtype comparisons. (Previously, also `False` with a `FutureWarning`) These changes align with the changes in promotion for structured dtypes. Structured dtypes will now correctly promote all included dtypes if the number of fields, names, and titles match exactly. Further, structured dtypes will remove any unnecessary empty space when promoted. The result is considered "canonical" with the removed padding. In alignment with the above changes to the promotion logic, the casting safety has been updated: * "equiv" enforces matching names and titles. The itemsize is allowed to differ due to padding. * "safe" allows mismatching field names and titles * The cast safety is limited by the cast safety of each included field. * The order of fields is used to decide cast safety of each individual field. Previously, the field names were used and only unsafe casts were possible when names mismatched. The main important change here is that name mismatches are now considered "safe" casts. The cast itself already worked using the field order. Cheers, Sebastian

1 0

[Feature Request] Add alias of np.concatenate as np.concat
by Iordanis Fostiropoulos May 10, 2022

May 10, 2022

In regard to Feature Request: https://github.com/numpy/numpy/issues/16469 It was suggested to sent to the mailing list. I think I can make a strong point as to why the support for this naming convention would make sense. Such as it would follow other frameworks that often work alongside numpy such as tensorflow. For backward compatibility, it can simply be an alias to np.concatenate I often convert portions of code from tf to np, it is as simple as changing the base module from tf to np. e.g. np.expand_dims -> tf.expand_dims. This is done either in debugging (e.g. converting tf to np without eager execution to debug portion of the code), or during prototyping, e.g. develop in numpy and convert in tf. I find myself more than at one occasion to getting syntax errors because of this particular function np.concatenate. It is unnecessarily long. I imagine there are more people that also run into the same problems. Pandas uses concat (torch on the other extreme uses simply cat, which I don't think is as descriptive).

7 6

NumPy community meeting – Wednesday, March 30th, 2022 at 18:00 (6 pm) UTC
by Inessa Pawson May 9, 2022

May 9, 2022

Our next NumPy community meeting is scheduled for Wednesday, March 30th at 18:00 (6 pm) UTC. Join us via Zoom: https://berkeley.zoom.us/j/762261535 As always, everyone is welcome and encouraged to attend. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy

1 2

Automatic formatters for only changed lines
by Trevor Gross May 8, 2022

May 8, 2022

Hey all, Matti pointed out to me that discussion of an autoformatter has come up before, with the decision that an entire codebase rewrite would be undesirable (ref: https://mail.python.org/pipermail/scipy-dev/2020-January/023927.html). As a result, tools/linter.py was added to only validate the changed lines. There is better support for autoformatters that only work on changed lines now, and I'd like to propose adding one. I already have a pre-commit demo implemented here, with an example run: https://github.com/numpy/numpy/pull/21449. It applies Black to changed python file lines plus flake8 to validate them (changed lines only), and clang-format with the pep7 config to any changed c/c++ lines. Pre-commit has the easy interface to run on staged files (default) all files, specific files, or files changed between --from-ref and --to-ref. I don't necessarily know that this should be enforced on CI off the bat (though in the future it could be), but at least having the tool available and easily usable would help to incrementally improve codebase style, and remove the need for formatting by hand. Any thoughts? - Trevor

10 14

ndarray repr - put shape & type first
by Ilya Kamenshchikov May 7, 2022

May 7, 2022

Hello, I wanted to discuss a pain point I've experienced while debugging numpy code. When dealing with e.g. transformed image arrays or other non-trivial ndarrays in debugger, I'm swamped by a bunch of numbers in their repr that don't help me at all. What I really care about is *shape and dtype*, as they are essentially what distinguishes complete type of numpy arrays. Example of not helpful output: ``` array([[[0.49113064, 0.42102904, 0.62108876], [0.25435884, 0.18665439, 0.53790145], ``` By counting the [ brackets, I can at least get the number of dimensions, but usefulness stops here. Could repr of an array start with something like: `array(*shape=[32,32,3], dtype=float, data=*([[[0.49113064, ...)` I know one invariant that repr likes to keep is that you can post repr and it should be the initialisation of the represented object, but given that we replace long number sequence with ..., this is already not the case. Short term: Perhaps a plugin / change to IDE could do what I ask, please let me know if something like this already exists :) Long term: I think that behavior will be more useful than what is currently there. It could also be conditionally there once we anyhow need to use ... for too long arrays. Best Regards, -- Ilya Kamen

2 1

Numpy-related jobs, grants, etc
by Renato Fabbri May 4, 2022

May 4, 2022

This is not only for me, but a question for the community in general. Do we have an open knowledge base of budget (interested companies or individuals, public funding) for developing numpy-related packages? Explaining a bit with a specific case... In the last few years, I have many times thought about the possibility of working with Numpy: finding payment for developing libraries using Numpy or even Numpy internals. Especially, I have written a music python library (called 'music' on PyPI). It holds many useful things, equal-loudness contours, musical notes, localization, HRTF, LUT, singing, glissandi, tremolos & vibratos with transitions, etc etc. I believe it is something well done, which involved scientific research and over a decade of experience with the algorithms, but it still lacks a better organization of the functionalities, better documentation, and better maintenance. To reach that point, one would probably have to be paid to take the necessary time. Beyond that, I know it is useful for making art/music, teaching, learning music, learning music-related signal processing and programming, etc. Maybe the most important point: this is probably the case with other libraries/projects. ((( On the other side, we as a company (here at Modena/Italy) may look into (public, private, own) funding to develop Numpy (internals, libraries). We would also enjoy developing Numpy-related libraries (internals?) if someone needs or wants them. ))) Should this be posted in another list/forum? Any thoughts? R. -- Renato Fabbri --

2 1

Discussion: History & Possible Deprecation/Removal of numpy.linalg.{eig(), eigvals()}?
by Leo Fang May 3, 2022

May 3, 2022

Hi, I am catching up with an assigned task that I slipped. I am wondering a few things about the numpy.linalg.eig() API (eigvals() is also included in this discussion, but for simplicity let's focus on eig()). AFAIU the purpose of eig() is for handling general eigen-systems which have left and right eigenvectors (that could differ beyond transpose/conjugate). For this, I have two questions: 1. What's the history of NumPy adding this eig() routine? My guess would be NumPy added it because Matlab had it, but I couldn't tell from a quick commit search. 2. Would it be possible to deprecate/remove this API? From past discussions with users and developers, I found: - numpy.linalg.eig() only returns the right eigenvectors, so it's not very useful: * When the left eigenvectors are not needed --- which is the majority of the use cases --- eigh() is the right routine, but most users (especially those transitioning from Matlab) don't know it's there * When the left eigenvectors are needed, users have to use scipy.linalg.eig() anyway - A general support for eig() could be challenging (numerical issues), and doing SVDs instead is usually the better way out (in terms of both numerical stability and mathematical sense). Thanks, Leo

3 2

Types in pure Python sources
by Sergei Lebedev May 3, 2022

May 3, 2022

Hi everyone, I've noticed that NumPy currently users separate type stub files for specifying types for both pure Python and native modules. For example the (untyped) implementation of np.core._asarray is in [1], but the types are in [2]. This works fine for type checkers, and if anything, improves type checking performance, because type stubs are much smaller than their pure Python counterparts. It does, however, create problems for IDEs which now have to either work with untyped APIs or manually merge them with type stubs to provide better coding assistance. Is there a reason for type stubs being separate (other than historical [3])? If not, how should I go about starting a conversation upstream about this? Sergei [1]: https://github.com/numpy/numpy/blob/main/numpy/core/_asarray.py [2]: https://github.com/numpy/numpy/blob/main/numpy/core/_asarray.pyi [3]: https://github.com/numpy/numpy-stubs

3 4

NumPy Development Meeting Wednesday - Triage Focus
by Sebastian Berg May 3, 2022

May 3, 2022

Hi all, Our bi-weekly triage-focused NumPy development meeting is Wednesday, May 4th at 16:00 UTC (9:00am Pacific Time). Everyone is invited to join in and edit the work-in-progress meeting topics and notes: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg I encourage everyone to notify us of issues or PRs that you feel should be prioritized, discussed, or reviewed. A calendar to can be found at: https://scientific-python.org/calendars/ please make sure to not import the static `ics` but the url. Best regards Sebastian

2 1