Hi all, just a note that I merged the PR with the following semantics: A new `kind` keyword-only argument: * `kind=None` uses a memory bound based heuristic to decide which method to use * `kind="table"` uses the new approach (integer arrays only) * `kind="sort"` forces the old behavior The new documentation is available at: https://numpy.org/devdocs/reference/generated/numpy.in1d.html It seems this addition should be useful in many cases, but if you have any concern about the choice of API please comment! Cheers, Sebastian On Thu, 2022-06-16 at 06:08 -0700, Sebastian Berg wrote:
Hi all,
there is a PR to add a faster path to `np.isin`, that uses a look-up- table for all the elements that are included in the haystack (`test_elements`):
https://github.com/numpy/numpy/pull/12065/files
Such a table means that the memory overhead can be very significant, but the speedup as well, so there was the idea of adding an option to pick which version is used.
The current documentation for this new `method` keyword argument would be. So the main questions are:
* Is there any concern about adding such a new kwarg? * Is `method` the best name? Sorts uses `kind` which may also be good
There is also the smaller question of what heuristic 'auto' would use, but that can be tweaked at any time.
``` method : {'auto', 'sort', 'dictionary'}, optional The algorithm to use. This will not affect the final result, but will affect the speed. Default is 'auto'.
- If 'sort', will use a mergesort-based approach. This will have a memory usage of roughly 6 times the sum of the sizes of `ar1` and `ar2`, not accounting for size of dtypes. - If 'dictionary', will use a key-dictionary approach similar to a counting sort. This is only available for boolean and integer arrays. This will have a memory usage of the size of `ar1` plus the max-min value of `ar2`. This tends to be the faster method if the following formula is true: `log10(len(ar2)) > (log10(max(ar2)-min(ar2)) - 2.27) / 0.927`, but may use greater memory. - If 'auto', will automatically choose the method which is expected to perform the fastest, using the above formula. For larger sizes or smaller range, 'dictionary' is chosen. For larger range or smaller sizes, 'sort' is chosen.` ```
Cheers,
Sebastian _______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net