Mailman 3 August 2021 - NumPy-Discussion

NumPy 1.21.2 released
by Charles R Harris 15 Aug '21

15 Aug '21

Hi All, On behalf of the NumPy team I am pleased to announce the release of NumPy 1.21.2. NumPy 1.21.2 is a maintenance release that fixes bugs discovered after 1.21.1. It also provides 64 bit manylinux Python 3.10.0rc1 wheels for downstream testing. Note that Python 3.10 is not yet final. There is also preliminary support for Windows on ARM64 builds, but there is no OpenBLAS for that platform and no wheels are available. The Python versions supported for this release are 3.7-3.9. The 1.21.2 release is compatible with Python 3.10.0rc1 and Python 3.10 will be officially supported after it is released. The previous problems with gcc-11.1 have been fixed by gcc-11.2, check your version if you are using gcc-11. Wheels can be downloaded from PyPI <https://pypi.org/project/numpy/1.21.2/>; source archives, release notes, and wheel hashes are available on Github <https://github.com/numpy/numpy/releases/tag/v1.21.2>. Linux users will need pip >= 0.19.3 in order to install manylinux2010 and manylinux2014 wheels. *Contributors* A total of 10 people contributed to this release. People with a "+" by their names contributed a patch for the first time. - Bas van Beek - Carl Johnsen + - Charles Harris - Gwyn Ciesla + - Matthieu Dartiailh - Matti Picus - Niyas Sait + - Ralf Gommers - Sayed Adel - Sebastian Berg *Pull requests merged* A total of 18 pull requests were merged for this release. - #19497: MAINT: set Python version for 1.21.x to ``<3.11`` - #19533: BUG: Fix an issue wherein importing ``numpy.typing`` could raise - #19646: MAINT: Update Cython version for Python 3.10. - #19648: TST: Bump the python 3.10 test version from beta4 to rc1 - #19651: TST: avoid distutils.sysconfig in runtests.py - #19652: MAINT: add missing dunder method to nditer type hints - #19656: BLD, SIMD: Fix testing extra checks when ``-Werror`` isn't applicable... - #19657: BUG: Remove logical object ufuncs with bool output - #19658: MAINT: Include .coveragerc in source distributions to support... - #19659: BUG: Fix bad write in masked iterator output copy paths - #19660: ENH: Add support for windows on arm targets - #19661: BUG: add base to templated arguments for platlib - #19662: BUG,DEP: Non-default UFunc signature/dtype usage should be deprecated - #19666: MAINT: Add Python 3.10 to supported versions. - #19668: TST,BUG: Sanitize path-separators when running ``runtest.py`` - #19671: BLD: load extra flags when checking for libflame - #19676: BLD: update circleCI docker image - #19677: REL: Prepare for 1.21.2 release. Cheers Charles Harris

1 0

Drop LGTM testing.
by Charles R Harris 14 Aug '21

14 Aug '21

Hi All, LGTM on github uses Python 3.7, which causes a problem if we drop 3.7 support. LGTM is nice for pointing to possible code improvements, but we mostly ignore it anyway. There are probably standalone code analysers that would serve our needs as well, so dropping it seems the easiest way forward. Thoughts? Chuck

3 4

Idea: Simplify passing of shared memory info for ndarrays
by Randall Pittman 13 Aug '21

13 Aug '21

Hi! First time poster to the list, long time happy user of NumPy (thank you, devs!!) I created a bit of a wrapper for using `ndarray` with Py 3.8+ `multiprocessing.shared_memory`. I really like the ability to use `SharedMemory` with `ndarray` via the `buffer` arg. (as in https://docs.python.org/3/library/multiprocessing.shared_memory.html#multip…) However, it seemed a bit clunky to worry about passing the SharedMemory object or name, dtype, and shape to reconstruct the `ndarray` in other processes. I came up with this SharedNDArray class that encapsulates that information and provides an ephemeral `ndarray` interface: https://gitlab.com/osu-nrsg/shared-ndarray2. It's especially meant for use in a `SharedMemoryManager` context manager. I shared it on r/Python and someone suggested I share it here for your input (and maybe even integration into NumPy rather than the lone-wolf project it currently is). I'm happy to receive any suggestions/input/criticism. As an aside, I discovered that it's not at all trivial and, at the moment, not technically possible to define all the possible typing overloads of ndarray `__setitem__` and `__getitem__`. Stll, I enjoyed making use of np.typing.NDArray. Thanks for your consideration, Randy

1 0

Dropping Python 3.7 for NumPy 1.22
by Charles R Harris 13 Aug '21

13 Aug '21

Hi All, This is to propose dropping Python 3.7 for NumPy 1.22. Doing so will allow merging the array API standard <https://github.com/numpy/numpy/pull/18585> (keyword only arguments), simplify removing import time compiles <https://github.com/numpy/numpy/pull/19638>, allow making 64 bit pickles the default, and bring annotations closer to current. NEP 29 <https://numpy.org/neps/nep-0029-deprecation_policy.html> suggests Dec 26, 2021 for the drop date, which is close to the likely 1.22 release date, but given the advantages of dropping 3.7 I think there are good reasons to cheat by a week or two if needed. Downstream projects releasing after 1.22 will probably want to drop Python 3.7 anyway, as they will be past the deadline. Thoughts? Chuck

2 1

Matti Picus and Sebastian Berg added to org-admin
by Charles R Harris 12 Aug '21

12 Aug '21

Hi All, Matti Picus and Sebastian Berg have been added to the org-admin team on github. That will allow them to help maintain the github site going forward. Chuck

1 0

Newcomer's Meeting Aug 12: special accessibility mini-sprint! (Part 2)
by Melissa Mendonça 11 Aug '21

11 Aug '21

Hey folks! This week we are repeating our low-code accessibility mini-sprint on August 12, at 4pm UTC. Last time we had a few people participate - you can see the work they did here: https://github.com/melissawm/numpy/pull/27 If you are interested, we'd be happy to help you get started. As usual, all are welcome - feel free to join even if you just want to observe! To join on Zoom, use this link: https://zoom.us/j/6345425936 Hope to see you around! - Melissa ** You can click this link to get the correct time at your timezone: https://www.timeanddate.com/worldclock/fixedtime.html?msg=NumPy+Newcomer%27… *** You can add the NumPy community calendar to your Google calendar by clicking this link: https://calendar.google.com/calendar /r?cid=YmVya2VsZXkuZWR1X2lla2dwaWdtMjMyamJobGRzZmIyYzJqODFjQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20 ---------- Forwarded message --------- De: Melissa Mendonça <melissawm(a)gmail.com> Date: seg, 26 de jul de 2021 17:03 Subject: Newcomer's Meeting July 29: special accessibility mini-sprint! To: Discussion of Numerical Python <numpy-discussion(a)python.org> Hello, folks! This week we have our planned Newcomer's meeting on July 29, at 8PM UTC. This time, we are teaming up with Tony Fast and Isabela Presedo-Floyd to propose a low-code mini-sprint focusing on accessibility. This is a great opportunity to make your first contribution to NumPy! We will be focusing on writing alt-text for images in our documentation. Alt-text provides a textual alternative to non-text content in web pages, and as cited in this WebAIM document [1]: "Alternative text serves several functions: - It is read by screen readers in place of images allowing the content and function of the image to be accessible to those with visual or certain cognitive disabilities. - It is displayed in place of the image in browsers if the image file is not loaded or when the user has chosen not to view images. - It provides a semantic meaning and description to images which can be read by search engines or be used to later determine the content of the image from page context alone." You can find some more information on how to add alt-text in [2]. As usual, all are welcome - feel free to join even if you just want to observe! To join on Zoom, use this link: https://zoom.us/j/6345425936 Hope to see you around! - Melissa [1] https://webaim.org/techniques/alttext/ [2] https://www.w3.org/WAI/tutorials/images/decision-tree/

1 0

NumPy Python 3.10.0rc1 wheels.
by Charles R Harris 10 Aug '21

10 Aug '21

Hi All, There are now NumPy Python 3.10.0rc1 wheels available for 64 bit Linux on Intel. You can install them from the nightly builds using "pip install -i https://pypi.anaconda.org/scipy-wheels-nightly/simple numpy". Test away. Cheers, Charles Harris

1 0

NumPy Development Meeting Wednesday - Triage Focus
by Sebastian Berg 10 Aug '21

10 Aug '21

Hi all, Our bi-weekly triage-focused NumPy development meeting is Wednesday, August 11th at 9 am Pacific Time (16:00 UTC). Everyone is invited to join in and edit the work-in-progress meeting topics and notes: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg I encourage everyone to notify us of issues or PRs that you feel should be prioritized, discussed, or reviewed. Best regards Sebastian

1 0

reducing effort spent on wheel builds?
by Ralf Gommers 09 Aug '21

09 Aug '21

Hey all, This whole thread is quite interesting: https://twitter.com/zooba/status/1415440484181417998. Given how much effort we are spending on really niche wheel builds, I’m wondering if we should just draw a line somewhere: - we do what we do now for the main platforms: Windows, Linux (x86, aarch64), macOS, *but*: - no wheels for ppc64le - no wheels for Alpine Linux - no wheels for PyPy - no wheels for Raspberry Pi, AIX or whatever other niche thing comes next. - drop 32-bit Linux in case it is becoming an annoyance. This is not an actual proposal (yet) and I should sleep on this some more, but I've seen Chuck and Matti burn a lot of time on the numpy-wheels repo again recently, and I've done the same for SciPy. The situation is not very sustainable and needs a rethink. The current recipe is "someone who cares about a platform writes a PEP, then pip/wheel add a platform tag for it (very little work), and then the maintainers of each Python package are now responsible for wheel builds (a ton of work)". Most of these platforms have package managers, which are all more capable than pip et al., and if they don't then wheels can be hosted elsewhere (example: https://www.piwheels.org/). And then there's Conda, Nix, Spack, etc. too of course. Drawing a line somewhere distributes the workload, where packagers who care about some platform and have better tools at hand can do the packaging, and maintainers can go do something with more impact like write new code or review PRs. <end of brainwave> Cheers, Ralf

8 14

Proposal for adding bit_count
by Ganesh Kathiresan 04 Aug '21

04 Aug '21

Hi All, I am working <https://github.com/numpy/numpy/pull/19355> on a new UFunc, ` bit_count <https://en.wikipedia.org/wiki/Hamming_weight>` (popcount in other languages) that aims to count the number of 1-bits in the absolute value of an Integer. Implementation ---------------------------------- The primary reference for the implementation is CountBitsSetParallel <http://graphics.stanford.edu/~seander/bithacks.html#CountBitsSetParallel>. Here we take 12 operations to achieve the result which is the same as the lookup table method but does not suffer from memory issues or cache misses. The implementation is aimed at unsigned integers, absolute value of signed integers and objects that support the operation. Usage -------------- >>> np.bit_count(1023) 10 >>> a = np.array([2**i - 1 for i in range(16)]) >>> np.bit_count(a) array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]) >>> np.int32(1023).bit_count() 10 Notes ------------- 1. Python has included this method here <https://github.com/python/cpython/commit/8bd216dfede9cb2d5bedb67f20a30c9984…> (3.10+). Tracking issue <https://bugs.python.org/issue29882> 2. NumPy tracking issue <https://github.com/numpy/numpy/issues/16325> 3. Interesting read <https://archive.org/details/dr_dobbs_journal_vol_08/page/n185/mode/2up> on how we get the magic number. Needed a bit of digging :) Please let us know what you think about the implementation and where we can improve in terms of performance or interface. Regards, Ganesh

3 4