Mailman 3 May 2024 - NumPy-Discussion

NumPy 2.0.0rc2 released
by Charles R Harris May 12, 2024

May 12, 2024

Hi All On behalf of the NumPy team, I'm pleased to announce the release of NumPy 2.0.0rc2. NumPy 2.0.0 is the first major release since 2006. It is the result of 11 months of development since the last feature release and is the work of 198 contributors spread over 1041 pull requests. It contains a large number of exciting new features as well as changes to both the Python and C APIs. This major release includes breaking changes that could not happen in a regular minor (feature) release - including an ABI break, changes to type promotion rules, and API changes which may not have been emitting deprecation warnings in 1.26.x. Key documents related to how to adapt to changes in NumPy 2.0 include: - The release notes on Github <https://github.com/numpy/numpy/releases/tag/v2.0.0rc1> - The numpy-2-migration-guide <https://numpy.org/devdocs/numpy_2_0_migration_guide.html> - The numpy 2.0-specific advice in for downstream package authors <https://numpy.org/devdocs/dev/depending_on_numpy.html> Highlights include: - A new variable-length string dtype, StringDType, and a new `numpy.strings` namespace with performant ufuncs for string operations. - Support for float32 and long double in all `numpy.fft` functions. - Support for the array API standard in the main numpy namespace. - MacOS Accelerate support and binary wheels for macOS >=14. - A new tracing and introspection API. - Performance improvements. - Python API improvements. - C API improvements. This release supports Python 3.9-3.12. Wheels can be downloaded from PyPI <https://pypi.org/project/numpy/2.0.0rc2>; source archives, release notes, and wheel hashes are available on Github <https://github.com/numpy/numpy/releases/tag/v2.0.0rc2>. *Contributors* A total of 198 people contributed to this release. People with a "+" by their names contributed a patch for the first time. - @Algorithmist-Girl + - @DWesl - @Illviljan - @ellaella12 + - @liang3zy22 + - @matoro + - @mcp292 + - @mykykh + - @pojaghi + - @pratiklp00 + - @stefan6419846 + - @undermyumbrella1 + - Aaron Meurer - Aditi Saluja + - Adrin Jalali + - Agriya Khetarpal + - Albert Steppi + - Alex Cabrera + - Alexander Grund - Andrea Bianchi + - Andreas Florath + - Andrew Ardill + - Andrew Ho + - Andrew Nelson - Andrey Rybakov + - Ankur Singh + - Anton Prosekin + - Antony Lee - Bas van Beek - Ben Woodruff + - Bharat Raghunathan - Bhavya Alekhya + - Brandon Smith + - Brian Walshe + - Brigitta Sipőcz - Brock Mendel - Carl Meyer + - Charles Bousseau + - Charles Harris - Chris Sidebottom - Christian Lorentzen - Christian Veenhuis - Christoph Reiter - Christopher Sidebottom - Clément Robert - Cédric Hannotier - D.J. Ramones + - DanShatford + - Daniel Li + - Daniel Vanzo - Daval Parmar - Developer-Ecosystem-Engineering - Dhruv Rawat + - Dimitri Papadopoulos Orfanos - Edward E - Edward Yang + - Eisuke Kawashima + - Eliah Kagan + - Élie Goudout + - Elliott Sales de Andrade - Emil Olszewski + - Emily Hunt + - Éric Piel + - Eric Wieser - Even Rouault + - Evgeni Burovski - Filipe Laíns + - Francisco Sousa + - Ganesh Kathiresan - Gonzalo Tornaría + - Hans Meine - Heberto Mayorquin + - Heinz-Alexander Fuetterer + - Hood Chatham - Hugo van Kemenade - Ivan A. Melnikov + - Jacob M. Casey + - Jake Lishman + - Jake VanderPlas - James Oliver + - Jan Wassenberg + - Janukan Sivajeyan + - Johann Rohwer + - Johannes Kaisinger + - John Muradeli + - Joris Van den Bossche - Kai Striega - Kevin Sheppard - Kevin Wu + - Khawaja Junaid + - Kit Lee + - Kristian Minchev + - Kristoffer Pedersen + - Kuan-Wei Chiu + - Lane Votapka + - Larry Bradley - Leo Singer - Liang Yan + - Linus Sommer + - Logan Thomas - Lucas Colley + - Lukas Geiger - Lysandros Nikolaou + - Maanas Arora + - Maharshi Basu + - Mahder Gebremedhin + - Marcel Bargull + - Mark Mentovai + - Mark Ryan + - Marten Henric van Kerkwijk + - Marten van Kerkwijk - Mateusz Sokół - Matt Haberland - Matthew Barber - Matthias Bussonnier - Matthias Koeppe - Matthias Schaufelberger + - Matti Picus - Maxwell Aladago - Maya Anderson + - Melissa Weber Mendonça - Meng Xiangzhuo + - Michael Kiffer - Miki Watanabe (渡邉美希) - Milan Curcic + - Miles Cranmer - Miro Hrončok + - Mohamed E. BRIKI + - Mohaned Qunaibit + - Mohit Kumar + - Muhammed Muhsin + - Mukulika Pahari - Munira Alduraibi + - Namami Shanker - Nathan Goldbaum - Nyakku Shigure + - Ola x Nilsson + - Olivier Mattelaer + - Omid Rajaei - Pablo Losada + - Pamphile Roy - Paul Reece + - Pedro Kaj Kjellerup Nacht + - Peiyuan Liu + - Peter Hawkins - Pierre - Pieter Eendebak - Quentin Barthélemy + - Raghuveer Devulapalli - Ralf Gommers - Randy Eckenrode + - Raquel Braunschweig + - Richard Howe + - Robert Kern - Rohit Goswami - Romain Geissler + - Ronald van Elburg + - Ross Barnowski - Sam James + - Sam Van Kooten + - Samuel Albanie + - Sarah Wang + - Sarah Zwiep + - Sarah-Yifei-Wang + - Sarthak Dawar + - Sayantika Banik - Sayed Adel - Sean Cheah + - Sebastian Berg - Serge Guelton - Shalini Roy + - Shen Zhou - Shubhal Gupta + - Stefan van der Walt - Stefano Rivera + - Takumasa N. + - Taras Tsugrii - Thomas A Caswell - Thomas Grainger + - Thomas Li - Tim Hoffmann - Tim Paine + - Trey Woodlief + - Tyler Reddy - Victor Tang + - Vladimir Fokow + - Warren Weckesser - Warrick Ball + - Will Ayd - William Andrea + - William Ayd + - Yash Pethe + - Yuki K - Zach Brugh + - Zach Rottman + - Zolisa Bleki - mgunyho + - msavinash + Cheers, Charles Harris

1 0

next NumPy triage meeting - May 15th, 2024 at 7 pm UTC
by Inessa Pawson May 12, 2024

May 12, 2024

The next NumPy triage meeting will be held this Wednesday, May 15th at 7pm UTC. This is a meeting where we synchronously triage prioritized PRs and issues. Join us via Zoom: https://numfocus-org.zoom.us/j/82096749952?pwd=MW9oUmtKQ1c3a2gydGk1RTdYUUVX… . Everyone is welcome to attend and contribute to a conversation. Please notify us of issues or PRs that you’d like to have reviewed by adding a GitHub link to them in the meeting agenda: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg. -- Cheers, Inessa Inessa Pawson GitHub: inessapawson

1 0

New feature
by Alexei Lisitsa May 11, 2024

May 11, 2024

I created an extension to numpy called numpy_list that allows to create numpy array from predefined lists for now it supports arithmetic and geometric progressions You can see info there https://github.com/alessionuovo/mionumpy/tree/list_operations/numpy_extensi… Thanks Alexei

2 4

Add context management for np.seterr
by hjenryin＠outlook.com May 9, 2024

May 9, 2024

The current way (according to 1.26 doc) of setting and resetting error is ``` old_settings = np.seterr(all='ignore') #seterr to known value np.seterr(over='raise') {'divide': 'ignore', 'over': 'ignore', 'under': 'ignore', 'invalid': 'ignore'} np.seterr(**old_settings) # reset to default ``` This may be tedious and not elegant when we need to suppress the error for some certain lines, for example, `np.nan_to_num(a/b) ` as we need to suppress divide here. I think it would be way more elegant to use `with` statement here, which should be able to be implemented with some simple changes. An ideal result would be like: ``` with np.seterr(divide='ignore'): np.nan_to_num(a/b) # no warning a/0 # still warn ```

2 1

numpy on iOS
by Vitalii May 8, 2024

May 8, 2024

Hi everyone, I wonder if anyone has successful experience in building numpy with native libs for iOS distribution. Please, share your knowledge. Thank you, Vitalii

3 3

Should we add a NumPy default protocol version for np.save?
by Chunqing Shan May 7, 2024

May 7, 2024

Currently, when NumPy saves data using pickle, it hard-coded the protocol version to 3, which was the default value from Python 3.0 to Python 3.7. However, since Python 3.7 has reached its end-of-life (EOL), there are no actively maintained Python versions that default to using pickle protocol 3. By allowing NumPy to use the default pickle protocol version, objects larger than 4GB can be pickled, resolving the issue described in #26224 [1]. Although this new protocol version is incompatible with Python 3.3 and earlier versions, Python 3.3 has long reached its EOL. Therefore, there is no need to force pickle protocol 3 when saving data using pickle. One of the reasons for having a hard-coded version is that upstream Python may bump the default version to a new version that will break the compatibility between two supported versions of NumPy. This scenario won't happen as Python core developers promised that it's a policy to only use the protocol supported by all currently supported versions of Python as default.[2][3] Another reason for having a different NumPy default version of pickle protocol is that because NumPy does not support every old Python version, we can actually bump the default version earlier than upstream Python to get the performance gains. I have done some experiments[4] that show while pickle protocol 4 improved about 4% for overall np.save performance, protocol 5 did not have that kind of performance improvement. Therefore, I'm not sure whether it is worth it to add a NumPy default version of pickle protocol. Protocol 5 use out-of-band buffer which may increase performance for lots of small pickles, but since np.save typical usage is saving to a file, filesystem overhead on creating lots of small files is the main bottleneck. There are two possible solutions to this problem. One is to just follow the upstream protocol version(which is currently 4) and expect them to bump it wisely when they should. The second is to add a pickle_protocol keyword to the numpy.save function, defaulting to the highest version that the oldest Python version NumPy supports (which is 5). Since this introduces some complexity into a pretty base interface for NumPy, I believe it needs members of NumPy to decide which way to go. I can finish the documentation (with an explanation of performance differences on different protocol versions of pickle) and add this to the release checklist of NumPy and code if we decide to add this keyword. I can also see if we can utilize pickle protocol 5's improvement with reuse of out-of-band buffers to keep things in cache and improve performance in another PR, if we decide to add NumPy's own default version of pickle protocol and set it to 5. Best regards, Chunqing Shan 1\. <https://github.com/numpy/numpy/issues/26224> 2\. <https://github.com/python/cpython/blob/5a1618a2c8c108b8c73aa9459b63f0dbd66b…> 3\. <https://github.com/numpy/numpy/pull/26388#issuecomment-2097197375> 4\. With a = np.array(np.zeros((1024, 1024, 24, 7)), dtype=object) # 1.5GB and time the following statement(/dev/shm is configured as a tmpfs) np.save('/dev/shm/a.npy', a) For protocol 3, on average 4.25s is required; for protocol 4, on average 4.07s; for protocol 5, on average 4.06s. This result is acquired on a Xeon E-2286G bare metal with 2 16GB 2667 MT/s DDR4 ECC, with Python 3.11.2 and NumPy 1.26.4.

1 0

ANN: DataLab v0.15
by Pierre Raybaut May 6, 2024

May 6, 2024

Hello everyone, We're excited to announce the release of DataLab v0.15! https://datalab-platform.com Note that the v0.15.1 release fixes a very annoying issue on macOS (screen flickering when switching from a signal/image object to another). DataLab is an open-source platform for scientific and technical data processing and visualization, based on NumPy, SciPy, ... and Qt. It's extendable with plugins, and works seamlessly as a complementary tool with your IDE (e.g. Spyder, VSCode, ...) and Project Jupyter notebooks. Once again, this release is the result of the feedback and contributions of our early adopters. We're grateful for your support and look forward to your continued insights. 💥 New Features: - Added support for large CSV/text files (when opening or importing data in Signal Panel, for example). Need to work on a GB file ? No worry, DataLab won't freeze - New auto downsampling feature for visualizing large signals -- handling 100Mpts signals without slowing down the user experience! - New MSI installer (replacing the old EXE installer for the stand-alone version on Windows) to avoid false positive detection by antiviral software 💡 You have a project or a use case in mind? (we can even write a tutorial for you...) You want to contribute? We'd love to hear from you! So please, don't hesitate to contact us or to open an issue ( https://github.com/DataLab-Platform/DataLab/issues/new/choose) on our GitHub project for any request or question. Thank you for taking the time to read my announcement. Looking forward to your thoughts. Best, Pierre Raybaut

1 0

next Documentation team meeting
by Mukulika Pahari May 6, 2024

May 6, 2024

Hi all, Our next Documentation Team meeting will happen on *Monday, December 4* at *11PM UTC*. If this time slot is inconvenient for you to join, please let me know in the replies or Slack and we will try to add another time slot. All are welcome - you don't need to already be a contributor to join. If you have questions or are curious about what we're doing, we'll be happy to meet you! If you wish to join on Zoom, use this (updated) link: https://numfocus-org.zoom.us/j/85016474448?pwd=TWEvaWJ1SklyVEpwNXUrcHV1YmFJQ... Here's the permanent hackmd document with the meeting notes (still being updated): https://hackmd.io/oB_boakvRqKR-_2jRV-Qjg Hope to see you around! Best wishes, Mukulika

2 6

NumPy Optimization Team meeting - May 6th, 5 pm UTC
by Inessa Pawson May 3, 2024

May 3, 2024

The next NumPy Optimization Team meeting will be held this Monday, May 6th at 5 pm UTC. Join us via Zoom: https://numfocus-org.zoom.us/j/81261288210?pwd=iwV99tGSjR61RTGEERKM4QKxe46g… . Everyone is welcome and encouraged to attend. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/dVdSlQ0TThWkOk0OkmGsmw?both. For the notes from the previous meetings, visit: https://github.com/numpy/archive/tree/main/optim_team_meetings. -- Cheers, Inessa Inessa Pawson GitHub: inessapawson

1 0