Mailman 3 August 2022 - NumPy-Discussion

next NumPy community meeting
by Inessa Pawson 04 Jun '23

04 Jun '23

The next NumPy community meeting will be held this Wednesday, August 17th at 18:00 (6 pm) UTC. Join us via Zoom: https://us06web.zoom.us/j/83278611437?pwd=ekhoLzlHRjdWc0NOY2FQM0NPemdkZz09 Everyone is welcome and encouraged to attend. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both For the notes from the previous meeting, visit: https://github.com/numpy/archive/blob/main/community_meetings/community-202… Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy https://numpy.org/ Twitter: @inessapawson

1 21

Invalid value encoutered : how to prevent numpy.where to do this?
by Eric Emsellem 18 Feb '23

18 Feb '23

Dear all, I have a code using lots of "numpy.where" to make some constrained calculations as in: data = arange(10) result = np.where(data == 0, 0., 1./data) # or data1 = arange(10) data2 = arange(10)+1.0 result = np.where(data1 > data2, np.sqrt(data1-data2), np.sqrt(data2-data2)) which then produces warnings like: /usr/bin/ipython:1: RuntimeWarning: invalid value encountered in sqrt or for the first example: /usr/bin/ipython:1: RuntimeWarning: divide by zero encountered in divide How do I avoid these messages to appear? I know that I could in principle use numpy.seterr. However, I do NOT want to remove these warnings for other potential divide/multiply/sqrt etc errors. Only when I am using a "where", to in fact avoid such warnings! Note that the warnings only happen once, but since I am going to release that code, I would like to avoid the user to get such messages which are irrelevant here (because I am testing, with the where, when NOT to divide by zero or take a sqrt of a negative number). thanks! Eric

5 4

Documentation Team meeting - Monday June 8th
by Melissa Mendonça 04 Dec '22

04 Dec '22

Hi all! A reminder that on Monday, June 8, we have another documentation team meeting at 3PM UTC**. If you wish to join on Zoom, you need to use this link https://zoom.us/j/420005230 Here's the permanent hackmd document with the meeting notes: https://hackmd.io/oB_boakvRqKR-_2jRV-Qjg <https://www.google.com/url?q=https%3A%2F%2Fhackmd.io%2FoB_boakvRqKR-_2jRV-Q…> Hope to see you around (especially if you want to introduce yourself or discuss ideas for Google Season of Docs). ** You can click this link to get the correct time at your timezone: https://www.timeanddate.com/worldclock/fixedtime.html?msg=NumPy+Documentati… - Melissa

6 74

next NumPy triage meeting
by Inessa Pawson 27 Nov '22

27 Nov '22

The next NumPy triage meeting will be held this Wednesday, August 10th at 16:00 (4 pm) UTC. This is a meeting where we synchronously triage prioritized PRs and issues. Join us via Zoom: https://us06web.zoom.us/j/82096749952?pwd=MW9oUmtKQ1c3a2gydGk1RTdYUUVXZz09 Everyone is welcome to attend and contribute to a conversation. Please notify us of issues or PRs that you’d like to have reviewed by adding a GitHub link to them in the meeting agenda: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg. Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy email: inessa(a)albuscode.org

1 8

next NumPy Newcomers' Hour
by Inessa Pawson 02 Oct '22

02 Oct '22

Our next Newcomers' Hour will be held this Thursday, June 16th, at 4 pm UTC. Stop by to ask questions or just to say hi. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/3f3otyyuTte3FU9y3QzsLg?both Join the meeting via Zoom: https://us02web.zoom.us/j/87192457898 Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy https://numpy.org/ Twitter: @inessapawson

1 3

f2py compile error with full path of signature file on Windows
by Johann Rohwer 04 Sep '22

04 Sep '22

2 1

Need help extending a NumPy array in C
by lpsmith＠uw.edu 03 Sep '22

03 Sep '22

Hello! I am maintaining a C++ codebase with extensive ties to Python bindings (via SWIG). One of the features of the code is that we define (in C) a subclass of a NumPy Array. Everything worked until we started getting this message with numpy 1.23: RuntimeError: Object of type <class 'NamedArray'> appears to be C subclassed NumPy array, void scalar, or allocated in a non-standard way.NumPy reserves the right to change the size of these structures. Projects are required to take this into account by either recompiling against a specific NumPy version or padding the struct and enforcing a maximum NumPy version. My problem is that I don't know how do to either of those things. I would have assumed that whatever I compiled against, it would always be compiled against a specific NumPy version, and I also assumed that 'enforcing a maximum NumPy version' would happen in requirements.txt for the Python package, but that also seems to not be the case. Any hints? Thank you! -Lucian Smith

3 6

Exporting numpy arrays to binary JSON (BJData) for better portability
by Qianqian Fang 01 Sep '22

01 Sep '22

To avoid derailing the other thread <https://mail.python.org/archives/list/numpy-discussion@python.org/thread/A4…> on extending .npy files, I am going to start a new thread on alternative array storage file formats using binary JSON - in case there is such a need and interest among numpy users specifically, i want to first follow up with Bill's question below regarding loading time On 8/25/22 11:02, Bill Ross wrote: > > |Can you give load times for these?| > as I mentioned in the earlier reply to Robert, the most memory-efficient (i.e. fast loading, disk-mmap-able) but not necessarily disk-efficient (i.e. may result in the largest data file sizes) BJData construct to store an ND array using BJData's ND-array container. I have to admit that both jdata and bjdata modules have not been extensively optimized for speed. with the current implementation, here are the loading time for a larger diagonal matrix (eye(10000)) a BJData file storing a single eye(10000) array using the ND-array container can be downloaded from here <http://neurojson.org/wiki/upload/eye1e4_bjd_raw_ndsyntax.jdb.zip>(file size: 1MB with zip, if decompressed, it is ~800MB, as the npy file) - this file was generated from a matlab encoder, but can be loaded using Python (see below Re Robert). |800000128 eye1e4.npy|| ||800000014 eye1e4_bjd_raw_ndsyntax.jdb|| || 813721 eye1e4_bjd_zlib.jdb|| || 113067 eye1e4_bjd_lzma.jdb| the loading time (from an nvme drive, Ubuntu 18.04, python 3.6.9, numpy 1.19.5) for each file is listed below: |0.179s eye1e4.npy (mmap_mode=None)|| ||0.001s eye1e4.npy (mmap_mode=r)|| ||0.718s eye1e4_bjd_raw_ndsyntax.jdb|| ||1.474s eye1e4_bjd_zlib.jdb|| ||0.635s eye1e4_bjd_lzma.jdb| clearly, mmapped loading is the fastest option without a surprise; it is true that the raw bjdata file is about 5x slower than npy loading, but given the main chunk of the data are stored identically (as contiguous buffer), I suppose with some optimization of the decoder, the gap between the two can be substantially shortened. The longer loading time of zlib/lzma (and similarly saving times) reflects a trade-off between smaller file sizes and time for compression/decompression/disk-IO. by no means I am saying the binary JSON format is readily available to deliver better speed with its current non-optimized implementation. I just want to bright the attention to this class of formats, and highlight that it's flexibility gives abundant mechanisms to create fast, disk-mapped IO, while allowing additional benefits such as compression, unlimited metadata for future extensions etc. > |> 8000128 eye5chunk.npy|| > ||> 5004297 eye5chunk_bjd_raw.jdb|| > ||> 10338 eye5chunk_bjd_zlib.jdb|| > ||> 2206 eye5chunk_bjd_lzma.jdb| > > For my case, I'd be curious about the time to add one 1T-entries file > to another. as I mentioned in the previous reply, bjdata is appendable <https://github.com/NeuroJSON/bjdata/blob/master/images/BJData_Diagram.pdf>, so you can simply append another array (or a slice) to the end of the file. > Thanks, > Bill also related, Re @Robert's question below > Are any of them supported by a Python BJData implementation? I didn't > see any option to get that done in the `bjdata` package you > recommended, for example. > https://github.com/NeuroJSON/pybj/blob/a46355a0b0df0bec1817b04368a5a5733586… the bjdata module currently only support nd-array in the decoder <https://github.com/NeuroJSON/pybj/blob/a46355a0b0df0bec1817b04368a5a5733586…> (i.e. map such buffer to a numpy.ndarray) - should be relatively trivial to add it to the encoder though. on the other side, the annotated format is currently supported. one can call jdata module (responsible for annotation-level encoding/decoding) as shown in my sample code, then call bjdata internally for data serialization. > Okay. Given your wording, it looked like you were claiming that the > binary JSON was supported by the whole ecosystem. Rather, it seems > like you can either get binary encoding OR the ecosystem support, but > not both at the same time. all in relative terms of course - json has ~100 listed parsers on it's website <https://www.json.org/json-en.html>, MessagePack - another flavor of binary JSON - listed <https://msgpack.org/index.html> ~50/60 parsers, and UBJSON listed <https://ubjson.org/libraries/> ~20 parsers. I am not familiar with npy parsers, but googling it returns only a few. also, most binary JSON implementations provided tools to convert to JSON and back, so, in that sense, whatever JSON has in its ecosystem can be "potentially" used for binary JSON files if one wants to. There are also recent publications comparing differences between various binary JSON formats in case anyone is interested https://github.com/ubjson/universal-binary-json/issues/115

5 10

presentation topics for NumPy Newcomers’ Hour
by Inessa Pawson 29 Aug '22

29 Aug '22

Presentations by active NumPy contributors and core developers at NumPy Newcomers’ Hour have become very popular amongst our community members. To suggest a topic for future presentations, please leave a comment in this tracking issue: https://github.com/numpy/archive/issues/58. P.S. Visit the official NumPy YouTube channel to view video recordings of the previous events: https://www.youtube.com/c/NumPy_team. -- Cheers, Inessa Inessa Pawson Contributor Experience Lead | NumPy https://numpy.org/ GitHub: inessapawson

1 0

ANN: SciPy 1.9.1
by Tyler Reddy 27 Aug '22

27 Aug '22

Hi all, On behalf of the SciPy development team, I'm pleased to announce the release of SciPy 1.9.1, which is a bug fix release that includes some important meson build fixes. Sources and binary wheels can be found at: https://pypi.org/project/scipy/ and at: https://github.com/scipy/scipy/releases/tag/v1.9.1 One of a few ways to install this release with pip: pip install scipy==1.9.1 ===================== SciPy 1.9.1 Release Notes ===================== SciPy 1.9.1 is a bug-fix release with no new features compared to 1.9.0. Notably, some important meson build fixes are included. Authors ====== * Anirudh Dagar (1) * Ralf Gommers (12) * Matt Haberland (2) * Andrew Nelson (1) * Tyler Reddy (14) * Atsushi Sakai (1) * Eli Schwartz (1) * Warren Weckesser (2) A total of 8 people contributed to this release. People with a "+" by their names contributed a patch for the first time. This list of names is automatically generated, and may not be fully complete. Issues closed for 1.9.1 ------------------------------ * `#14517 <https://github.com/scipy/scipy/issues/14517>`__: scipy/linalg/tests/test_decomp.py::TestSchur::test_sort test... * `#16765 <https://github.com/scipy/scipy/issues/16765>`__: DOC: \`scipy.stats.skew\` no longer returns 0 on constant input * `#16787 <https://github.com/scipy/scipy/issues/16787>`__: BUG: Can't build 1.10 with mingw-w64 toolchain and numpy 1.21.6... * `#16813 <https://github.com/scipy/scipy/issues/16813>`__: BUG: scipy.interpolate interp1d extrapolate behaviour change... * `#16878 <https://github.com/scipy/scipy/issues/16878>`__: BUG: optimize.milp fails to execute when given exactly 3 constraints Pull requests for 1.9.1 ------------------------------ * `#16709 <https://github.com/scipy/scipy/pull/16709>`__: BLD: make the way we count commits for version numbering more... * `#16736 <https://github.com/scipy/scipy/pull/16736>`__: REL: prep for SciPy 1.9.1 * `#16749 <https://github.com/scipy/scipy/pull/16749>`__: BLD: install missing \`.pxd\` files, and update TODOs/FIXMEs... * `#16750 <https://github.com/scipy/scipy/pull/16750>`__: BLD: make OpenBLAS detection work with CMake * `#16755 <https://github.com/scipy/scipy/pull/16755>`__: TST: sparse.linalg: Loosen tolerance for the lobpcg test 'test_tolerance_float32' * `#16760 <https://github.com/scipy/scipy/pull/16760>`__: BLD: use a bit more idiomatic approach to constructing paths... * `#16768 <https://github.com/scipy/scipy/pull/16768>`__: DOC: stats.skew/kurtosis: returns NaN when input has only one... * `#16794 <https://github.com/scipy/scipy/pull/16794>`__: BLD/REL: on Windows use numpy 1.22.3 as the version to build... * `#16822 <https://github.com/scipy/scipy/pull/16822>`__: BUG/TST: linalg: Check the results of 'schur' more carefully. * `#16825 <https://github.com/scipy/scipy/pull/16825>`__: BUG: interpolate: fix "previous" and "next" extrapolate logic... * `#16862 <https://github.com/scipy/scipy/pull/16862>`__: BUG, DOC: Fix sphinx autosummary generation for \`odr\` and \`czt\` * `#16881 <https://github.com/scipy/scipy/pull/16881>`__: MAINT: optimize.milp: fix input validation when three constraints... * `#16901 <https://github.com/scipy/scipy/pull/16901>`__: MAINT: 1.9.1 backports * `#16904 <https://github.com/scipy/scipy/pull/16904>`__: BLD: update dependency ranges for meson-python and pybind11 for... Checksums ========= MD5 ~~~ 65d2b065c09d40b07272ae30332f6202 Changelog 69dd050c058c46e67d05956bf06e0566 README.txt 6108e7119fd6ae6d6338adede1904bb7 scipy-1.9.1-cp310-cp310-macosx_10_9_x86_64.whl 301053451a09b81268734133ce6d7bda scipy-1.9.1-cp310-cp310-macosx_12_0_arm64.whl 43a2fa03b951738d0deed6ddbc0a9f4d scipy-1.9.1-cp310-cp310-macosx_12_0_universal2.macosx_10_9_x86_64.whl 2339afc328d3c92cf85d1d86c5be4e19 scipy-1.9.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl d95584ddfe0703ba3b4624001a1f84f7 scipy-1.9.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 01937d372de92b3bcf9196b81bf8041d scipy-1.9.1-cp310-cp310-win_amd64.whl 3f85c3d508fced31f420ea31dccb1f83 scipy-1.9.1-cp38-cp38-macosx_10_9_x86_64.whl 3e07b61125901375f3ab99766465a428 scipy-1.9.1-cp38-cp38-macosx_12_0_arm64.whl 5cb2848a03030d3cec82a406b6d032de scipy-1.9.1-cp38-cp38-macosx_12_0_universal2.macosx_10_9_x86_64.whl cffdc85788b0d11e00f5c055e303538e scipy-1.9.1-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl ad0b411b353608b9fdd2fb3da25dd9ef scipy-1.9.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl fa1872fdcc63b57b95bffdcd2142a54e scipy-1.9.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 673905ce31524d16e5d44e600c48ff4d scipy-1.9.1-cp38-cp38-win32.whl bc421013525427036272cafa47c41f71 scipy-1.9.1-cp38-cp38-win_amd64.whl 19aa6604a3c39cbcc58f7a626e3fad89 scipy-1.9.1-cp39-cp39-macosx_10_9_x86_64.whl 0f6312acdf5edf5a6a5d2bf453e1c6cc scipy-1.9.1-cp39-cp39-macosx_12_0_arm64.whl 94ab8626a5d2aa5ed936f8571593567b scipy-1.9.1-cp39-cp39-macosx_12_0_universal2.macosx_10_9_x86_64.whl 4a9c22a4a059383cfb96cc2373156a30 scipy-1.9.1-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl 156f552987ff57cc347326b1df0a5916 scipy-1.9.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 8937a4cece2682585ceaf4115e526e21 scipy-1.9.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 9e1d2a5368f20c120da578a9f99b3986 scipy-1.9.1-cp39-cp39-win32.whl 3571be3d0e79cd46b0ab4c6418f6d44b scipy-1.9.1-cp39-cp39-win_amd64.whl SHA256 ~~~~~~ b6b1343d400fb775a7aaefe51654f56478a795318f11c29762a949a024cef9ca Changelog 1f18fbebfc36e5f6b3971b399fda124bd0c307308b407475d37059ba6a66be99 README.txt c61b4a91a702e8e04aeb0bfc40460e1f17a640977c04dda8757efb0199c75332 scipy-1.9.1-cp310-cp310-macosx_10_9_x86_64.whl d79da472015d0120ba9b357b28a99146cd6c17b9609403164b1a8ed149b4dfc8 scipy-1.9.1-cp310-cp310-macosx_12_0_arm64.whl 825951b88f56765aeb6e5e38ac9d7d47407cfaaeb008d40aa1b45a2d7ea2731e scipy-1.9.1-cp310-cp310-macosx_12_0_universal2.macosx_10_9_x86_64.whl f950a04b33e17b38ff561d5a0951caf3f5b47caa841edd772ffb7959f20a6af0 scipy-1.9.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 8cc81ac25659fec73599ccc52c989670e5ccd8974cf34bacd7b54a8d809aff1a scipy-1.9.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 8d3faa40ac16c6357aaf7ea50394ea6f1e8e99d75e927a51102b1943b311b4d9 scipy-1.9.1-cp310-cp310-win_amd64.whl 7a412c476a91b080e456229e413792bbb5d6202865dae963d1e6e28c2bb58691 scipy-1.9.1-cp38-cp38-macosx_10_9_x86_64.whl eb954f5aca4d26f468bbebcdc5448348eb287f7bea536c6306f62ea062f63d9a scipy-1.9.1-cp38-cp38-macosx_12_0_arm64.whl 3c6f5d1d4b9a5e4fe5e14f26ffc9444fc59473bbf8d45dc4a9a15283b7063a72 scipy-1.9.1-cp38-cp38-macosx_12_0_universal2.macosx_10_9_x86_64.whl bc4e2c77d4cd015d739e75e74ebbafed59ba8497a7ed0fd400231ed7683497c4 scipy-1.9.1-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl 0419485dbcd0ed78c0d5bf234c5dd63e86065b39b4d669e45810d42199d49521 scipy-1.9.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 34441dfbee5b002f9e15285014fd56e5e3372493c3e64ae297bae2c4b9659f5a scipy-1.9.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl b97b479f39c7e4aaf807efd0424dec74bbb379108f7d22cf09323086afcd312c scipy-1.9.1-cp38-cp38-win32.whl e8fe305d9d67a81255e06203454729405706907dccbdfcc330b7b3482a6c371d scipy-1.9.1-cp38-cp38-win_amd64.whl 39ab9240cd215a9349c85ab908dda6d732f7d3b4b192fa05780812495536acc4 scipy-1.9.1-cp39-cp39-macosx_10_9_x86_64.whl 71487c503e036740635f18324f62a11f283a632ace9d35933b2b0a04fd898c98 scipy-1.9.1-cp39-cp39-macosx_12_0_arm64.whl 3bc1ab68b9a096f368ba06c3a5e1d1d50957a86665fc929c4332d21355e7e8f4 scipy-1.9.1-cp39-cp39-macosx_12_0_universal2.macosx_10_9_x86_64.whl f7c39f7dbb57cce00c108d06d731f3b0e2a4d3a95c66d96bce697684876ce4d4 scipy-1.9.1-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl 47d1a95bd9d37302afcfe1b84c8011377c4f81e33649c5a5785db9ab827a6ade scipy-1.9.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 96d7cf7b25c9f23c59a766385f6370dab0659741699ecc7a451f9b94604938ce scipy-1.9.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 09412eb7fb60b8f00b328037fd814d25d261066ebc43a1e339cdce4f7502877e scipy-1.9.1-cp39-cp39-win32.whl 90c805f30c46cf60f1e76e947574f02954d25e3bb1e97aa8a07bc53aa31cf7d1 scipy-1.9.1-cp39-cp39-win_amd64.whl

1 0

NumPy-Discussion August 2022