Mailman 3 October 2024 - NumPy-Discussion

Canonical way of serialising Generators
by Andrew Nelson Oct. 27, 2024

Oct. 27, 2024

Hi all, is there a canonical way of serialising Generators (not via pickle). Would the following be reasonable for saving and restoring state: ``` def serialize_rng(rng): klass = rng.bit_generator.state['bit_generator'] entropy = rng.bit_generator.seed_seq.entropy return klass, entropy def deserialize_rng(klass, entropy, rng=None): if rng is not None and klass == rng.bit_generator.state['bit_generator']: rng.bit_generator.seed_seq.entropy = entropy return rng BG = getattr(np.random, klass) bg = BG(np.random.SeedSequence(entropy)) return np.random.default_rng(bg) ```

2 2

Endorsing SPECs 1, 6, 7, and 8
by Sebastian Berg Oct. 25, 2024

Oct. 25, 2024

Hi all, TL;DR: NumPy should endorse some or all of the new SPECs if we like them. If you don't or do like them, please discuss, otherwise I suspect we will propose and endorsing them soon and do it if a few core maintainers agree. --- The Scientific Python project has the SPEC process to write up helpful tips/patterns [1] with the idea that projects "endorse" them to indicate that we think following the SPEC is a good idea. One example probably known best is SPEC 0, which is NEP 27 (the suggested minimum supported versions for dependencies, e.g. Python/library versions). There has been a lot of work recently to write some new SPECs, and the following ones are up for endorsement by NumPy: * 1 -- Lazy Loading of Submodules and Functions https://scientific-python.org/specs/spec-0001 * 6 -- Keys to the Castle https://scientific-python.org/specs/spec-0006/ * 7 -- Seeding Pseudo-Random Number Generation https://scientific-python.org/specs/spec-0007/ * 8 -- Securing the Release Process https://scientific-python.org/specs/spec-0008/ Endorsing them means we think they are good guidance for projects, not necessarily strictly following them [2]. Without input, I suspect we may endorse them soon (if a few maintainers give a thumbs up). But of course it would be great to discuss and maybe improve the SPECs in the progress! SPEC 7 is important, because it describes how we would like other libraries to use the new random number generation API in the future (but actually doesn't apply directly). Cheers, Sebastian [1] The SPECs are not necessarily fixed. If you think they are missing a note/solution, we can modify them. [2] With it's flat namespace, I am not sure e.g. lazy-loading is of much use to NumPy (just like we have longer compatibility than SPEC 0).

8 9

Re: np.ndenumerate doesn't obey mask?
by Stefano Miccoli Oct. 23, 2024

Oct. 23, 2024

> On 22 Oct 2024, at 18:00, numpy-discussion-request(a)python.org wrote: > > From: Neal Becker <ndbecker2(a)gmail.com> > Subject: [Numpy-discussion] np.ndenumerate doesn't obey mask? > Date: 21 October 2024 at 18:52:41 CEST > To: Discussion of Numerical Python <numpy-discussion(a)python.org> > Reply-To: Discussion of Numerical Python <numpy-discussion(a)python.org> > > > I was using ndenuerate with a masked array, and it seems that the mask is ignored. Is this true? If so, isn't that a bug? > This is expected behaviour: you should use the function from the “.ma" namespace “numpy.ma.ndenumerate" <https://numpy.org/devdocs/reference/generated/numpy.ma.ndenumerate.html#num…> for skipping masked elements. By design numpy.enumerate does not skip masked elements. Stefano

1 0

np.ndenumerate doesn't obey mask?
by Neal Becker Oct. 21, 2024

Oct. 21, 2024

I was using ndenuerate with a masked array, and it seems that the mask is ignored. Is this true? If so, isn't that a bug?

1 0

What to do with np.matrix
by Dan Schult Oct. 20, 2024

Oct. 20, 2024

Hello everyone, I've been helping scipy.sparse provide a sparse array interface along-side sparse matrix. With that effort making good progress, we are starting to think about bigger implications. Now may be a good time to reconsider *what to do with np.matrix*. I think we have three options: 1) Leave np.matrix as is (with the docs discouraging its use) 2) Deprecate and then remove np.matrix 3) Split np.matrix out as a separate library `numpy-matrix` (maybe eventually with spmatrix) (Below, I've listed some arguments for each option from previous mailing list conversations.) I'm planning on writing a NEP to summarize the discussion that follows, before proceeding with any work. I'd appreciate your feedback on the feasibility of the above options, or ideas for other options to consider. Thank you! Dan =============================================== *From previous conversations (links below):* Reasons for *1) leave as is*: - Nothing needs to be done. - Use for np.matrix is declining anyway. - Old code can still use np.matrix without any changes. Reasons for *2) remove np.matrix*: - “The purpose here is to remove np.matrix from numpy so beginners will never see it.“ - Option 3) has little payoff if the “teaching advantages” of np.matrix are largely addressed by `@`. - Slightly less maintenance time after the removal. (But more to remove it.) Reasons for *3) numpy-matrix library*: - Allows/Forces old code to switch once to `npm.matrix` (better than multiple times, worse than never). - Old code can start to switch now… and will still work for any previous numpy version. - Devs still have to maintain the code. - Beginners do not see npm.matrix when starting with numpy. But still available. *Details and summary of history:* *Q*: Why now? *A*: We have made progress with handling np.matrix: - Python 2.7 is no longer supported - @ is now available - Docs now suggest that np.matrix should not be used for new code - Docs are largely free of np.matrix (but not all – subclass examples use np.matrix) - Scipy.sparse supports sparray (spmatrix is still there too) - Array API sets expectations for containers. - We have the history of numpy-financial as a previous split-off library. Previous discussion ideas not yet implemented: (Many others are already implemented) - “The purpose here is to remove np.matrix from numpy so beginners will never see it.“ - Easy for linear algebra, but later hard to extend to powerful ideas of ndarrays. - `@` should make ndarray easy enough for linear algebra. - 3 different semantics for ndarray, matrix, spmatrix. Matrix does evil by existing. - Teaching concerns: `np.matrix` has powerful (readable) notation for matrix mult, pow, and inverse. Most agree that `@` is all that’s needed. The `pow` and `inv` methods are OK. - Idea: Split out np.matrix as a library `numpy-matrix`-> npm.matrix - Make the new library look at numpy version and if old, use np.matrix code. If numpy is new, use npm.matrix code. (Allows people to switch their code while still supporting old numpy. Creates pointers to np.matrix for type checking, etc.) - Remove remaining examples of np.matrix from numpy docs. - `spmatrix` and `np.matrix` are linked. Sometimes `spmatrix` returns `np.matrix`. *Links:* - 2017 mailing list thread <https://mail.python.org/archives/list/numpy-discussion@python.org/thread/5F…> - 2014 mailing list thread <https://mail.python.org/archives/list/numpy-discussion@python.org/thread/FV…> - The mailing list in 2008 has multiple threads related to np.matrix. Most posts are not directly related to this question. But if you search the archives for "2008 np.matrix" you will find lots of discussion.

6 8

Community Review #4 of the NumPy Comics
by Mars Lee Oct. 14, 2024

Oct. 14, 2024

Hi all, The “How to Contribute to NumPy” comics are open for review again! Since the last review, I have added the last 6 pages, back cover and credits section. Here’s the issue link: https://github.com/numpy/numpy/issues/27375 Here’s the comic link: https://heyzine.com/flip-book/3e66a13901.html I have conducted some review sessions at the NumPy Community Call and NumPy Documentation call this and last month. The issue will be open for a few more days and will close on Thurs, Oct 17. Best, Mars

1 0

Expected behavior of np.array(..., copy=True)
by Kevin Sheppard Oct. 8, 2024

Oct. 8, 2024

Can anyone shed some light on the expected behavior of code using array(..., copy=True) with pandas objects? We ran into this in statsmodels and I think there are probably plenty of places where we explicitly call array(..., copy=True) and think we should have a totally independent copy of the data. One workaround is to use np.require(...,requirements="O") but it would help to understand the expected behavior. Here is a simple example: import numpy as np import pandas as pd weeks = 2 now = pd.to_datetime('2024-01-01') testdata = pd.DataFrame(columns=['dates', 'values']) rg = np.random.default_rng(0) testdata['dates'] = pd.date_range(start=now, periods=weeks * 7, freq='D') testdata['values']=rg.integers(0, 100, size=(weeks * 7)) values = testdata['values'] print("*"*10, " Before ", "*"*10) print(values.head()) arr = np.array(values, copy=True) arr.sort() print("*"*10, " After ", "*"*10) print(values.head()) print("*"*10, " Flags ", "*"*10) print(arr.flags) This produces ********** Before ********** 0 85 1 63 2 51 3 26 4 30 Name: values, dtype: int64 ********** After ********** 0 1 1 4 2 7 3 17 4 26 Name: values, dtype: int64 ********** Flags ********** C_CONTIGUOUS : True F_CONTIGUOUS : True OWNDATA : False WRITEABLE : True ALIGNED : True WRITEBACKIFCOPY : False Thanks, Kevin

2 1

NumPy 2.1.2 released.
by Charles R Harris Oct. 5, 2024

Oct. 5, 2024

Hi All, On behalf of the NumPy team, I'm pleased to announce the release of NumPy 2.1.2. NumPy 2.1.2 is a maintenance release that fixes bugs and regressions discovered after the 2.1.1 release. This release supports Python 3.10-3.13. Wheels can be downloaded from PyPI <https://pypi.org/project/numpy/2.1.2>; source archives, release notes, and wheel hashes are available on Github <https://github.com/numpy/numpy/releases/tag/v2.1.2>. *Contributors* A total of 11 people contributed to this release. People with a "+" by theirnames contributed a patch for the first time. - Charles Harris - Chris Sidebottom - Ishan Koradia + - João Eiras + - Katie Rust + - Marten van Kerkwijk - Matti Picus - Nathan Goldbaum - Peter Hawkins - Pieter Eendebak - Slava Gorloff + *Pull requests merged* A total of 14 pull requests were merged for this release. - #27333: MAINT: prepare 2.1.x for further development - #27400: BUG: apply critical sections around populating the dispatch cache - #27406: BUG: Stub out get_build_msvc_version if distutils.msvccompiler... - #27416: BUILD: fix missing include for std::ptrdiff_t for C++23 language... - #27433: BLD: pin setuptools to avoid breaking numpy.distutils - #27437: BUG: Allow unsigned shift argument for np.roll - #27439: BUG: Disable SVE VQSort - #27471: BUG: rfftn axis bug - #27479: BUG: Fix extra decref of PyArray_UInt8DType. - #27480: CI: use PyPI not scientific-python-nightly-wheels for CI doc... - #27481: MAINT: Check for SVE support on demand - #27484: BUG: initialize the promotion state to be weak - #27501: MAINT: Bump pypa/cibuildwheel from 2.20.0 to 2.21.2 - #27506: BUG: avoid segfault on bad arguments in ndarray.__array_function__ Cheers, Charles Harris

1 0

next NumPy Optimization Team meeting - June 24th, 2024 at 5 pm UTC
by Inessa Pawson Oct. 4, 2024

Oct. 4, 2024

The next NumPy Optimization Team meeting will be held this Monday, June 24th at 5 pm UTC. Join us via Zoom: https://numfocus-org.zoom.us/j/81261288210?pwd=iwV99tGSjR61RTGEERKM4QKxe46g… . Everyone is welcome and encouraged to attend. To add to the meeting agenda the topics you’d like to discuss, follow the link: https://hackmd.io/dVdSlQ0TThWkOk0OkmGsmw?both. For the notes from the previous meetings, visit: https://github.com/numpy/archive/tree/main/optim_team_meetings. -- Cheers, Inessa Inessa Pawson GitHub: inessapawson

1 5

help
by Usha Gayatri Oct. 2, 2024

Oct. 2, 2024

I am working on a Jupyter notebook in Anaconda Navigator. I have done some projects in 2021, 2022,2023 and 2024. When I run my old project which was created in 2021. it is giving errors.I am just testing import numpy as np import pandas as pd which is giving an error. File E:\anaconda3\Lib\site-packages\pandas\_libs\interval.pyx:1, in init pandas._libs.interval() ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject. I did uninstall numpy, pandas and again installed. even updated anaconda. Please help me. I am unable to run any program. Thank you. Usha

5 4